Something went wrong. Try again.
My own corner of monopam
Something went wrong. Try again.
80 kB · 1785 lines
at main
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099110011011102110311041105110611071108110911101111111211131114111511161117111811191120112111221123112411251126112711281129113011311132113311341135113611371138113911401141114211431144114511461147114811491150115111521153115411551156115711581159116011611162116311641165116611671168116911701171117211731174117511761177117811791180118111821183118411851186118711881189119011911192119311941195119611971198119912001201120212031204120512061207120812091210121112121213121412151216121712181219122012211222122312241225122612271228122912301231123212331234123512361237123812391240124112421243124412451246124712481249125012511252125312541255125612571258125912601261126212631264126512661267126812691270127112721273127412751276127712781279128012811282128312841285128612871288128912901291129212931294129512961297129812991300130113021303130413051306130713081309131013111312131313141315131613171318131913201321132213231324132513261327132813291330133113321333133413351336133713381339134013411342134313441345134613471348134913501351135213531354135513561357135813591360136113621363136413651366136713681369137013711372137313741375137613771378137913801381138213831384138513861387138813891390139113921393139413951396139713981399140014011402140314041405140614071408140914101411141214131414141514161417141814191420142114221423142414251426142714281429143014311432143314341435143614371438143914401441144214431444144514461447144814491450145114521453145414551456145714581459146014611462146314641465146614671468146914701471147214731474147514761477147814791480148114821483148414851486148714881489149014911492149314941495149614971498149915001501150215031504150515061507150815091510151115121513151415151516151715181519152015211522152315241525152615271528152915301531153215331534153515361537153815391540154115421543154415451546154715481549155015511552155315541555155615571558155915601561156215631564156515661567156815691570157115721573157415751576157715781579158015811582158315841585158615871588158915901591159215931594159515961597159815991600160116021603160416051606160716081609161016111612161316141615161616171618161916201621162216231624162516261627162816291630163116321633163416351636163716381639164016411642164316441645164616471648164916501651165216531654165516561657165816591660166116621663166416651666166716681669167016711672167316741675167616771678167916801681168216831684168516861687168816891690169116921693169416951696169716981699170017011702170317041705170617071708170917101711171217131714171517161717171817191720172117221723172417251726172717281729173017311732173317341735173617371738173917401741174217431744174517461747174817491750175117521753175417551756175717581759176017611762176317641765176617671768176917701771177217731774177517761777177817791780178117821783178417851786(*--------------------------------------------------------------------------- Copyright (c) 2025 Thomas Gazagnaire. All rights reserved. SPDX-License-Identifier: ISC
Menhir grammar for the SQLite SQL subset.
CREATE TABLE column bodies are collected as token lists and classified in Ast.classify_column — the grammar handles structure (parens, commas, table-vs-column dispatch), OCaml handles the type/constraint semantics. Queries (SELECT) get a real expression grammar with precedence. ---------------------------------------------------------------------------*)
%{ open Ast
(* A numeric token as its value, and as an expression. Both are {!Ast}'s: the sign a unary minus contributes has to be applied to the token's text and not to the value it would otherwise have had, so the reading and the negation are one pair and live where the tree that carries the unsigned text does. *)let number_to_value = Ast.number
(* Wrap an aggregate call [base] in its optional FILTER then OVER decorations. *)let decorate base filter window = let base = match filter with Some p -> Filter (base, p) | None -> base in match window with None -> base | Some w -> Window (base, w)
(* The call [name(DISTINCT args)], [over] saying whether an OVER follows it.
The grammar builds the keyword before an argument list of any length, as parse.y does ([expr ::= idj LP distinct exprlist RP sortorder]); which of three refusals a bad call earns is then decided here, by the name and the count alone, in the order sqlite3 decides it (resolveExprStep's TK_FUNCTION case, src/resolve.c).
The window refusal comes first and turns on neither the name nor the count: sqlite3WindowAttach (src/window.c) rejects DISTINCT as it attaches the window, before any name is looked up, exempting only the case where the "window" is a bare FILTER, which is an ordinary aggregate.
Then the name is looked up at the arity written. A call that resolves to no fold there is a scalar call carrying a DISTINCT that governs nothing, and it is built as one -- which is what [min(DISTINCT k, v)], the scalar [min] rather than the one-argument fold, is. Its own refusal, if the name resolves to nothing at all or to nothing at that arity, is {!Func.apply}'s, in the two sentences that rule out ever reaching the count rule below: "no such function: f" and "wrong number of arguments to function f()".
Only a call past both, resolved to an aggregate, reaches the rule lang_aggfunc.html states -- DISTINCT over a single argument -- which sqlite3 words as a sentence of its own rather than by declining to parse the call. *)let distinct_call ~over name args = let name = String.lowercase_ascii name in if over then Catalog.Error.fail_prepare "DISTINCT is not supported for window functions"; if not (Func.aggregate name ~nargs:(List.length args)) then Func (name, args) else match args with | [ e ] -> Distinct_agg (name, e) | _ -> Catalog.Error.fail_prepare "DISTINCT aggregates must have exactly one argument"
(* [e d NULLS FIRST|LAST]: NULLS is a soft keyword (a plain identifier elsewhere), matched contextually here. The default null ordering already matches sqlite3 (NULL sorts lowest, so ASC is nulls-first and DESC nulls-last); an explicit clause overrides it with a leading [e IS NULL] key. *)let nulls_order e d kw pos = if String.uppercase_ascii kw <> "NULLS" then Catalog.Error.fail_prepare "syntax error: expected NULLS in ORDER BY term"; match String.uppercase_ascii pos with | "FIRST" -> [ (Is_null e, Desc); (e, d) ] | "LAST" -> [ (Is_null e, Asc); (e, d) ] | _ -> Catalog.Error.fail_prepare "syntax error: expected NULLS FIRST or NULLS LAST"
(* A WITH list declares each name once. The whole list is one scope, so a repeated name would leave which body a reader gets to the order the binder folds in; sqlite3 refuses the statement instead ([sqlite3WithAdd], build.c), before either body is resolved, and reports the second occurrence's own spelling. Names fold, so two spellings of one name are a duplicate. *)let no_duplicate_cte (l : Ast.cte list) = let rec go seen = function | [] -> () | (c : Ast.cte) :: tl -> if List.exists (Ast.ident_equal c.name) seen then Catalog.Error.fail_prepare "duplicate WITH table name: %s" c.name; go (c.name :: seen) tl in go [] l
(* An ON CONFLICT target, absent meaning the clause named none: every unique constraint is then its own, and no predicate picks between them. *)(* [EXPLAIN stmt] and [EXPLAIN QUERY PLAN stmt]: the words are ordinary identifiers to the lexer, so the spelling is checked here. A leading word that is not EXPLAIN is a statement starting with a bare name, which is a syntax error at that word -- sqlite3's own wording for it. *)let is_word w text = String.uppercase_ascii w = text
(* A token an action refuses where the grammar had to admit it to see the rest of the shape. sqlite3's parser has no transition at that point and its message names the token, so this is the same refusal reached a step later. *)let near tok = Catalog.Error.fail_prepare "near %S: syntax error" tok
let explain_word id s = if not (is_word id "EXPLAIN") then Catalog.Error.fail_prepare "near %S: syntax error" id; s
let explain_words id q p s = if not (is_word id "EXPLAIN") then Catalog.Error.fail_prepare "near %S: syntax error" id; if not (is_word q "QUERY" && is_word p "PLAN") then Catalog.Error.fail_prepare "near %S: syntax error" q; s
(* parse.y's [transtype]: none is DEFERRED ([transtype ::= .], whose action is [A = TK_DEFERRED]), else one word of DEFERRED, IMMEDIATE and EXCLUSIVE. The three are %fallback identifiers there and stay identifiers to the lexer, as EXPLAIN does, so the spelling is checked in this action. *)let transaction_type = function | None -> Deferred | Some w when is_word w "DEFERRED" -> Deferred | Some w when is_word w "IMMEDIATE" -> Immediate | Some w when is_word w "EXCLUSIVE" -> Exclusive | Some w -> near w
let conflict_of = function | None -> ([], None) | Some (cols, where) -> (cols, where)
(* [a IS b] / [a IS NOT b]: NULL-safe (not-)distinct. A NULL right operand folds to the dedicated Is_null/Is_not_null nodes; otherwise it is the general NULL-safe equality node (IS NOT is its boolean complement). *)let mk_is a = function Lit Null -> Is_null a | b -> Is (a, b)let mk_is_not a = function | Lit Null -> Is_not_null a | b -> Unop_not (Is (a, b))
(* [a IS TRUE]/[IS FALSE] (and their NOT forms) test truthiness, not equality: sqlite3 defines [a IS TRUE] as [a<>0] and [a IS FALSE] as [a=0], each mapping a NULL [a] to 0 (false). Desugar to a CASE so [a] is evaluated once and the result is always 0/1, never NULL. *)(* Whether [e]'s value is settled before any row is read, which is what decides the [IN] rewrite below. sqlite3ExprIsConstant (src/expr.c) walks the expression and stops at a column, an aggregate, a window function and a subquery; a literal, a bound parameter, and any operator, CAST, CASE or [SQLITE_FUNC_CONSTANT] function over constants pass. So [lower('ABC')] and [datetime('now')] are constant -- 'now' is fixed for the statement -- while [random()] and [lower(v)] are not.
Whether a call folds a group is asked of the name and its argument count together, as sqlite3FindFunction asks it: [max(v)] is the aggregate and [max(a, b)] the scalar of the same name.
The descent is {!Ast.fold_expr}, not a match of its own, so a new expression constructor is answered where every other traversal answers it. *)let rec constant e = match e with | Col _ | Schema_col _ | Star | Qualified_star _ -> false | Scalar_select _ | Exists _ | In_select _ -> false | Distinct_agg _ | Filter _ | Ordered_agg _ | Window _ -> false | Func (n, args) when Func.aggregate n ~nargs:(List.length args) -> false | Func (n, _) when not (Func.deterministic n) -> false | e -> Ast.fold_expr ~select:(fun _ _ -> false) (fun acc x -> acc && constant x) true e
(* [x IN (list)]: a list whose sole element is a parenthesised subquery (so the source wrote [x IN ((SELECT ...))], any depth of extra parens collapsing to a scalar subquery) is the subquery membership test [x IN (SELECT ...)], matching every result row -- not a one-element value list.
A list whose sole element is constant is no membership test at all: sqlite3 rewrites [x IN (v)] to [x = +v] here, as it builds the node (sqlite/src/parse.y, the [expr in_op LP exprlist RP] rule). The unary plus is part of the rewrite -- it leaves the collating sequence alone and drops the affinity the element would otherwise state -- and the caller's [NOT] over the result is what [NOT IN] becomes.
It changes the answer because the two forms read a collating sequence by different rules: datatype3.html sec.7.1 gives [x IN (y, z, ...)] the sequence of [x] alone, so a COLLATE among the candidates names nothing, where an equality takes the left operand's and failing that the right's. *)let mk_in e = function | [ Scalar_select s ] -> In_select (e, s) | [ v ] when constant v -> Binary (Eq, e, Unary_plus v) | l -> In_list (e, l)
(* [SELECT * FROM <src>], the derived table a join operand is bound as. A table-valued function joins through this: the join operand is a table_ref, which names a table or wraps a subquery, so the function becomes a subquery over itself and scans like any other derived table. *)let select_star_from from = { ctes = []; distinct = false; cols = [ { expr = Star; alias = None } ]; from; joins = []; where = None; group_by = []; having = None; compound = []; order_by = []; limit = None; offset = None }
let zero = Lit (Int 0L)let one = Lit (Int 1L)
(* Unary minus: [0 - e], except over a numeric literal whose value the sign has yet to decide, where the minus goes on the token text ({!Ast.negate}). *)let negate = Ast.negate
let is_true a = Case (None, [ (Binary (Neq, a, zero), one) ], Some zero)let is_false a = Case (None, [ (Binary (Eq, a, zero), one) ], Some zero)let is_not_true a = Case (None, [ (Binary (Neq, a, zero), zero) ], Some one)let is_not_false a = Case (None, [ (Binary (Eq, a, zero), zero) ], Some one)
(* The options a table declares after its column list. [WITHOUT ROWID] and [STRICT] are the only ones sqlite3 knows, and it rejects any other word -- as we do, since a table option that parses and does nothing is a promise the table does not keep. *)type table_opts = { without_rowid : bool; strict : bool }
(* One option as written: the word following [WITHOUT], or a bare word. *)type table_opt = Without of string | Bare of string
let no_table_opts = { without_rowid = false; strict = false }
let add_table_opt opts = function | Without w when String.uppercase_ascii w = "ROWID" -> { opts with without_rowid = true } | Bare w when String.uppercase_ascii w = "STRICT" -> { opts with strict = true } | Without w | Bare w -> Catalog.Error.fail_prepare "unknown table option: %s" w
(* The [ON DELETE] / [ON UPDATE] actions a REFERENCES clause names, as the pair every foreign key carries. A kind the clause never names is NO ACTION (foreignkeys.html sec.4.3: "the default action is NO ACTION"), and a kind it names twice keeps the last, as sqlite3's fkey.c reads the clause left to right into one pair. Shared by the table-level FOREIGN KEY and the column-level REFERENCES, which is the same clause on the same tail. *)let fk_actions acts = let pick sel = List.fold_left (fun a x -> match sel x with Some v -> v | None -> a) No_action acts in (pick (function `Del d -> Some d | `Upd _ -> None), pick (function `Upd u -> Some u | `Del _ -> None))
(* Build a [column_decl] from a column's name and parsed body: one reading for a CREATE TABLE column and an ADD COLUMN alike. *)let make_column_decl cname (toks, col_checks, def, gen, acts) = (* The VIRTUAL / STORED keyword closing a generation clause arrives as an opaque column token, ahead of any constraint the column also declares. It names the kind and is no part of the declared type, so it leaves the stream the affinity is built from. *) let toks, col_generated = match gen with | None -> (toks, None) | Some expr -> let kind, toks = split_generated_kind toks in (toks, Some ({ kind; expr } : generated_column)) in let col_foreign_key = let on_delete, on_update = fk_actions acts in column_foreign_key ~on_delete ~on_update cname toks in { def = classify_column cname toks; col_checks; col_default = def; col_generated; col_foreign_key }
(* Build a [create_table] from a parsed column/constraint list (shared by the standalone CREATE TABLE parser and the statement form). *)let make_create_table ?(opts = no_table_opts) name if_not_exists defs = let mk_fk cols (parent, pcols, od, ou) = { cols = cols; parent = parent; parent_cols = pcols; on_delete = od; on_update = ou } in let columns, tcs, checks, defaults, gens, fks = List.fold_left (fun (cs, ts, ks, ds, gs, fs) -> function | `Col (c : column_decl) -> let cname = c.def.name in let ds = match c.col_default with Some e -> (cname, e) :: ds | None -> ds in let gs = match c.col_generated with | Some g -> (cname, g) :: gs | None -> gs in let fs = match c.col_foreign_key with Some fk -> fk :: fs | None -> fs in (c.def :: cs, ts, List.rev_append c.col_checks ks, ds, gs, fs) | `Tbl t -> (cs, t :: ts, ks, ds, gs, fs) | `Check k -> (cs, ts, k :: ks, ds, gs, fs) | `Fk (cols, r) -> (cs, ts, ks, ds, gs, mk_fk cols r :: fs)) ([], [], [], [], [], []) defs in { tbl_name = name; ct_if_not_exists = if_not_exists; columns = List.rev columns; table_constraints = List.rev tcs; checks = List.rev checks; defaults = List.rev defaults; generated = List.rev gens; foreign_keys = List.rev fks; ct_without_rowid = opts.without_rowid; ct_strict = opts.strict }%}
(* Payload tokens *)%token <string> IDENT%token <string> NUMBER%token <string> STRING
(* SQL keywords — all carry original text for case preservation *)%token <string> CREATE TABLE ALTER ADD COLUMN IF NOT EXISTS DROP VIEW RENAME%token <string> TRIGGER BEFORE AFTER INSTEAD FOR EACH ROW OF%token <string> CASCADE RESTRICT NO ACTION CROSS SAVEPOINT RELEASE TO%token <string> PRIMARY KEY UNIQUE NULL DEFAULT CHECK REFERENCES TRUE FALSE%token <string> COLLATE GENERATED ALWAYS AS AUTOINCREMENT%token <string> FOREIGN CONSTRAINT ON ASC DESC%token <string> SELECT DISTINCT FROM WHERE JOIN INNER LEFT OUTER ORDER GROUP BY LIMIT%token <string> NATURAL RIGHT FULL USING%token <string> HAVING OFFSET CASE WHEN THEN ELSE END RETURNING FILTER%token <string> OVER PARTITION ROWS RANGE PRECEDING FOLLOWING UNBOUNDED CURRENT%token <string> CONFLICT DO NOTHING%token <string> WITH RECURSIVE UNION ALL EXCEPT INTERSECT%token <string> AND OR IN IS LIKE GLOB BETWEEN CAST ESCAPE ISNULL NOTNULL%token <string> MATERIALIZED WITHOUT%token <string> INSERT INTO VALUES REPLACE DELETE UPDATE SET INDEX PRAGMA%token <string> BEGIN COMMIT ROLLBACK TRANSACTION VACUUM ANALYZE RAISE(* CURRENT_TIMESTAMP / CURRENT_DATE / CURRENT_TIME, folded into one token as tokenize.c folds them; the payload is which of the three was written. *)%token <string> CTIME_KW
(* Delimiters and operators *)%token LPAREN RPAREN COMMA SEMI DOT STAR QUESTION PLUS MINUS SLASH PERCENT%token EQ NEQ LT LE GT GE EOF CONCAT%token BITAND BITOR SHL SHR ARROW ARROW2%token <string> BLOB(* [?NNN]: the placeholder that names its own parameter, one-based as sqlite3 numbers them (lang_expr.html#varparam). The bare [?] is QUESTION. *)%token <int> VARIABLE(* Phantom token: precedence marker for an absent join constraint (never lexed). *)%token NO_CONSTRAINT
%start <Ast.create_table> create_table%start <Ast.stmt list> statements
(* Precedence: lowest first. NOT binds tighter than AND/OR but looser than the comparison operators; bitwise (& | << >>) binds tighter than comparison, additive tighter still, then multiplicative, matching SQLite. *)(* A bare join with no [ON]/[USING] reduces at the lowest precedence, so a following [ON] is shifted into the join constraint rather than ending the join (which would mis-bind [INSERT ... SELECT ... JOIN b ON CONFLICT]). *)%nonassoc NO_CONSTRAINT%nonassoc ON%left OR%left AND%right NOT%nonassoc EQ NEQ LT LE GT GE LIKE GLOB IN IS BETWEEN ISNULL NOTNULL(* ESCAPE binds tighter than LIKE so [a LIKE b ESCAPE c] shifts the ESCAPE rather than reducing [a LIKE b] first. *)%nonassoc ESCAPE%left BITAND BITOR SHL SHR%left PLUS MINUS%left STAR SLASH PERCENT%left CONCAT ARROW ARROW2%right UMINUS(* COLLATE is the highest-precedence operator (datatype3.html), binding tighter than any arithmetic or comparison operator. *)%left COLLATE
%%
(* ── Statements ──────────────────────────────────────────────── *)
statements: | EOF { [] } | s = statement; t = stmt_tail { s :: t } | SEMI; t = statements { t }
stmt_tail: | EOF { [] } | SEMI; t = statements { t }
(* [EXPLAIN [QUERY PLAN] stmt]. sqlite3 keeps all three words usable as identifiers -- [CREATE TABLE explain(query INT, plan INT)] parses there -- so they are matched here as the plain identifiers they lex as, contextually, the way NULLS is. The described statement is not itself an EXPLAIN: sqlite3 does not nest them either. *)statement: | id = IDENT; s = explainable { Explain { query_plan = false; explained = explain_word id s } } | id = IDENT; q = IDENT; p = IDENT; s = explainable { Explain { query_plan = true; explained = explain_words id q p s } } | s = explainable { s }
explainable: | s = select_stmt { Select s } | i = insert_stmt { Insert i } (* [WITH ctes INSERT ... SELECT ...]: the CTEs are in scope for the inserted query, so inject them into its embedded SELECT (a recursive CTE feeding an INSERT is the common shape). *) | l = with_clause; i = insert_stmt { Insert (match i.source with | Ast.Select s -> { i with source = Ast.Select { s with ctes = l } } | _ -> i) } | d = delete_stmt { Delete d } | u = update_stmt { Update u } (* [WITH ctes UPDATE/DELETE ...]: the CTEs are in scope for the statement's WHERE and SET subqueries. *) | l = with_clause; u = update_stmt { Update { u with ctes = l } } | l = with_clause; d = delete_stmt { Delete { d with ctes = l } } | s = create_table_statement { s } | c = create_index_stmt { Create_index c } | v = create_view_stmt { Create_view v } | tg = create_trigger_stmt { Create_trigger tg } | d = drop_stmt { Drop d } | a = alter_table_stmt { Alter_table a } | p = pragma_stmt { Pragma p } | BEGIN; t = ioption(IDENT); option(TRANSACTION) { Begin (transaction_type t) } | COMMIT; option(TRANSACTION) { Commit } | ROLLBACK; option(TRANSACTION); r = rollback_tail { r } | SAVEPOINT; name = ident { Savepoint name } | RELEASE; ioption(SAVEPOINT); name = ident { Release name } | VACUUM; option(ident) { Vacuum } | ANALYZE; ioption(analyze_target) { Analyze }
(* ANALYZE's optional argument: a schema, table, or index name, optionally schema-qualified. The name is dropped here: what a backend does with the statement is its own, and measuring the whole database where one table was named leaves no statistic wrong. *)analyze_target: | ident { () } | ident; DOT; ident { () }
(* Plain ROLLBACK, or ROLLBACK TO a savepoint. *)rollback_tail: | { Rollback } | TO; ioption(SAVEPOINT); name = ident { Rollback_to name }
(* ── INSERT ──────────────────────────────────────────────────── *)
(* The source is a query: a bare [VALUES (..), (..)] list or a [SELECT]. Both are [select_stmt] cores, so [INSERT ... VALUES] and [INSERT ... SELECT] share one production (the executor fast-paths a literal VALUES list). *)insert_stmt: | INSERT; c = insert_conflict; INTO; name = ident; cols = loption(insert_columns); s = select_stmt; up = ioption(upsert_clause); ret = loption(returning_clause) { { conflict = c; table = name; columns = cols; source = Select s; upsert = up; returning = ret } } | INSERT; c = insert_conflict; INTO; name = ident; DEFAULT; VALUES; ret = loption(returning_clause) { { conflict = c; table = name; columns = []; source = Default_values; upsert = None; returning = ret } } | REPLACE; INTO; name = ident; cols = loption(insert_columns); s = select_stmt; ret = loption(returning_clause) { { conflict = Replace; table = name; columns = cols; source = Select s; upsert = None; returning = ret } }
(* The [OR <action>] a statement may override its target's own conflict resolutions with. A bare INSERT or UPDATE overrides nothing and is [Unspecified], which is not the same as [Abort]: [INSERT OR ABORT] overrides every column's own rule, a bare INSERT lets each column's stand. REPLACE and ROLLBACK are keyword tokens; IGNORE/ABORT/FAIL are context-sensitive (lexed as IDENT, so they stay usable as identifiers elsewhere). *)insert_conflict: | c = ioption(or_conflict) { match c with Some c -> c | None -> Unspecified }
or_conflict: | OR; REPLACE { Replace } | OR; ROLLBACK { Rollback } | OR; id = IDENT { match String.uppercase_ascii id with | "IGNORE" -> Ignore | "ABORT" -> Abort | "FAIL" -> Fail | "REPLACE" -> Replace | "ROLLBACK" -> Rollback | _ -> Catalog.Error.fail_prepare "OR: unknown conflict action %s" id }
insert_columns: | LPAREN; cs = names; RPAREN { cs }
(* ON CONFLICT [(target cols) [WHERE p]] DO NOTHING | DO UPDATE SET ... [WHERE ...] *)upsert_clause: | ON; CONFLICT; t = ioption(conflict_target); DO; NOTHING { let target, target_where = conflict_of t in { target; target_where; action = Nothing } } | ON; CONFLICT; t = ioption(conflict_target); DO; UPDATE; SET; sets = separated_nonempty_list(COMMA, assignment); w = option(where_clause) { let target, target_where = conflict_of t in { target; target_where; action = Update (sets, w) } }
(* The target's own WHERE picks a partial UNIQUE index out of those keyed on the same columns; [DO] closes it, so it never reaches the DO UPDATE's WHERE. *)conflict_target: | LPAREN; cs = names; RPAREN; w = ioption(where_clause) { (cs, w) }
value_tuple: | LPAREN; es = separated_nonempty_list(COMMA, expr); RPAREN { es }
(* RETURNING expr, expr, ... (shared by INSERT / UPDATE / DELETE). *)returning_clause: | RETURNING; cs = separated_nonempty_list(COMMA, result_col) { cs }
(* ── DELETE ──────────────────────────────────────────────────── *)
delete_stmt: | DELETE; FROM; name = ident; alias = option(table_alias); where = option(where_clause); ret = loption(returning_clause) { { ctes = []; table = name; alias = alias; where = where; returning = ret } }
(* ── UPDATE ──────────────────────────────────────────────────── *)
update_stmt: | UPDATE; c = insert_conflict; name = ident; alias = ioption(update_alias); SET; sets = separated_nonempty_list(COMMA, assignment); f = ioption(from_part); where = option(where_clause); ret = loption(returning_clause) { { conflict = c; ctes = []; table = name; alias = alias; sets = sets; from = f; where = where; returning = ret } }
(* The name the row being rewritten is bound under for the whole statement, the SET expressions and the WHERE alike. Written with AS: sqlite3 takes the target as an xfullname (parse.y), which carries the keyword and takes no bare alias, so [UPDATE t q SET ..] is a syntax error at that word. *)update_alias: | AS; s = ident { s }
assignment: | col = column_name; EQ; e = expr { (col, e) }
(* ── CREATE INDEX ────────────────────────────────────────────── *)
create_index_stmt: | CREATE; u = boption(UNIQUE); INDEX; e = if_not_exists_b; name = ident; ON; table = ident; LPAREN; cs = separated_nonempty_list(COMMA, indexed_column); RPAREN; w = ioption(index_where) { { if_not_exists = e; unique = u; name = name; table = table; keys = cs; where = w } }
(* An indexed key: a plain column or an expression over the row (sqlite.org/expridx.html). For a column key both of the things that place its entries are kept -- the COLLATE wrapping the column names the sequence they are ordered by, the ASC/DESC the direction they run in. An expression key keeps its own direction and the expression, COLLATE and all: the sequence is read back out of the expression, the direction has nowhere else to go.
Only an unqualified name is a column key. sqlite3 keeps [t.a] an expression and then refuses the "." operator while resolving it (resolve.c, TK_DOT under NC_IdxExpr), so reading it as the column would accept a key sqlite3 does not. *)indexed_column: | e = expr; d = key_direction { match e with | Col (None, name) -> Column { column = name; collation = None; desc = d } | Collate (Col (None, name), c) -> Column { column = name; collation = Some c; desc = d } | _ -> Expression { expr = e; desc = d } }
(* [DESC] on a key column, or [ASC]/nothing for the default ascending. *)key_direction: | { false } | ASC { false } | DESC { true }
(* [WHERE p] makes a partial index: it holds an entry for a row only where [p] is true of that row. A schema qualifier inside it is dropped unlooked-at, as sqlite3 drops one here (resolve.c, lookupName), so a predicate naming a database that does not exist still resolves its column. *)index_where: | WHERE; e = expr { ignore_schema_qualifiers e }
(* ── CREATE VIEW ─────────────────────────────────────────────── *)
create_view_stmt: | CREATE; VIEW; e = if_not_exists_b; name = ident; cols = loption(view_columns); AS; s = select_stmt { { name = name; if_not_exists = e; columns = cols; select = s } }
view_columns: | LPAREN; cs = names; RPAREN { cs }
(* ── CREATE TRIGGER ──────────────────────────────────────────── *)
(* [CREATE TRIGGER name [BEFORE|AFTER|INSTEAD OF] event ON table [FOR EACH ROW] [WHEN expr] BEGIN stmt; ... END]. Each body statement is terminated by its own semicolon (including the last, before END). *)create_trigger_stmt: | CREATE; TRIGGER; e = if_not_exists_b; name = ident; timing = trigger_timing; ev = trigger_event; ON; tbl = ident; ioption(for_each_row); w = ioption(when_clause); BEGIN; body = trigger_body; END { { name = name; if_not_exists = e; timing = timing; event = ev; table = tbl; when_ = w; body = body } }
trigger_timing: | BEFORE { Before } | AFTER { After } | INSTEAD; OF { Instead_of } | { Before }
trigger_event: (* These constructor names are shared with [stmt]; the [stmt] arm wins as the last definition, so a [trigger_event] use needs the cast. *) | INSERT { (Insert : trigger_event) } | DELETE { (Delete : trigger_event) } | UPDATE { (Update [] : trigger_event) } | UPDATE; OF; cs = names { (Update cs : trigger_event) }
for_each_row: | FOR; EACH; ROW { () }
when_clause: | WHEN; e = expr { e }
trigger_body: | { [] } | s = trigger_action; SEMI; rest = trigger_body { s :: rest }
trigger_action: | s = select_stmt { Select s } | i = insert_stmt { Insert i } | u = update_stmt { Update u } | d = delete_stmt { Delete d }
(* ── DROP TABLE / INDEX / VIEW / TRIGGER ─────────────────────── *)
drop_stmt: | DROP; TABLE; e = if_exists_b; name = ident { { object_ = Table; name = name; if_exists = e } } | DROP; INDEX; e = if_exists_b; name = ident { { object_ = Index; name = name; if_exists = e } } | DROP; VIEW; e = if_exists_b; name = ident { { object_ = View; name = name; if_exists = e } } | DROP; TRIGGER; e = if_exists_b; name = ident { { object_ = Trigger; name = name; if_exists = e } }
if_exists_b: | IF; EXISTS { true } | { false }
(* ── ALTER TABLE ADD COLUMN ──────────────────────────────────── *)
(* [ADD [COLUMN] column]: the {!column_decl} a CREATE TABLE column is, as sqlite3 spells the command [cmd ::= ALTER TABLE add_column_fullname ADD kwcolumn_opt columnname carglist] (parse.y), [carglist] being that same list of column constraints. Which of them a table accepts is the engine's question. The offsets delimit the declaration's text, which the stored schema carries as written. *)alter_table_stmt: | ALTER; TABLE; name = ident; ADD; ioption(COLUMN); c = column_decl { { table = name; op = Add_column { column = c; start = $startofs(c); stop = $endofs(c) } } } | ALTER; TABLE; name = ident; RENAME; TO; nn = ident { { table = name; op = Rename_table nn } } | ALTER; TABLE; name = ident; RENAME; ioption(COLUMN); c = ident; TO; nn = ident { { table = name; op = Rename_column (c, nn) } } | ALTER; TABLE; name = ident; DROP; ioption(COLUMN); c = ident { { table = name; op = Drop_column c } }
if_not_exists_b: | IF; NOT; EXISTS { true } | { false }
(* ── PRAGMA ──────────────────────────────────────────────────── *)
pragma_stmt: | PRAGMA; name = ident { { name = name; value = None } } | PRAGMA; name = ident; EQ; v = pragma_value { { name = name; value = Some v } } | PRAGMA; name = ident; LPAREN; v = pragma_value; RPAREN { { name = name; value = Some v } }
(* A PRAGMA value is parse.y's [nmnum]: a signed number, an [nm], or one of ON, DELETE and DEFAULT -- three reserved words listed there by hand, because the pragmas need them ([foreign_keys=on], [journal_mode=delete], [temp_store=default]). An [nm] is any {!fallback_ident}, which is what lets [synchronous=full] and [locking_mode=exclusive] be written the way the documentation writes them.
COLUMN and UNBOUNDED are in: nothing here reads either as its keyword, so both spell a value as they do in sqlite3. The text is kept as written; the pragmas compare it without regard to case, as sqlite3 does.
The number is spelled out here rather than reached through {!literal}, because the two are different sets in both directions. [nmnum] carries a sign ([cmd ::= PRAGMA nm dbnm EQ minus_num] and [nmnum ::= plus_num]) and {!literal} carries none, while {!literal} carries NULL and a blob and [nmnum] carries neither: [PRAGMA journal_mode=NULL] and [PRAGMA journal_mode=x'00'] are both syntax errors in sqlite3.
The sign belongs to the number and not to the position, so [PRAGMA journal_mode=-wal] stays a syntax error. It is read by {!Ast.number}, the reading the DEFAULT clause already takes, so that [-9223372036854775808] stays the INTEGER min_int rather than becoming a negated REAL. *)pragma_value: | n = NUMBER { number_to_value n } | MINUS; n = NUMBER { Ast.number ~negative:true n } | PLUS; n = NUMBER { number_to_value n } | s = STRING { Text s } | s = IDENT { Text s } | s = fallback_ident { Text s } | s = CTIME_KW { Text s } | s = COLUMN { Text s } | s = UNBOUNDED { Text s } | s = ON { Text s } | s = DELETE { Text s } | s = DEFAULT { Text s }
(* ── SELECT ──────────────────────────────────────────────────── *)
(* A query: one core, then [UNION]/[EXCEPT]/[INTERSECT] arms, then the compound-wide ORDER BY / LIMIT, optionally prefixed by WITH. The arms and the trailing clauses are folded onto the first core's record. *)select_stmt: | ctes = loption(with_clause); first = select_core; arms = list(compound_arm); order_by = loption(order_clause); lim = option(limit_clause) { let limit, offset = match lim with None -> (None, None) | Some (l, o) -> (Some l, o) in { first with ctes; compound = arms; order_by; limit; offset } }
select_core: | SELECT; d = boption(DISTINCT); cols = separated_nonempty_list(COMMA, result_col); src = option(from_part); where = option(where_clause); group_by = loption(group_clause); having = option(having_clause) { let from, joins = match src with Some (f, js) -> (f, js) | None -> (Unit, []) in { ctes = []; distinct = d; cols; from; joins; where; group_by; having; compound = []; order_by = []; limit = None; offset = None } } (* A bare VALUES list is a core: rows over columns column1..columnN. *) | VALUES; rows = separated_nonempty_list(COMMA, value_tuple) { { ctes = []; distinct = false; cols = [ { expr = Star; alias = None } ]; from = Values rows; joins = []; where = None; group_by = []; having = None; compound = []; order_by = []; limit = None; offset = None } }
from_part: | FROM; from = from_clause; joins = list(join_item) { (from, joins) }
having_clause: | HAVING; e = expr { e }
compound_arm: | UNION; ALL; c = select_core { (Union_all, c) } | UNION; c = select_core { (Union, c) } | EXCEPT; c = select_core { (Except, c) } | INTERSECT; c = select_core { (Intersect, c) }
(* WITH [RECURSIVE] cte, cte, ... (RECURSIVE is accepted; a self-referential body is handled by the recursive runner.) *)with_clause: | WITH; option(RECURSIVE); l = separated_nonempty_list(COMMA, cte_def) { no_duplicate_cte l; l }
cte_def: | name = ident; cols = loption(cte_columns); AS; m = ioption(materialized); LPAREN; s = select_stmt; RPAREN { { Ast.name; columns = cols; query = s; materialized = Option.value m ~default:Ast.No_hint } }
(* An [AS [NOT] MATERIALIZED] hint on a CTE. It changes no result, only whether the body may be spliced into the query that reads it. *)materialized: | MATERIALIZED { Ast.Materialized } | NOT; MATERIALIZED { Ast.Not_materialized }
cte_columns: | LPAREN; cs = names; RPAREN { cs }
group_clause: | GROUP; BY; l = separated_nonempty_list(COMMA, expr) { l }
from_clause: (* [Table]/[Select]/[Values] are shared with other Ast types whose definitions come later and so win by default; cast to pick the [from_clause] arm. *) | t = table_ref { (Table t : from_clause) } | LPAREN; s = select_stmt; RPAREN; a = option(table_alias) { (Select { fs_select = s; fs_alias = a } : from_clause) } | name = IDENT; LPAREN; args = separated_list(COMMA, expr); RPAREN; a = option(table_alias) { (Table_function { tf_name = String.lowercase_ascii name; tf_args = args; tf_alias = a } : from_clause) }
result_col: | STAR { { expr = Star; alias = None } } | q = IDENT; DOT; STAR { { expr = Qualified_star q; alias = None } } | e = expr; a = option(col_alias) { { expr = e; alias = a } }
(* ASC and DESC are among the identifiers parse.y lists as %fallback, so each names a result column in either spelling, with the AS and without it. *)col_alias: | AS; s = IDENT { s } | AS; s = STRING { s } (* a quoted string is accepted as a column alias *) | AS; s = ASC { s } | AS; s = DESC { s } | AS; s = CTIME_KW { s } | s = IDENT { s } | s = ASC { s } | s = DESC { s } | s = CTIME_KW { s }
(* A named table in a FROM clause, with its optional alias. The name is an {!ident}: parse.y reads the FROM list's tables as [nm] too ([seltablist ::= stl_prefix nm dbnm ...]), so a table called [row] is scanned under the name it was created under. The alias stays narrower, as parse.y's [as] is. *)table_ref: | name = ident; a = option(table_alias) { { name = name; alias = a; subquery = None } }
table_alias: | AS; s = IDENT { s } | s = IDENT { s }
(* A join operand: a named table, a table-valued function, or a parenthesised subquery bound under its alias (an unaliased one gets a unique synthetic name from its source offset, never referenced by the user). A function with no alias binds under its own name, as it does in the leading FROM, so [JOIN pragma_table_info('t')] is qualifiable as [pragma_table_info.name]. *)join_source: | t = table_ref { t } | name = IDENT; LPAREN; args = separated_list(COMMA, expr); RPAREN; a = option(table_alias) { let tf_name = String.lowercase_ascii name in let tf = Table_function { tf_name; tf_args = args; tf_alias = None } in { name = (match a with Some a -> a | None -> tf_name); alias = a; subquery = Some (select_star_from tf) } } | LPAREN; s = select_stmt; RPAREN; a = table_alias { { name = a; alias = Some a; subquery = Some s } } | LPAREN; s = select_stmt; RPAREN { let nm = Printf.sprintf "__sub%d" $startofs in { name = nm; alias = None; subquery = Some s } }
(* An extra FROM table: a join (with an optional ON/USING constraint -- an unconstrained [A JOIN B] is allowed, like sqlite3), or a Cartesian product -- [CROSS JOIN] or a comma join. *)join_item: | k = join_kw; t = join_source; c = join_constraint { let on, using = c in { table = t; on; type_ = k; natural = false; using } } | NATURAL; k = join_kw; t = join_source { { table = t; on = Lit (Int 1L); type_ = k; natural = true; using = [] } } | CROSS; JOIN; t = join_source { { table = t; on = Lit (Int 1L); type_ = Inner; natural = false; using = [] } } | COMMA; t = join_source { { table = t; on = Lit (Int 1L); type_ = Inner; natural = false; using = [] } }
(* The optional join constraint: [ON expr], [USING (cols)], or nothing (an unconstrained join, equivalent to ON true). *)join_constraint: | ON; e = expr { (e, []) } | USING; LPAREN; cs = names; RPAREN { (Lit (Int 1L), cs) } | %prec NO_CONSTRAINT { (Lit (Int 1L), []) }
join_kw: | JOIN { Inner } | INNER; JOIN { Inner } | LEFT; JOIN { Left } | LEFT; OUTER; JOIN { Left } | RIGHT; JOIN { Right } | RIGHT; OUTER; JOIN { Right } | FULL; JOIN { Full } | FULL; OUTER; JOIN { Full }
where_clause: | WHERE; e = expr { e }
order_clause: | ORDER; BY; l = separated_nonempty_list(COMMA, order_term) { List.concat l }
order_term: | e = expr; d = dir { [ (e, d) ] } | e = expr; d = dir; kw = IDENT; pos = IDENT { nulls_order e d kw pos }
dir: | ASC { Asc } | DESC { Desc } | { Asc }
(* Returns (limit, offset option). Note the comma form [LIMIT off, cnt] puts the OFFSET first -- the reverse of the [OFFSET] keyword form. *)limit_clause: | LIMIT; n = limit_value { (n, None) } | LIMIT; n = limit_value; OFFSET; o = limit_value { (n, Some o) } | LIMIT; o = limit_value; COMMA; n = limit_value { (n, Some o) }
limit_value: | n = NUMBER { Lit (number_to_value n) } | MINUS; n = NUMBER { Lit (Ast.number ~negative:true n) } | QUESTION { Param unnumbered } | n = VARIABLE { Param (n - 1) }
(* ── Expressions ─────────────────────────────────────────────── *)
expr: | e1 = expr; OR; e2 = expr { Binary (Or, e1, e2) } | e1 = expr; AND; e2 = expr { Binary (And, e1, e2) } | NOT; e = expr { Unop_not e } | e1 = expr; EQ; e2 = expr { Binary (Eq, e1, e2) } | e1 = expr; NEQ; e2 = expr { Binary (Neq, e1, e2) } | e1 = expr; LT; e2 = expr { Binary (Lt, e1, e2) } | e1 = expr; LE; e2 = expr { Binary (Le, e1, e2) } | e1 = expr; GT; e2 = expr { Binary (Gt, e1, e2) } | e1 = expr; GE; e2 = expr { Binary (Ge, e1, e2) } | e1 = expr; LIKE; e2 = expr { Binary (Like, e1, e2) } | e1 = expr; LIKE; e2 = expr; ESCAPE; e3 = expr { Func ("like", [ e2; e1; e3 ]) } | e1 = expr; NOT; LIKE; e2 = expr %prec LIKE { Unop_not (Binary (Like, e1, e2)) } | e1 = expr; NOT; LIKE; e2 = expr; ESCAPE; e3 = expr { Unop_not (Func ("like", [ e2; e1; e3 ])) } | e1 = expr; GLOB; e2 = expr { Binary (Glob, e1, e2) } | e1 = expr; NOT; GLOB; e2 = expr %prec GLOB { Unop_not (Binary (Glob, e1, e2)) } | e1 = expr; PLUS; e2 = expr { Binary (Add, e1, e2) } | e1 = expr; MINUS; e2 = expr { Binary (Sub, e1, e2) } | e1 = expr; STAR; e2 = expr { Binary (Mul, e1, e2) } | e1 = expr; SLASH; e2 = expr { Binary (Div, e1, e2) } | e1 = expr; PERCENT; e2 = expr { Binary (Mod, e1, e2) } | e1 = expr; BITAND; e2 = expr { Binary (Bit_and, e1, e2) } | e1 = expr; BITOR; e2 = expr { Binary (Bit_or, e1, e2) } | e1 = expr; SHL; e2 = expr { Binary (Shl, e1, e2) } | e1 = expr; SHR; e2 = expr { Binary (Shr, e1, e2) } | e1 = expr; CONCAT; e2 = expr { Binary (Concat, e1, e2) } | e1 = expr; ARROW; e2 = expr { Binary (Json_get, e1, e2) } | e1 = expr; ARROW2; e2 = expr { Binary (Json_get_text, e1, e2) } (* [e COLLATE name]: highest precedence, so it binds to the nearest operand. The name is kept as written -- sqlite3 stores the spelling and folds case only when it resolves one, and the pragmas print it back. *) | e = expr; COLLATE; c = IDENT { Collate (e, c) } (* Unary [-e] is [0 - e], except over a numeric literal whose value the sign has yet to decide ({!Ast.negate}). *) | MINUS; e = expr %prec UMINUS { negate e } (* Unary [+] is a value no-op (lang_expr.html) but not an AST no-op: it keeps the operand's collating sequence and drops its affinity, which is the exact opposite of what [COLLATE BINARY] does, so it gets its own node. *) | PLUS; e = expr %prec UMINUS { Unary_plus e } (* [x BETWEEN lo AND hi] desugars to [x >= lo AND x <= hi]; bounds are atoms (optionally with a trailing COLLATE) so the inner AND can't be mistaken for a boolean conjunction. *) | e = expr; BETWEEN; lo = between_bound; AND; hi = between_bound { Binary (And, Binary (Ge, e, lo), Binary (Le, e, hi)) } | e = expr; NOT; BETWEEN; lo = between_bound; AND; hi = between_bound { Unop_not (Binary (And, Binary (Ge, e, lo), Binary (Le, e, hi))) } (* The right operand is an [atom] (not a full [expr]): an atom cannot begin with NOT, so [IS NOT] is unambiguous, and atoms bind tighter than IS so there is no associativity conflict. A complex RHS is parenthesised. This covers [IS NULL], [IS NOT NULL], [col IS col], [x IS <literal>]. *) (* [IS TRUE]/[IS FALSE] test truthiness, distinct from [IS 1]/[IS 0] equality, so they take the keyword tokens; the general RHS is a [nonbool_atom], which excludes the bare TRUE/FALSE literals to keep these unambiguous. *) | e1 = expr; IS; TRUE { is_true e1 } | e1 = expr; IS; FALSE { is_false e1 } | e1 = expr; IS; NOT; TRUE { is_not_true e1 } | e1 = expr; IS; NOT; FALSE { is_not_false e1 } | e1 = expr; IS; e2 = is_rhs { mk_is e1 e2 } | e1 = expr; IS; NOT; e2 = is_rhs { mk_is_not e1 e2 } (* [x ISNULL] / [x NOTNULL]: the one-word postfix synonyms for IS [NOT] NULL. *) | e = expr; ISNULL { Is_null e } | e = expr; NOTNULL { Is_not_null e } | e = expr; IN; LPAREN; l = separated_nonempty_list(COMMA, expr); RPAREN { mk_in e l } | e = expr; IN; LPAREN; RPAREN { In_list (e, []) } | e = expr; IN; LPAREN; s = select_stmt; RPAREN { In_select (e, s) } | e = expr; NOT; IN; LPAREN; l = separated_nonempty_list(COMMA, expr); RPAREN { Unop_not (mk_in e l) } | e = expr; NOT; IN; LPAREN; RPAREN { Unop_not (In_list (e, [])) } | e = expr; NOT; IN; LPAREN; s = select_stmt; RPAREN { Unop_not (In_select (e, s)) } | a = atom { a }
(* The right operand of IS: an atom, optionally with a trailing COLLATE, and optionally signed. [IS] is a comparison and reads a sequence as [=] does, and COLLATE binds tighter than both, so the wrapper belongs to the operand and not to the [IS] -- which has no value for one. [%prec IS] (below COLLATE) is what settles that.
sqlite3 takes a whole expression on either side of [IS] (parse.y routes it through the same rule as [=]), so a sign is nothing special there; here the operand is narrowed to an atom to keep [IS NOT] unambiguous, and the sign has to be spelled out or [c IS -4713] -- which every date column comparison writes -- is a syntax error. A sign applied to an operand is the operator it is elsewhere, so [-] leaves an expression with no affinity and [+] leaves the collating sequence alone, as {!Ast.Unary_plus} does. *)is_rhs: | a = nonbool_atom %prec IS { a } | a = nonbool_atom; COLLATE; c = IDENT { Collate (a, c) } | MINUS; a = is_rhs { negate a } | PLUS; a = is_rhs { Unary_plus a }
(* A BETWEEN bound: an atom, optionally with a trailing COLLATE (which binds tighter than the comparison the bound feeds into), and optionally signed for the same reason an IS operand is -- [x BETWEEN -1 AND 1] is a whole expression per bound in sqlite3. *)between_bound: (* [%prec BETWEEN] (below COLLATE) makes a trailing COLLATE bind to the bound, [b COLLATE c], rather than to the whole BETWEEN expression. *) | a = atom %prec BETWEEN { a } | a = atom; COLLATE; c = IDENT { Collate (a, c) } | MINUS; b = between_bound { negate b } | PLUS; b = between_bound { Unary_plus b }
(* [atom] is a [nonbool_atom] plus the bare TRUE/FALSE literals. The IS rules use [nonbool_atom] so [IS TRUE]/[IS FALSE] (truthiness) stay distinct from [IS <literal>] (equality) without a grammar conflict; every other context uses [atom], where TRUE/FALSE are the integers 1/0 (lang_expr.html). *)atom: | a = nonbool_atom { a } | TRUE { Lit (Int 1L) } | FALSE { Lit (Int 0L) }
nonbool_atom: | LPAREN; e = expr; RPAREN { e } (* A parenthesised subquery used as a value: SELECT/WITH/VALUES after LPAREN are not in [expr]'s first set, so this never conflicts with [( expr )]. *) | LPAREN; s = select_stmt; RPAREN { Scalar_select s } | EXISTS; LPAREN; s = select_stmt; RPAREN { Exists s } | l = literal_expr { l } (* A bare [?] takes its number from the statement ({!Ast.unnumbered} until {!Param_number} hands one out); [?NNN] spells the one it reads, one-based in the text against the zero-based index the AST carries. *) | QUESTION { Param unnumbered } | n = VARIABLE { Param (n - 1) } | CAST; LPAREN; e = expr; AS; ty = IDENT; RPAREN { Cast (e, ty) } (* CASE [base] WHEN c THEN r ... [ELSE e] END. The optional base before the first WHEN gives the simple form; without it, the searched form. *) | CASE; base = ioption(expr); branches = nonempty_list(case_when); els = ioption(case_else); END { Case (base, branches, els) } | name = IDENT; LPAREN; DISTINCT; args = distinct_args; RPAREN; f = ioption(filter_clause); w = ioption(window_clause) { decorate (distinct_call ~over:(w <> None) name args) f w } | name = IDENT; LPAREN; args = func_args; RPAREN; f = ioption(filter_clause); w = ioption(window_clause) { decorate (Func (String.lowercase_ascii name, args)) f w } (* An in-call ORDER BY sorts the rows an aggregate is stepped with. parse.y puts it on the one call production ([expr ::= idj LP distinct exprlist RP sortorder]), so DISTINCT and it are alternatives to nothing rather than to each other and a call may carry both. No OVER follows either: an in-call ORDER BY belongs to the fold, and a window has its own. *) | name = IDENT; LPAREN; args = func_args; ob = order_clause; RPAREN; f = ioption(filter_clause) { let a = Ordered_agg (Func (String.lowercase_ascii name, args), ob) in match f with Some p -> Filter (a, p) | None -> a } | name = IDENT; LPAREN; DISTINCT; args = distinct_args; ob = order_clause; RPAREN; f = ioption(filter_clause) { let a = Ordered_agg (distinct_call ~over:false name args, ob) in match f with Some p -> Filter (a, p) | None -> a } (* [replace] is also a function, but REPLACE is a keyword token (INSERT OR REPLACE), so it does not arrive as IDENT. *) | REPLACE; LPAREN; args = func_args; RPAREN { Func ("replace", args) } (* [like]/[glob] are also functions, but LIKE/GLOB are keyword tokens (the binary operators), so they do not arrive as IDENT. *) | LIKE; LPAREN; args = func_args; RPAREN { Func ("like", args) } | GLOB; LPAREN; args = func_args; RPAREN { Func ("glob", args) } (* [RAISE ( IGNORE )] and [RAISE ( ABORT|ROLLBACK|FAIL , message )], the two productions parse.y gives it and the only two. It is not a call of a function named "raise": the action is a keyword, not an argument, and nothing else fits between the parentheses. *) | RAISE; LPAREN; r = raise_arg; RPAREN { let action, message = r in Raise (action, message) } (* Most of SQLite's keywords are non-reserved: parse.y ends with a %fallback list of words the parser retries as an identifier wherever it cannot shift them as a keyword, and its idj token class (ID | INDEXED | JOIN_KW) puts the join words in identifier position outright. So each is an ordinary column name outside the clause that gives it meaning, and a key's direction still parses: [ORDER BY desc DESC] reads the first as the column and the second as the direction, since no expression continues with a bare name.
UNBOUNDED is the one word on the list left reserved. Read as a column it makes [ROWS UNBOUNDED PRECEDING] a shift/reduce conflict against [ROWS <expr> PRECEDING] -- the one place a bare word and a frame bound begin alike -- and lemon resolves that silently where menhir would not. *) | q = IDENT; DOT; name = column_ident { Col (Some q, name) } (* [schema.table.column]. This engine has one schema -- there is no ATTACH and no temp database -- so a qualifier that folds to [main] says nothing and comes off, and any other names a schema holding nothing. Keeping it is what lets the refusal spell the name that was written; sqlite3 looks the qualifier up among the attached databases and answers "no such column: sch.t.c" the same way. Where sqlite3 ignores it instead -- a CHECK constraint, a partial index predicate -- the grammar drops it below, at those productions. *) | s = IDENT; DOT; q = IDENT; DOT; name = column_ident { if Catalog.Ident.equal s "main" then Col (Some q, name) else Schema_col (s, q, name) } | name = IDENT { Col (None, name) } | name = KEY { Col (None, name) } | name = ASC { Col (None, name) } | name = DESC { Col (None, name) } | name = ACTION { Col (None, name) } | name = AFTER { Col (None, name) } | name = ALWAYS { Col (None, name) } | name = ANALYZE { Col (None, name) } | name = BEFORE { Col (None, name) } | name = BEGIN { Col (None, name) } | name = BY { Col (None, name) } | name = CASCADE { Col (None, name) } | name = COLUMN { Col (None, name) } | name = CONFLICT { Col (None, name) } | name = CROSS { Col (None, name) } | name = CURRENT { Col (None, name) } | name = DO { Col (None, name) } | name = EACH { Col (None, name) } | name = END { Col (None, name) } | name = FOLLOWING { Col (None, name) } | name = FOR { Col (None, name) } | name = FULL { Col (None, name) } | name = GENERATED { Col (None, name) } | name = GLOB { Col (None, name) } | name = IF { Col (None, name) } | name = INNER { Col (None, name) } | name = INSTEAD { Col (None, name) } | name = LEFT { Col (None, name) } | name = LIKE { Col (None, name) } | name = MATERIALIZED { Col (None, name) } | name = NATURAL { Col (None, name) } | name = NO { Col (None, name) } | name = OF { Col (None, name) } | name = OFFSET { Col (None, name) } | name = OUTER { Col (None, name) } | name = PARTITION { Col (None, name) } | name = PRAGMA { Col (None, name) } | name = PRECEDING { Col (None, name) } | name = RANGE { Col (None, name) } | name = RECURSIVE { Col (None, name) } | name = RELEASE { Col (None, name) } | name = RENAME { Col (None, name) } | name = REPLACE { Col (None, name) } | name = RESTRICT { Col (None, name) } | name = RIGHT { Col (None, name) } | name = ROLLBACK { Col (None, name) } | name = ROW { Col (None, name) } | name = ROWS { Col (None, name) } | name = SAVEPOINT { Col (None, name) } | name = TRIGGER { Col (None, name) } | name = VACUUM { Col (None, name) } | name = VIEW { Col (None, name) } | name = WITH { Col (None, name) } | name = WITHOUT { Col (None, name) } (* [CURRENT_TIMESTAMP] and its two siblings are calls, not names: sqlite3 reads the bare word as a zero-argument function (parse.y, [term ::= CTIME_KW]) and has no production for an argument list after it, so [current_timestamp()] is a syntax error there and here. *) | s = CTIME_KW { Func (String.lowercase_ascii s, []) }
(* What RAISE takes: IGNORE alone, or one of the three undoing actions and the message it carries. ROLLBACK is a keyword token; ABORT, FAIL and IGNORE lex as IDENT (they stay usable as identifiers elsewhere), so they are matched on their text, as [or_conflict] matches the same three. A word that is none of them is a syntax error, as it is in sqlite3, where raisetype has exactly these productions. *)raise_arg: | id = IDENT { match String.uppercase_ascii id with | "IGNORE" -> (Ignore, None) (* [RAISE(ABORT)] and [RAISE(FAIL)]: the action is one raisetype takes, so lemon shifts it and stops at the ")" that should have been a comma. Anything else is no raisetype at all and stops on the word. *) | "ABORT" | "FAIL" -> near ")" | _ -> near id } | ROLLBACK; COMMA; e = expr { (Rollback, Some e) } | id = IDENT; COMMA; e = expr { match String.uppercase_ascii id with | "ABORT" -> (Abort, Some e) | "FAIL" -> (Fail, Some e) (* [RAISE(IGNORE)] is complete at the word, so the comma is what has no transition after it. *) | "IGNORE" -> near "," | _ -> near id }
(* The column half of a qualified reference. Bare, each of these is a rule of its own above, because IDENT there also begins a function call. *)column_ident: | s = IDENT { s } | s = fallback_ident { s } | s = CTIME_KW { s } (* COLUMN is a name here and not in {!name}: a qualifier has already settled that a name is what follows, so nothing reads it as the keyword of [ALTER TABLE t ADD COLUMN]. *) | s = COLUMN { s }
filter_clause: | FILTER; LPAREN; WHERE; p = expr; RPAREN { p }
(* OVER (PARTITION BY ... ORDER BY ...): an explicit frame clause is not parsed; the default frame applies. *)window_clause: | OVER; LPAREN; pb = loption(partition_clause); ob = loption(order_clause); fr = ioption(frame_clause); RPAREN { { partition = pb; order = ob; frame = fr } }
partition_clause: | PARTITION; BY; l = separated_nonempty_list(COMMA, expr) { l }
(* A frame: [ROWS|RANGE] BETWEEN lo AND hi, or a single bound [ROWS|RANGE] lo (shorthand for BETWEEN lo AND CURRENT ROW). *)frame_clause: | k = frame_kind; BETWEEN; lo = frame_bound; AND; hi = frame_bound { { rows = k; start = lo; end_ = hi } } | k = frame_kind; lo = frame_bound { { rows = k; start = lo; end_ = Current_row } }
frame_kind: | ROWS { true } | RANGE { false }
frame_bound: | UNBOUNDED; PRECEDING { Unbounded_preceding } | UNBOUNDED; FOLLOWING { Unbounded_following } | CURRENT; ROW { Current_row } | n = expr; PRECEDING { Preceding n } | n = expr; FOLLOWING { Following n }
case_when: | WHEN; c = expr; THEN; r = expr { (c, r) }
case_else: | ELSE; e = expr { e }
func_args: | STAR { [ Star ] } | l = separated_list(COMMA, expr) { l }
(* The argument list of a call written with DISTINCT: an expression list of any length, the empty one included ([count(DISTINCT)] is a call the grammar builds and {!distinct_call} refuses). A star is not one of them: parse.y gives it a production of its own ([expr ::= idj LP STAR RP]), with no [distinct] before it, so a star after DISTINCT is a syntax error there as it is here. *)distinct_args: | l = separated_list(COMMA, expr) { l }
literal: | n = NUMBER { number_to_value n } | s = STRING { Text s } | b = BLOB { Blob b } | NULL { Null }
(* A literal in expression position. Not {!literal} wrapped in [Lit]: a decimal integer no int64 holds keeps its text here ({!Ast.number_expr}), because a unary minus over it decides whether it is a real or [min_int], while the DEFAULT and PRAGMA contexts that read {!literal} take a settled value and spell their own sign. *)literal_expr: | n = NUMBER { Ast.number_expr n } | s = STRING { Lit (Text s) } | b = BLOB { Lit (Blob b) } | NULL { Lit Null }
(* ── CREATE TABLE ────────────────────────────────────────────── *)
create_table: | b = create_table_body; EOF { b }
create_table_body: | CREATE; TABLE; e = if_not_exists_b; name = ident; LPAREN; defs = separated_nonempty_list(COMMA, column_or_constraint); RPAREN; opts = table_options { make_create_table ~opts name e defs }
(* The statement form left-factors the column-def and [AS query] tails so the shared [CREATE TABLE name] prefix has no conflict. *)create_table_statement: | CREATE; TABLE; e = if_not_exists_b; name = ident; r = create_table_tail { match r with | `Cols (defs, opts) -> Create_table (make_create_table ~opts name e defs) | `As s -> Create_table_as { name = name; if_not_exists = e; query = s } }
create_table_tail: | LPAREN; defs = separated_nonempty_list(COMMA, column_or_constraint); RPAREN; opts = table_options { `Cols (defs, opts) } | AS; s = select_stmt { `As s }
(* Trailing table options after the column list: [WITHOUT ROWID] and [STRICT], comma-separated in any order. Any other word is rejected, as sqlite3 rejects it. *)table_options: | (* empty *) { no_table_opts } | l = separated_nonempty_list(COMMA, table_option) { List.fold_left add_table_opt no_table_opts l }
table_option: | WITHOUT; w = ident { Without w } | w = IDENT { Bare w }
(* Dispatch: table constraints start with PRIMARY, UNIQUE, FOREIGN, CHECK, or CONSTRAINT. Everything else is a column definition. *)column_or_constraint: | t = table_constraint { `Tbl t } | k = table_check { `Check k } | f = table_fk { `Fk f } | c = column_decl { `Col c }
(* One column: its name, then its type and constraints. *)column_decl: | name = name; body = col_body { make_column_decl name body }
(* The name a column declaration begins with: {!fallback_ident} plus the words a column declaration settles on its own. COLUMN is the one word of the fallback set left out, because [ALTER TABLE t ADD COLUMN] would then be ambiguous between the keyword and a column of that name. sqlite3 has the same ambiguity and lemon shifts the keyword ([ADD COLUMN INT] declares "INT"); leaving COLUMN out here is that same reading, without a conflict menhir would resolve silently. *)name: | s = IDENT { s } | s = fallback_ident { s } | s = CTIME_KW { s } (* UNBOUNDED is a name here and not in {!nonbool_atom}: a column declaration is not a frame bound, so nothing reads it as the one in [ROWS UNBOUNDED PRECEDING]. *) | s = UNBOUNDED { s } (* Words parse.y keeps reserved, which a column declaration takes here because the column body is a token list rather than a grammar of its own. sqlite3 refuses [CREATE TABLE t(select INT)]; this accepts it. *) | s = TABLE { s } | s = CREATE { s } | s = NOT { s } | s = NULL { s } | s = DEFAULT { s } | s = REFERENCES { s } | s = COLLATE { s } | s = AS { s } | s = AUTOINCREMENT { s } | s = EXISTS { s } | s = ON { s } | s = SELECT { s } | s = DISTINCT { s } | s = FROM { s } | s = WHERE { s } | s = JOIN { s } | s = ORDER { s } | s = LIMIT { s } | s = AND { s } | s = OR { s } | s = IN { s } | s = IS { s }
(* ── Table-level constraints ─────────────────────────────────── *)
(* Each carries the same optional [ON CONFLICT <action>] tail a column constraint may end with; sqlite3 keeps it on the index the constraint builds. *)table_constraint: | UNIQUE; LPAREN; cs = indexed_names; RPAREN; r = table_on_conflict { Unique (cs, r) } | PRIMARY; KEY; LPAREN; cs = indexed_names; RPAREN; r = table_on_conflict { Primary_key (cs, r) } | CONSTRAINT; ident; UNIQUE; LPAREN; cs = indexed_names; RPAREN; r = table_on_conflict { Unique (cs, r) } | CONSTRAINT; ident; PRIMARY; KEY; LPAREN; cs = indexed_names; RPAREN; r = table_on_conflict { Primary_key (cs, r) }
(* [ON CONFLICT <action>], absent meaning the constraint names no resolution. REPLACE and ROLLBACK are keyword tokens; IGNORE/ABORT/FAIL lex as IDENT so they stay usable as identifiers elsewhere. *)table_on_conflict: | (* empty *) { Unspecified } | ON; CONFLICT; REPLACE { Replace } | ON; CONFLICT; ROLLBACK { Rollback } | ON; CONFLICT; id = IDENT { match String.uppercase_ascii id with | "IGNORE" -> Ignore | "ABORT" -> Abort | "FAIL" -> Fail | _ -> Catalog.Error.fail_prepare "ON CONFLICT: unknown conflict action %s" id }
(* A PRIMARY KEY / UNIQUE column list: each column may carry a COLLATE, naming the sequence the constraint's autoindex orders its keys by, and an ASC/DESC naming the direction they run in. *)indexed_names: | separated_nonempty_list(COMMA, indexed_name) { $1 }
indexed_name: | c = ident; k = ioption(constraint_collate); d = key_direction { { column = c; collation = k; desc = d } }
constraint_collate: | COLLATE; c = ident { c }
(* A CHECK constraint, column- or table-level. The predicate is parsed as a real expression and its verbatim source offsets are captured so a violation can quote it exactly -- schema qualifier and all, since the offsets bracket the source rather than the tree. CONSTRAINT names are accepted but unused (SQLite reports the predicate text, not the name). *)table_check: | CHECK; LPAREN; e = expr; RPAREN { { expr = ignore_schema_qualifiers e; start = $startofs(e); stop = $endofs(e) } } | CONSTRAINT; ident; CHECK; LPAREN; e = expr; RPAREN { { expr = ignore_schema_qualifiers e; start = $startofs(e); stop = $endofs(e) } }
(* A table-level FOREIGN KEY. The child columns pair with a [fk_clause] ([REFERENCES parent [(cols)] [ON DELETE/UPDATE action]]); the result carries the child columns so [create_table_body] can build the {!Ast.foreign_key}. *)table_fk: | FOREIGN; KEY; LPAREN; cs = names; RPAREN; r = fk_clause { (cs, r) } | CONSTRAINT; ident; FOREIGN; KEY; LPAREN; cs = names; RPAREN; r = fk_clause { (cs, r) }
(* The shared REFERENCES tail, also used for a column-level [col REFERENCES ...]. Returns [(parent, parent_cols, on_delete, on_update)]; parent_cols [] means the parent's primary key. *)fk_clause: | REFERENCES; parent = ident; pcols = loption(paren_names); acts = list(fk_ref_action) { let on_delete, on_update = fk_actions acts in (parent, pcols, on_delete, on_update) }
fk_ref_action: | ON; DELETE; a = fk_action { `Del a } | ON; UPDATE; a = fk_action { `Upd a }
fk_action: | SET; NULL { Set_null } | SET; DEFAULT { Set_default } | CASCADE { Cascade } | RESTRICT { Restrict } | NO; ACTION { No_action }
paren_names: | LPAREN; cs = names; RPAREN { cs }
names: | separated_nonempty_list(COMMA, column_name) { $1 }
(* The name a statement writes to or keys on: an INSERT's column list, an UPDATE's assignment target, a foreign key's columns, an upsert's target. parse.y takes each as [nm] -- [idlist ::= idlist COMMA nm], [setlist ::= nm EQ expr] -- the same word a CREATE TABLE declares the column under, which is the point: a column declared as [row] has to be writable as [row].
Wider than {!name} by COLUMN and UNBOUNDED, because neither [ALTER TABLE ... ADD COLUMN] nor [ROWS UNBOUNDED PRECEDING] stands anywhere near a column list; there is no keyword reading here to weigh the name against. *)column_name: | s = IDENT { s } | s = fallback_ident { s } | s = CTIME_KW { s } | s = COLUMN { s } | s = UNBOUNDED { s }
(* ── Column body: flat token list ────────────────────────────── *)
col_body: | items = list(col_body_item) { let toks = List.filter_map (function `T t -> Some t | `K _ | `D _ | `G _ | `A _ -> None) items in let checks = List.filter_map (function `K k -> Some k | `T _ | `D _ | `G _ | `A _ -> None) items in (* A column has at most one DEFAULT and one generation expr; keep the first if repeated. *) let default = List.find_map (function `D e -> Some e | `T _ | `K _ | `G _ | `A _ -> None) items in let generated = List.find_map (function `G e -> Some e | `T _ | `K _ | `D _ | `A _ -> None) items in let actions = List.filter_map (function `A a -> Some a | `T _ | `K _ | `D _ | `G _ -> None) items in (toks, checks, default, generated, actions) }
col_body_item: | t = col_token { `T t } | k = col_check { `K k } | e = col_default { `D e } | e = col_generated { `G e } (* The referential actions of a column-level [REFERENCES], as their own item rather than as opaque tokens: they are the same [fk_ref_action] the table-level clause takes, so both spellings build one foreign key from one production. A column's stream still holds the REFERENCES word and the parent it names -- [Ast.column_foreign_key] reads those -- and the words ON/DELETE/UPDATE/CASCADE would be a declared type if they stayed in it. *) | a = fk_ref_action { `A a }
(* A generated (computed) column: [[GENERATED ALWAYS] AS (expr) [VIRTUAL|STORED]]. The trailing STORED/VIRTUAL keyword is collected as an opaque column token, and {!Ast.split_generated_kind} reads the kind off it once the column's whole stream is in hand: neither word is reserved, so the grammar cannot claim it here without taking it from a column that declares it as a type. *)col_generated: | GENERATED; ALWAYS; AS; LPAREN; e = expr; RPAREN { e } | AS; LPAREN; e = expr; RPAREN { e }
(* A column-level CHECK. Like the table-level form, the predicate is a real expression whose verbatim source offsets are captured for the error text. *)col_check: | CHECK; LPAREN; e = expr; RPAREN { { expr = ignore_schema_qualifiers e; start = $startofs(e); stop = $endofs(e) } }
(* A column DEFAULT: a constant literal, a signed number, a bare word, or a parenthesised expression. The parenthesised form is checked for constancy when the table is created (a column reference there is an error). *)col_default: | DEFAULT; l = literal { Lit l } | DEFAULT; MINUS; n = NUMBER { Lit (Ast.number ~negative:true n) } | DEFAULT; PLUS; n = NUMBER { Lit (number_to_value n) } | DEFAULT; LPAREN; e = expr; RPAREN { e } (* [DEFAULT CURRENT_TIMESTAMP], unparenthesised: sqlite3's DEFAULT takes a [term], and a CTIME_KW is one (parse.y, [ccons ::= DEFAULT term]). *) | DEFAULT; s = CTIME_KW { Func (String.lowercase_ascii s, []) } (* [DEFAULT c], the bare word: the string 'c' and not the column c. parse.y gives it a production of its own, [ccons ::= DEFAULT scanpt id], whose action builds a TK_STRING -- a column reference would have nothing to read anyway, since a default is evaluated with no row in hand. The word is [id], so {!fallback_ident} spells one, as do COLUMN and UNBOUNDED; CTIME_KW does not, because the production above already reads it as the call it is. *) | DEFAULT; s = IDENT { Lit (Text s) } | DEFAULT; s = fallback_ident { Lit (Text s) } | DEFAULT; s = COLUMN { Lit (Text s) } | DEFAULT; s = UNBOUNDED { Lit (Text s) }
%inline col_token: | s = IDENT { Word s } | s = NUMBER { Number s } | s = STRING { Number ("'" ^ s ^ "'") } | k = keyword { Word k } | p = punct { Word p } | LPAREN; ts = list(paren_token); RPAREN { Parens ts }
paren_token: | s = IDENT { Word s } | s = NUMBER { Number s } | s = STRING { Number ("'" ^ s ^ "'") } | k = keyword { Word k } | p = punct { Word p } | COMMA { Word "," } | LPAREN; ts = list(paren_token); RPAREN { Parens ts }
(* Operators that may appear inside CHECK / DEFAULT / GENERATED expressions. They are opaque to column classification, but must be accepted so the DDL parser does not choke on them. *)%inline punct: | EQ { "=" } | NEQ { "<>" } | LT { "<" } | LE { "<=" } | GT { ">" } | GE { ">=" } | STAR { "*" } | DOT { "." } | PLUS { "+" } | MINUS { "-" } | SLASH { "/" } | PERCENT { "%" } | BITAND { "&" } | BITOR { "|" } | SHL { "<<" } | SHR { ">>" } | CONCAT { "||" }
(* ── Keywords usable as column body tokens ───────────────────── *)
%inline keyword: | s = PRIMARY { s } | s = KEY { s } | s = NOT { s } | s = NULL { s } | s = UNIQUE { s } | s = REFERENCES { s } | s = COLLATE { s } | s = AUTOINCREMENT { s } | s = CONSTRAINT { s } (* [ON CONFLICT <action>] on a column constraint: collected as opaque tokens so the column still parses (the action keywords ABORT/FAIL/IGNORE lex as IDENT; REPLACE/ROLLBACK/CONFLICT are keyword tokens). *) | s = CONFLICT { s } | s = REPLACE { s } | s = ROLLBACK { s } | s = EXISTS { s } | s = CREATE { s } | s = TABLE { s } | s = IF { s } | s = ON { s } | s = ASC { s } | s = DESC { s } | s = FOREIGN { s } | s = SELECT { s } | s = DISTINCT { s } | s = FROM { s } | s = WHERE { s } | s = JOIN { s } | s = INNER { s } | s = LEFT { s } | s = OUTER { s } | s = ORDER { s } | s = BY { s } | s = LIMIT { s } | s = AND { s } | s = OR { s } | s = IN { s } | s = IS { s } | s = LIKE { s }
(* ── Identifiers (including quoted) ──────────────────────────── *)
(* parse.y's [id] once its %fallback list has been applied -- the words lemon retries as an identifier wherever it cannot shift them as a keyword, plus the join words its [idj] class (ID | INDEXED | JOIN_KW) puts in identifier position outright -- less the four this grammar holds back. Every name position in this grammar is this set plus or minus the words its own neighbourhood settles, so it is stated once here and each position names its exceptions with the reason.
The four held back are GLOB, IF, LIKE and RECURSIVE. Each is read a second way at a point where an object name also stands, and {!ident} states the reading and what leaving it out costs. {!fallback_ident} is this set with the four added back, for the positions that name no object.
Inlined, so a use site gets one production per word. Reaching the words through a shared nonterminal instead makes the parser choose that reduction before the token that would settle it, which costs reduce/reduce conflicts; %inline expands to the same productions writing them out by hand would. *)%inline fallback_ident_nm: | s = ACTION { s } | s = AFTER { s } | s = ALWAYS { s } | s = ANALYZE { s } | s = ASC { s } | s = BEFORE { s } | s = BEGIN { s } | s = BY { s } | s = CASCADE { s } | s = CAST { s } | s = CONFLICT { s } | s = CROSS { s } | s = CURRENT { s } | s = DESC { s } | s = DO { s } | s = EACH { s } | s = END { s } | s = FOLLOWING { s } | s = FOR { s } | s = FULL { s } | s = GENERATED { s } | s = INNER { s } | s = INSTEAD { s } | s = KEY { s } | s = LEFT { s } | s = MATERIALIZED { s } | s = NATURAL { s } | s = NO { s } | s = OF { s } | s = OFFSET { s } | s = OUTER { s } | s = PARTITION { s } | s = PRAGMA { s } | s = PRECEDING { s } | s = RAISE { s } | s = RANGE { s } | s = RELEASE { s } | s = RENAME { s } | s = REPLACE { s } | s = RESTRICT { s } | s = RIGHT { s } | s = ROLLBACK { s } | s = ROW { s } | s = ROWS { s } | s = SAVEPOINT { s } | s = TRIGGER { s } | s = VACUUM { s } | s = VIEW { s } | s = WITH { s } | s = WITHOUT { s }
(* parse.y's [id] in full: {!fallback_ident_nm} plus the four words no name position in this grammar can spell, which the positions that are not name positions still take. *)%inline fallback_ident: | s = fallback_ident_nm { s } | s = GLOB { s } | s = IF { s } | s = LIKE { s } | s = RECURSIVE { s }
(* The name an object is created, dropped and referred to under: a table, an index, a view, a trigger, a savepoint, a constraint, a CTE, and the table a query scans. parse.y takes every one of them as [nm], which is [idj] and so every %fallback word, so a table called [row] is written unquoted in each position that names it. Measured against sqlite3 3.54.0:
CREATE TABLE row(a); CREATE TABLE after(a); CREATE TABLE replace(a); SELECT * FROM row; no such table: row
The last one is the grammar answering before the schema does. The words parse.y keeps reserved stay reserved, which is what stops this from being "take every token as a name":
CREATE TABLE select(a); near "select": syntax error SELECT * FROM select; near "select": syntax error
The set is {!fallback_ident_nm} and not {!fallback_ident}, because four of the %fallback words carry a second reading at the very point a name stands, and one lookahead token does not say which reading was meant.
IF and RECURSIVE each open a clause. [DROP TABLE IF] is the table called [if] or the head of [IF EXISTS], and [WITH RECURSIVE] is the CTE called [recursive] or the keyword before the first one; both are shift/reduce. sqlite3 reads the keyword in both and refuses the name, so leaving them out is its own answer there:
CREATE TABLE if(a); near "(": syntax error DROP TABLE if; near ";": syntax error WITH recursive AS (SELECT 1) SELECT 1; near "AS": syntax error
LIKE and GLOB each spell an operator, and the clash is reduce/reduce. After the [WITH] that opens a parenthesised query, [LIKE] begins the first CTE's name; after a column called [with], the same token is the operator over it. [(with LIKE 'x')] and [(WITH like AS ...)] part company at the token after the one in hand.
lemon settles all four by retrying a token as an identifier only in the states where it cannot shift it, and that per-state answer has no spelling in menhir. So the four are left out of every name position, rather than kept by weakening the rules they clash with. Three names sqlite3 3.54.0 takes are what that costs, each accepted there and refused here:
CREATE TABLE recursive(a); CREATE TABLE like(a); CREATE TABLE glob(a);
IF costs one more, since sqlite3 reads it as a name wherever no clause competes: [ALTER TABLE if RENAME TO x] answers "no such table: if" there.
Menhir is at zero conflicts, and each of the four on its own puts it back: IF nine shift/reduce states, RECURSIVE two, LIKE and GLOB one reduce/reduce state each. Keep it at zero.
COLUMN and UNBOUNDED are in, as they are in {!column_name} and for the same reason: no object-name position stands next to [ALTER TABLE ... ADD COLUMN] or to [ROWS UNBOUNDED PRECEDING], so there is no keyword reading to weigh the name against. *)ident: | s = IDENT { s } | s = fallback_ident_nm { s } | s = CTIME_KW { s } | s = COLUMN { s } | s = UNBOUNDED { s }