Matt's Lexicon Format #

A human-friendly DSL for ATProto Lexicons

This is a work in progress, things are subject to break and change

What it looks like #

/// Blog-style post record for the Bluesky feed.
@const("revision", 3)
self {}

record post {
    text!: string constrained {
        maxLength: 3000,
        maxGraphemes: 300,
    },
    createdAt!: Datetime,
    reply: replyRef,
}

def type replyRef = {
    root!: com.atproto.repo.strongRef,
    parent!: com.atproto.repo.strongRef,
};

self {} represents the lexicon itself. Docs on it map to the top-level description. @const and @reference annotations carry non-spec or vendor extension fields through to JSON; attach them to a def, a field, or self {}.

XRPC endpoints are first-class, including declared errors and subscriptions:

query getPost(
    uri!: AtUri,
): post | error {
    NotFound,
};

Try it without installing anything in the playground, which resolves use imports against live published lexicons.

What the CLI does #

Command	Purpose
mlf init	Scaffold a project (mlf.toml + source tree)
mlf check	Parse + resolve + typecheck the whole workspace
mlf fmt	Canonical formatting (--check for CI)
mlf generate lexicon	.mlf → Lexicon JSON
mlf generate code -g <lang>	.mlf → TypeScript / Go / Rust bindings
mlf generate mlf	Lexicon JSON → .mlf (import existing lexicons)
mlf fetch	Fetch published lexicons your imports reference
mlf validate	Validate a JSON record against a schema
mlf publish / unpublish / status / diff	Publish lexicons to your PDS and manage _lexicon DNS records
mlf explain <code>	Extended explanation for an error code (like rustc --explain)

Converting existing published lexicons is lossless: JSON → MLF → JSON round-trips canonically, and anything MLF cannot represent is warned about rather than silently dropped.

Installation #

Right now you can only install mlf from source:

CLI Tool #

# Install with all code generators (default: TypeScript, Go, Rust)
cargo install --path mlf-cli --all-features

# Install with only specific generators
cargo install --path mlf-cli --no-default-features --features typescript,go

# Install with JSON generation only
cargo install --path mlf-cli --no-default-features

Language Server #

For editor integration with real-time diagnostics:

# Build the language server
cargo build --release -p mlf-lsp

# The binary will be at: target/release/mlf-lsp

Editor Setup #

VS Code: Create an extension configuration:

{
  "languageServer": {
    "module": "/path/to/mlf-lsp",
    "args": [],
    "filetypes": ["mlf"]
  }
}

Neovim: Add to your LSP config:

local lspconfig = require('lspconfig')
local configs = require('lspconfig.configs')

configs.mlf = {
  default_config = {
    cmd = { '/path/to/mlf-lsp' },
    filetypes = { 'mlf' },
    root_dir = lspconfig.util.root_pattern('mlf.toml', '.git'),
  },
}

lspconfig.mlf.setup{}

Helix: Add to languages.toml:

[[language]]
name = "mlf"
scope = "source.mlf"
file-types = ["mlf"]
language-servers = ["mlf-lsp"]

[language-server.mlf-lsp]
command = "/path/to/mlf-lsp"

See mlf-lsp/README.md for more details.

Documentation #

The MLF website has the full documentation, guides, and examples.

Architecture #

See ARCHITECTURE.md for the workspace overview (pipelines, crate map, invariants), PLUGIN_ARCHITECTURE.md for the codegen/DNS plugin split, and docs/diagnostics.md for how errors are worded and rendered.

Development #

just install-tools   # one-time: cargo-nextest
just test            # full test suite
just lint            # clippy (warnings are errors in CI)
just fmt             # rustfmt

Real-world corpus #

mlf-real-examples holds 2,675 lexicons published by other people across 155 authorities, in both their original JSON and converted MLF. Most of the bugs worth finding come from schemas written without MLF in mind.

tests/real_world/ is separate. It fetches nine of those authorities from the live network on each run and checks they still round-trip, so CI catches upstream drift without depending on the corpus repo.

With the corpus checked out next to this repo, just roundtrip converts every lexicon in it to MLF, generates JSON back, and reports how much survived:

authority                   total  identical  differs   noconv    noref   match
pub.chive                     205        205        0        0        0  100.0%
social.colibri                204        204        0        0        0  100.0%
...
TOTAL                        2675       2581       38        1       55   98.5%

noconv counts lexicons MLF cannot express, noref those referencing schemas nobody published; neither counts against the match rate. Add -v to list every lexicon that is not identical, or --json for a machine-readable report. The conversion runs fresh on each invocation, so the numbers track the current converter rather than whenever the corpus was last regenerated.

License #

MIT