Something went wrong. Try again.
Markdown parser fork with extended syntax for personal use.
Something went wrong. Try again.
8.2 kB · 290 lines
Rust
at main
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291//! Frontmatter occurs at the start of the document.//!//! ## Grammar//!//! Frontmatter forms with the following BNF//! (<small>see [construct][crate::construct] for character groups</small>)://!//! ```bnf//! frontmatter ::= fence_open *( eol *byte ) eol fence_close//! fence_open ::= sequence *space_or_tab//! ; Restriction: markers in `sequence` must match markers in opening sequence.//! fence_close ::= sequence *space_or_tab//! sequence ::= 3'+' | 3'-'//! ```//!//! Frontmatter can only occur once.//! It cannot occur in a container.//! It must have a closing fence.//! Like flow constructs, it must be followed by an eol (line ending) or//! eof (end of file).//!//! ## Extension//!//! > 👉 **Note**: frontmatter is not part of `CommonMark`, so frontmatter is//! > not enabled by default.//! > You need to enable it manually.//! > See [`Constructs`][constructs] for more info.//!//! As there is no spec for frontmatter in markdown, this extension follows how//! YAML frontmatter works on `github.com`.//! It also parses TOML frontmatter, just like YAML except that it uses a `+`.//!//! ## Recommendation//!//! When authoring markdown with frontmatter, it’s recommended to use YAML//! frontmatter if possible.//! While YAML has some warts, it works in the most places, so using it//! guarantees the highest chance of portability.//!//! In certain ecosystems, other flavors are widely used.//! For example, in the Rust ecosystem, TOML is often used.//! In such cases, using TOML is an okay choice.//!//! ## Tokens//!//! * [`Frontmatter`][Name::Frontmatter]//! * [`FrontmatterFence`][Name::FrontmatterFence]//! * [`FrontmatterSequence`][Name::FrontmatterSequence]//! * [`FrontmatterChunk`][Name::FrontmatterChunk]//! * [`LineEnding`][Name::LineEnding]//! * [`SpaceOrTab`][Name::SpaceOrTab]//!//! ## References//!//! * [`micromark-extension-frontmatter`](https://github.com/micromark/micromark-extension-frontmatter)//!//! [constructs]: crate::Constructs
use crate::construct::partial_space_or_tab::space_or_tab;use crate::event::Name;use crate::state::{Name as StateName, State};use crate::tokenizer::Tokenizer;use crate::util::constant::FRONTMATTER_SEQUENCE_SIZE;
/// Start of frontmatter.////// ```markdown/// > | ---/// ^/// | title: "Venus"/// | ---/// ```pub fn start(tokenizer: &mut Tokenizer) -> State { // Indent not allowed. if tokenizer.parse_state.options.constructs.frontmatter && matches!(tokenizer.current, Some(b'+' | b'-')) { tokenizer.tokenize_state.marker = tokenizer.current.unwrap(); tokenizer.enter(Name::Frontmatter); tokenizer.enter(Name::FrontmatterFence); tokenizer.enter(Name::FrontmatterSequence); State::Retry(StateName::FrontmatterOpenSequence) } else { State::Nok }}
/// In open sequence.////// ```markdown/// > | ---/// ^/// | title: "Venus"/// | ---/// ```pub fn open_sequence(tokenizer: &mut Tokenizer) -> State { if tokenizer.current == Some(tokenizer.tokenize_state.marker) { tokenizer.tokenize_state.size += 1; tokenizer.consume(); State::Next(StateName::FrontmatterOpenSequence) } else if tokenizer.tokenize_state.size == FRONTMATTER_SEQUENCE_SIZE { tokenizer.tokenize_state.size = 0; tokenizer.exit(Name::FrontmatterSequence);
if matches!(tokenizer.current, Some(b'\t' | b' ')) { tokenizer.attempt(State::Next(StateName::FrontmatterOpenAfter), State::Nok); State::Retry(space_or_tab(tokenizer)) } else { State::Retry(StateName::FrontmatterOpenAfter) } } else { tokenizer.tokenize_state.marker = 0; tokenizer.tokenize_state.size = 0; State::Nok }}
/// After open sequence.////// ```markdown/// > | ---/// ^/// | title: "Venus"/// | ---/// ```pub fn open_after(tokenizer: &mut Tokenizer) -> State { if let Some(b'\n') = tokenizer.current { tokenizer.exit(Name::FrontmatterFence); tokenizer.enter(Name::LineEnding); tokenizer.consume(); tokenizer.exit(Name::LineEnding); tokenizer.attempt( State::Next(StateName::FrontmatterAfter), State::Next(StateName::FrontmatterContentStart), ); State::Next(StateName::FrontmatterCloseStart) } else { tokenizer.tokenize_state.marker = 0; State::Nok }}
/// Start of close sequence.////// ```markdown/// | ---/// | title: "Venus"/// > | ---/// ^/// ```pub fn close_start(tokenizer: &mut Tokenizer) -> State { if tokenizer.current == Some(tokenizer.tokenize_state.marker) { tokenizer.enter(Name::FrontmatterFence); tokenizer.enter(Name::FrontmatterSequence); State::Retry(StateName::FrontmatterCloseSequence) } else { State::Nok }}
/// In close sequence.////// ```markdown/// | ---/// | title: "Venus"/// > | ---/// ^/// ```pub fn close_sequence(tokenizer: &mut Tokenizer) -> State { if tokenizer.current == Some(tokenizer.tokenize_state.marker) { tokenizer.tokenize_state.size += 1; tokenizer.consume(); State::Next(StateName::FrontmatterCloseSequence) } else if tokenizer.tokenize_state.size == FRONTMATTER_SEQUENCE_SIZE { tokenizer.tokenize_state.size = 0; tokenizer.exit(Name::FrontmatterSequence);
if matches!(tokenizer.current, Some(b'\t' | b' ')) { tokenizer.attempt(State::Next(StateName::FrontmatterCloseAfter), State::Nok); State::Retry(space_or_tab(tokenizer)) } else { State::Retry(StateName::FrontmatterCloseAfter) } } else { tokenizer.tokenize_state.size = 0; State::Nok }}
/// After close sequence.////// ```markdown/// | ---/// | title: "Venus"/// > | ---/// ^/// ```pub fn close_after(tokenizer: &mut Tokenizer) -> State { match tokenizer.current { None | Some(b'\n') => { tokenizer.exit(Name::FrontmatterFence); State::Ok } _ => State::Nok, }}
/// Start of content chunk.////// ```markdown/// | ---/// > | title: "Venus"/// ^/// | ---/// ```pub fn content_start(tokenizer: &mut Tokenizer) -> State { match tokenizer.current { None | Some(b'\n') => State::Retry(StateName::FrontmatterContentEnd), Some(_) => { tokenizer.enter(Name::FrontmatterChunk); State::Retry(StateName::FrontmatterContentInside) } }}
/// In content chunk.////// ```markdown/// | ---/// > | title: "Venus"/// ^/// | ---/// ```pub fn content_inside(tokenizer: &mut Tokenizer) -> State { match tokenizer.current { None | Some(b'\n') => { tokenizer.exit(Name::FrontmatterChunk); State::Retry(StateName::FrontmatterContentEnd) } Some(_) => { tokenizer.consume(); State::Next(StateName::FrontmatterContentInside) } }}
/// End of content chunk.////// ```markdown/// | ---/// > | title: "Venus"/// ^/// | ---/// ```pub fn content_end(tokenizer: &mut Tokenizer) -> State { match tokenizer.current { None => { tokenizer.tokenize_state.marker = 0; State::Nok } Some(b'\n') => { tokenizer.enter(Name::LineEnding); tokenizer.consume(); tokenizer.exit(Name::LineEnding); tokenizer.attempt( State::Next(StateName::FrontmatterAfter), State::Next(StateName::FrontmatterContentStart), ); State::Next(StateName::FrontmatterCloseStart) } Some(_) => unreachable!("expected eof/eol"), }}
/// After frontmatter.////// ```markdown/// | ---/// | title: "Venus"/// > | ---/// ^/// ```pub fn after(tokenizer: &mut Tokenizer) -> State { debug_assert!( matches!(tokenizer.current, None | Some(b'\n')), "expected eol/eof after closing fence" ); tokenizer.exit(Name::Frontmatter); State::Ok}