A Fennel Parser for IncDown
Find a file
2023-11-29 23:04:10 -08:00
flake.lock feat: init block tokenier 2023-11-27 23:36:16 -08:00
flake.nix feat: init block tokenier 2023-11-27 23:36:16 -08:00
main.fnl feat: tags and footnotes 2023-11-29 23:04:10 -08:00
README.incd feat: tags and footnotes 2023-11-29 23:04:10 -08:00

=== IncDown Parser ===

Turn IncDown into HTML. 
First attempt at a parser, written in [Fennel](https://fennel-lang.org/)

Input -> Block Lexer -> Inline Lexer -> Renderer

I think that's how this will work

= Status =

Block Lexer currently working_[#], but doesn't support footnotes, not sure how to address that. Thinking I might do what I've done here use `::` to denote footnotes, instead of using metadata attributes. Not decided
I'm also wondering how to do the inline lexer. Should I store the index start and end values of specific decorations / lexically meaningful parts? I suppose I could use that to enter whatever sequence was necessary to render them. There's also only a handful. 
The other thing I need to work out is how to handle metadata and sections, and what piece of the pipeline would handle that. Because for rendering HTML, sections don't really matter, but for handling the metadata it does. So the question kind of is what sort of application needs the metadata, and how should that metadata be stored? If there just block tokens should it be the renderer's job to handle? Not sure yet. 

:: 

- Code fences are now working, but it's ugly as fuck. I have previously been using the older markdown starnard that uses `{{{` to open a fence and `}}}` to close one. This meant I needed a seperate check to parse the end of the block, but all other blocks work by applying the same regex (which should match *every* line). Code fences, (even if using ``` instead of {{{) would not match every block, but instead begins a "mode", which even if the pattern doesn't match, we add it to the current block. 

::

-----
%status archived

= The previous problem with code fences =

The code fence regex is wrong, since it looks for the entire block, instead of just the first line. Not quite sure how to use a different regex or "swap modes" into code fence mode such that everything until `}}}` is part of the codeblock. 

-----

== Spec ==

= Blocks =
Each block is generally seperated by \n 
This means we require lookahead?

Block parser breaks the text up into blocks:
- Metadata
  - %attribute
- Paragraphs
  - Any character not claimed by the others?
- Lists
  - # or - started lines (any number of spaces)
- Headings
  - = start and ending
- Quotes
  - > start (any number of spaces)
- Images
  - ![Alt](Link)
- Foot notes
  - _[#] and then %footnote ??? need to think about this one
- Code blocks
  - {{{code hint\n code}}}
- Section breaks
  - 5 continuous - on a single line

= Inline =
- Decorations
- Links (Internal & External)
- Images? Maybe images can't be inlined? 
- Footnotes

== Future Work ==

If this goes well may attempt to write:
- Blog engine in Fennel on top of this
- TreeSitter Grammar
- NVim Theme (does this come free with treesitter?)
- Pandoc Grammer? (Seems unlikely)


-----

= Things skipped because I don't use them (yet?) =
- Raw HTML entries 
- Tables