A Fennel Parser for IncDown
Find a file
2023-12-19 00:11:05 -08:00
block.fnl feat(inline): footnote parser 2023-12-19 00:11:05 -08:00
flake.lock feat: init block tokenier 2023-11-27 23:36:16 -08:00
flake.nix feat: init block tokenier 2023-11-27 23:36:16 -08:00
inline.fnl feat(inline): footnote parser 2023-12-19 00:11:05 -08:00
main.fnl feat(inline): footnote parser 2023-12-19 00:11:05 -08:00
README.incd feat: inline-parser 2023-12-03 23:47:21 -08:00
render.fnl feat(inline): footnote parser 2023-12-19 00:11:05 -08:00

=== IncDown Parser ===

Turn IncDown into HTML. 
First attempt at a parser, written in [Fennel](https://fennel-lang.org/)

Input -> Block Lexer -> Inline Lexer -> Renderer

I think that's how this will work

= Status =
Started writing the inline lexer sort of, but not sure I'm doing it correctly. Since we now know the "type" of each block, we can extract out semantically meaningful data. The one thing I'm still unsure about is how to apply decoration checks to *all* contents. Regardless of it's a header, a paragraph, or a list, I should still be able to apply any of the decorations I can define. Image alt descriptions should also be decorated, and so too should footnote content.

I'm also thinking about whether I want to be able to link to specific sections. The simplest way to implement this wouold be to have a "reverse" counter: the last parsed section is 0 and we count upwards to the most recent. Since by convention we insert data at the "start" of the file. I could have a special character (I was thinking `#`) to refer to a section, but that means the filename cannot have that character. Personally, that seems okay?  

I'm also currently unsure how I'm to link footnotes against their reference. 

-----
%date 23-12-01

Block Lexer currently working_[#], but doesn't support footnotes, not sure how to address that. Thinking I might do what I've done here use `::` to denote footnotes, instead of using metadata attributes. Not decided
I'm also wondering how to do the inline lexer. Should I store the index start and end values of specific decorations / lexically meaningful parts? I suppose I could use that to enter whatever sequence was necessary to render them. There's also only a handful. 
The other thing I need to work out is how to handle metadata and sections, and what piece of the pipeline would handle that. Because for rendering HTML, sections don't really matter, but for handling the metadata it does. So the question kind of is what sort of application needs the metadata, and how should that metadata be stored? If there just block tokens should it be the renderer's job to handle? Not sure yet. 

:: 

- Code fences are now working, but it's ugly as fuck. I have previously been using the older markdown starnard that uses `{{{` to open a fence and `}}}` to close one. This meant I needed a seperate check to parse the end of the block, but all other blocks work by applying the same regex (which should match *every* line). Code fences, (even if using ``` instead of {{{) would not match every block, but instead begins a "mode", which even if the pattern doesn't match, we add it to the current block. 
::

-----
%status archived
%date 23-11-30

= The previous problem with code fences =

The code fence regex is wrong, since it looks for the entire block, instead of just the first line. Not quite sure how to use a different regex or "swap modes" into code fence mode such that everything until `}}}` is part of the codeblock. 

-----

== Spec ==

= Blocks =
Each block is generally seperated by \n 
This means we require lookahead?

Block parser breaks the text up into blocks:
- Metadata
  - %attribute
- Paragraphs
  - Any character not claimed by the others?
- Lists
  - # or - started lines (any number of spaces)
	- Task lists?
- Headings
  - = start and ending
- Quotes
  - > start (any number of spaces)
- Images
  - ![Alt](Link)
- Foot notes
  - _[#] and then %footnote ??? need to think about this one
- Code blocks
  - {{{code hint\n code}}}
- Section breaks
  - 5 continuous - on a single line

= Inline =
- Decorations
  - Strikethrough ~
  - Italics _
  - Strong *
	- Monospace `
	- Spoiler? >! (This clashes with quotes?)
	- Censor? >~  (This clashes with quotes?)
- Links (Internal & External) [[]] or []()

- Images? Maybe images can't be inlined? 
- Footnotes

== Future Work ==

If this goes well may attempt to write:
- Blog engine in Fennel on top of this
- TreeSitter Grammar
- NVim Theme (does this come free with treesitter?)
- Pandoc Grammer? (Seems unlikely)

-----

= Things skipped because I don't use them (yet?) =
- Raw HTML entries 
- Tables