Sebastian Schkudlara
Look at your RAG pipeline right now. Actually look at it.
You are taking structured Markdown — a format inherently designed to convey hierarchy — and grinding it into a fine paste of flat text chunks before feeding it to your LLM. You are throwing away the very semantics that make the document legible.
That’s not retrieval augmented generation. That’s destruction before digestion.
The Flat Text Fallacy
The assumption underlying most standard chunking strategies is that proximity equals context. If two sentences are next to each other, they must be related. Split at 512 tokens and call it a day.
But Markdown doesn’t work like that. A table row embedded in a document is completely meaningless when severed from its header row above it and the explanatory prose that introduces it. A code block is noise without the H3 heading that names the function it implements.
When you chunk Markdown by character count, you are not preserving context. You are fracturing it.
Your agents are not reading documents. They are reading shards of documents, and they are guessing at what the surrounding context was.
Stop Parsing. Start Compiling.
For the last decade, programmers have known that you don’t compile a C program by splitting the source file at arbitrary byte offsets and hoping the semantics survive. You parse it into an Abstract Syntax Tree (AST), and every subsequent transformation respects the tree structure.
The same principle applies to Markdown for LLMs.
Markdown is assembly for AI agents. It is the low-level instruction set that dictates how context is structured, scoped, and related within a language model’s working memory. Headings are scope boundaries. Lists are structured data. Code blocks are typed payloads.
If that’s true — and it is — then we need to stop parsing Markdown and start compiling it.
The Structural Compiler Pattern
A proper Markdown Compiler for AI pipelines works like this:
- Build the AST. Parse the document into a full tree: headings, paragraphs, lists, tables, code blocks, and their parent-child relationships.
- Chunk the tree, not the string. When you need to embed a passage, you traverse the tree. Every chunk carries a pointer to its parent heading, its sibling tables, and its foundational context. The chunk is semantically complete.
- Embed structured context. The embedding isn’t just the chunk text. It includes the document title, the parent H2, the immediate H3, and the chunk type. The embedding vector actually represents the position in the knowledge structure, not just the raw words.
The result? An agent that doesn’t just read words. An agent that navigates relationships and can reason about where a piece of information sits within the larger knowledge base.
Until your data pipeline respects the AST, your agents will remain structurally blind. Fix your pipeline. Compile, don’t just parse.
Production-Ready P2P: How We Hardened Traylinx Stargate for the Real World
