7 ms·

Is that even true? I most often use HTML. HTML is about 5%-20% more tokens than a similar Markdown. As a rule of thumb, the number of tags/structural tokens d

by honr 21d ago

Is that even true? I most often use HTML. HTML is about 5%-20% more tokens than a similar Markdown. As a rule of thumb, the number of tags/structural tokens doubles, when going from markdown to html, while the rest don't change much. On the other hand, I can view HTML without any extra/unusual tools. And composing HTML when I need a bit of structure is far easier than composing markdown.

slowin 21d ago
> On the other hand, I can view HTML without any extra/unusual tools. And composing HTML when I need a bit of structure is far easier than composing markdown. This is kind of the opposite of reality no? Markdown is just plain text and meant to be human readable. You don't need XML tags to read and write it, opposed to html where you do and you need a browser to properly view it.
- honr 21d ago
  No, it's just that I wasn't very clear. HTML I can view in any browser / webview / etc. Good markdown viewers are fewer / more special, or end up translating md to html for display. And by composing, I didn't mean writing by hand. We are talking about prompting, right? Or that is what I thought we are talking about. Composing HTML "components" into a final prompt HTML is easier than composing markdown snippets into the final prompt. That is because with HTML there are several ergonomic libraries to parse HTML to AST and to format AST back to HTML. The libraries (for parsing to AST and back to strings) are more limited with markdown.
  - sebastiennight 20d ago
    A "prompt" is what goes into the LLM, so I'm not sure what you mean by "a final prompt HTML".
    - honr 20d ago
      I often compose prompts from various sources (my "AGENTS.md", "CURRENT_TASK", "CURRENT_PHASE", ...). Last year I was composing in markdown and that got very tedious very quickly. I tried various text and text-like formats, and it turned out HTML is already one of the best formats for this, including for an "AGENTS.md" (which I keep in HTML despite the required ".md" extension). Of course, if you build up your prompt entirely by hand (copy pasting snippets, etc.) you can get away with markdown or plain text. That works okay for some harnesses, but it leaves too much control (or more accurately, opportunity to misunderstand) to the harness.
xienze 21d ago
That's highly dependent on what sites you're visiting. Take a look around at a lot of modern sites, there's a sea of divs and spans. Markdown conversion helps LLMs a lot.
- honr 21d ago
  Aah, I thought we are talking about prompting or providing information to AI agents in either html or md form, and comparing the two. Assuming that is what we are talking about, HTML is easier to work with than Markdown, unless you are writing it by hand. That is, composing semantic HTML is more ergonomic than composing a Markdown formatted document from components / snippets, programmatically. The libraries are just better and more versatile in most programming languages. Typically you go from HTML or Markdown to AST, then you compose them to end up with the final tree, then you format the tree to HTML or Markdown. LLMs treat them basically identically (context in HTML or context in Markdown), so I have ended up forming complex prompts / context parts using HTML.
SyneRyder 21d ago
> HTML is about 5%-20% more tokens than a similar Markdown. I'm not seeing this at all. I've got a small search engine I made that strips HTML back to Markdown for its full-text indexing. HTML is typically 10x bigger than the Markdown of the actual content, but that's because the majority of HTML out there is truly terrible. I personally like HTML, and my own webpages are all hand-coded HTML. In that case, it's probably a closer ratio to what you describe. I'd suggest it's much higher than 20% more, but it's not likely more than double. But that's assuming someone paying attention to the efficiency of the HTML, and most people / websites just don't. Markdown is even more readable without tools than HTML - it's essentially a plain text document - but I agree that HTML is better for actual semantic structure.
- honr 20d ago
  We are comparing different things. I am comparing expressing information in semantic HTML or in Markdown. HTML doesn't bloat it much, and it is easier to work with (programmatically in python, ts, go, java, etc.; including parsing and composing and formatting). I am not talking about web apps, which commonly have more "app" than "information". And the ratio is really closer to 1.1x tokens than 1.2x, unless the information you are expressing is highly structured. Typically 1 structural element (html tag, markdown marks such as ###) ends up being each 1 token. HTML often has the matching closing tag, and that is really what the difference in number of tokens boils down to. Feel free to run your own tests and compare (I ran a few and it was within 1.09-1.14x).
  - SyneRyder 20d ago
    I'll give a concrete example, but I think we might be talking at cross-purposes. A reasonably randomly chosen news article: https://www.abc.net.au/news/2026-08-27/shania-twain-interview-take-5-zan-rowe/107057510 https://www.abc.net.au/news/2026-08-27/shania-twain-intervie... This HTML page is 360523 bytes of raw HTML. After it has gone through my Markdown parser, it is just 9915 bytes of plain text / Markdown, about a 97% reduction. Only 3% of the HTML is actual content. The point of Accept Markdown is to save web hosts bandwidth. An AI harness can (and already does) download the HTML & parses out a Markdown version so that it is only minimal tokens before it hits the context window. But I still need to download the 360KB of HTML from the server in order to extract the 9KB I actually need. By serving Markdown versions of your page to AI agents, you can save 97% of the bandwidth that AI agents might be incurring. There's no reason this page needs to be 360KB of HTML. A handcoded / handoptimized HTML file might be only 12KB - converting from Markdown to HTML is only going to minimal file increase. But that isn't what the web is. It's full of slop generated by CMS applications & Bootstrap & web frameworks and relics of Frontpage edited WYSIWYG HTML editors. Accept Markdown is trying to get webhosts to save everyone bandwidth by serving the Markdown from their side. Maybe it has a chance if it gets baked in at the web server level, or because Cloudflare is applying it to sites that flow through their network. But I think it's ultimately futile - the same people who don't know their Wordpress output is garbage, also won't know how to configure Markdown on their server.

by honr 21d ago

slowin 21d ago

> On the other hand, I can view HTML without any extra/unusual tools. And composing HTML when I need a bit of structure is far easier than composing markdown. This is kind of the opposite of reality no? Markdown is just plain text and meant to be human readable. You don't need XML tags to read and write it, opposed to html where you do and you need a browser to properly view it.

honr 21d ago
No, it's just that I wasn't very clear. HTML I can view in any browser / webview / etc. Good markdown viewers are fewer / more special, or end up translating md to html for display. And by composing, I didn't mean writing by hand. We are talking about prompting, right? Or that is what I thought we are talking about. Composing HTML "components" into a final prompt HTML is easier than composing markdown snippets into the final prompt. That is because with HTML there are several ergonomic libraries to parse HTML to AST and to format AST back to HTML. The libraries (for parsing to AST and back to strings) are more limited with markdown.
- sebastiennight 20d ago
  A "prompt" is what goes into the LLM, so I'm not sure what you mean by "a final prompt HTML".
  - honr 20d ago
    I often compose prompts from various sources (my "AGENTS.md", "CURRENT_TASK", "CURRENT_PHASE", ...). Last year I was composing in markdown and that got very tedious very quickly. I tried various text and text-like formats, and it turned out HTML is already one of the best formats for this, including for an "AGENTS.md" (which I keep in HTML despite the required ".md" extension). Of course, if you build up your prompt entirely by hand (copy pasting snippets, etc.) you can get away with markdown or plain text. That works okay for some harnesses, but it leaves too much control (or more accurately, opportunity to misunderstand) to the harness.

xienze 21d ago

That's highly dependent on what sites you're visiting. Take a look around at a lot of modern sites, there's a sea of divs and spans. Markdown conversion helps LLMs a lot.

honr 21d ago
Aah, I thought we are talking about prompting or providing information to AI agents in either html or md form, and comparing the two. Assuming that is what we are talking about, HTML is easier to work with than Markdown, unless you are writing it by hand. That is, composing semantic HTML is more ergonomic than composing a Markdown formatted document from components / snippets, programmatically. The libraries are just better and more versatile in most programming languages. Typically you go from HTML or Markdown to AST, then you compose them to end up with the final tree, then you format the tree to HTML or Markdown. LLMs treat them basically identically (context in HTML or context in Markdown), so I have ended up forming complex prompts / context parts using HTML.

SyneRyder 21d ago

> HTML is about 5%-20% more tokens than a similar Markdown. I'm not seeing this at all. I've got a small search engine I made that strips HTML back to Markdown for its full-text indexing. HTML is typically 10x bigger than the Markdown of the actual content, but that's because the majority of HTML out there is truly terrible. I personally like HTML, and my own webpages are all hand-coded HTML. In that case, it's probably a closer ratio to what you describe. I'd suggest it's much higher than 20% more, but it's not likely more than double. But that's assuming someone paying attention to the efficiency of the HTML, and most people / websites just don't. Markdown is even more readable without tools than HTML - it's essentially a plain text document - but I agree that HTML is better for actual semantic structure.

honr 20d ago
We are comparing different things. I am comparing expressing information in semantic HTML or in Markdown. HTML doesn't bloat it much, and it is easier to work with (programmatically in python, ts, go, java, etc.; including parsing and composing and formatting). I am not talking about web apps, which commonly have more "app" than "information". And the ratio is really closer to 1.1x tokens than 1.2x, unless the information you are expressing is highly structured. Typically 1 structural element (html tag, markdown marks such as ###) ends up being each 1 token. HTML often has the matching closing tag, and that is really what the difference in number of tokens boils down to. Feel free to run your own tests and compare (I ran a few and it was within 1.09-1.14x).
- SyneRyder 20d ago
  I'll give a concrete example, but I think we might be talking at cross-purposes. A reasonably randomly chosen news article: https://www.abc.net.au/news/2026-08-27/shania-twain-interview-take-5-zan-rowe/107057510 https://www.abc.net.au/news/2026-08-27/shania-twain-intervie... This HTML page is 360523 bytes of raw HTML. After it has gone through my Markdown parser, it is just 9915 bytes of plain text / Markdown, about a 97% reduction. Only 3% of the HTML is actual content. The point of Accept Markdown is to save web hosts bandwidth. An AI harness can (and already does) download the HTML & parses out a Markdown version so that it is only minimal tokens before it hits the context window. But I still need to download the 360KB of HTML from the server in order to extract the 9KB I actually need. By serving Markdown versions of your page to AI agents, you can save 97% of the bandwidth that AI agents might be incurring. There's no reason this page needs to be 360KB of HTML. A handcoded / handoptimized HTML file might be only 12KB - converting from Markdown to HTML is only going to minimal file increase. But that isn't what the web is. It's full of slop generated by CMS applications & Bootstrap & web frameworks and relics of Frontpage edited WYSIWYG HTML editors. Accept Markdown is trying to get webhosts to save everyone bandwidth by serving the Markdown from their side. Maybe it has a chance if it gets baked in at the web server level, or because Cloudflare is applying it to sites that flow through their network. But I think it's ultimately futile - the same people who don't know their Wordpress output is garbage, also won't know how to configure Markdown on their server.