6 ms·

In general I think I just don't like the idea of one URL being able to return different content. Forces me to think about what each system I give that URL to ma

by k1m 21d ago

In general I think I just don't like the idea of one URL being able to return different content. Forces me to think about what each system I give that URL to may be sending in content negotiation headers. Would rather the HTML is returned and alternatives listed in HTML head. But for HTML and Markdown in particular, there's been so much useful work done in the semantic HTML space and microformats, that I don't know why anyone interested in this wouldn't just improve their HTML markup and leave it to the agent to do the rest. Convert to markdown or extract the useful HTML before handing it to model.

troupo 21d ago
> I just don't like the idea of one URL being able to return different content. It's different content representations. A text in a markdown file is conceptually the same content as the same text in HTML (or PDF).
spider-mario 21d ago
It also means you can’t easily know the set of formats that the server could respond with, which in turns makes e.g. archival more difficult.
- 9dev 21d ago
  That entirely depends on the server. A good solution would be to include Link headers in all responses: Link: /some-page rel="canonical" Link: /some-page.json rel="alternate" type="application/json" Link: /some-page.html rel="alternate" type="text/html" Link: /some-page.md rel="alternate" type="text/markdown"
  - iamflimflam1 21d ago
    That only works if the client looks at it. The current Claude fetch system does not.
    - 9dev 21d ago
      Should we let vibe-coded agent harnesses dictate protocol design now..? On the flip side, I'd argue that the current centralisation of user agents (in the classical sense here) that benefit from programmatic content negotiation in form of a handful of harnesses like Claude or Codex is a great lever toward forcing the ecosystem to adopt better practices: If Anthropic added content negotiation as described in this thread to Claude, many sites would be incentivised to improve their web servers.
  - _heimdall 21d ago
    For the LLM use, the challenge is that it will only discover those after first requesting and parsing the HTML version. Maybe it will notice those, and maybe it will figure out the pattern for follow-up page requests, but there's no guarantee and it won't help the first request.
    - 9dev 21d ago
      Not necessarily. They could also send a HEAD request to the URL first, to see the headers only and decide on the available alternates. I am well aware that few sites are taking that much care of their API in terms of HTTP features, but all of the problems discussed here have solid and battle-tested answers.
      - _heimdall 20d ago
        The parent was referring to adding `link` tags in the `<head>`, not in HTTP headers.
        9dev 20d ago
        I realised that after posting, but then I suggested sending them as header in the first place.
_heimdall 21d ago
The ideas is that the URL references the resource and the content type requested is only asking for that content in a different projection or representation. The content at a URL should always match, the format in which its represented can be different based on the request. Its a bit like buying a book in hard copy or paperback, same book different format.

by k1m 21d ago

troupo 21d ago

> I just don't like the idea of one URL being able to return different content. It's different content representations. A text in a markdown file is conceptually the same content as the same text in HTML (or PDF).

spider-mario 21d ago

It also means you can’t easily know the set of formats that the server could respond with, which in turns makes e.g. archival more difficult.

9dev 21d ago
That entirely depends on the server. A good solution would be to include Link headers in all responses: Link: /some-page rel="canonical" Link: /some-page.json rel="alternate" type="application/json" Link: /some-page.html rel="alternate" type="text/html" Link: /some-page.md rel="alternate" type="text/markdown"
- iamflimflam1 21d ago
  That only works if the client looks at it. The current Claude fetch system does not.
  - 9dev 21d ago
    Should we let vibe-coded agent harnesses dictate protocol design now..? On the flip side, I'd argue that the current centralisation of user agents (in the classical sense here) that benefit from programmatic content negotiation in form of a handful of harnesses like Claude or Codex is a great lever toward forcing the ecosystem to adopt better practices: If Anthropic added content negotiation as described in this thread to Claude, many sites would be incentivised to improve their web servers.
- _heimdall 21d ago
  For the LLM use, the challenge is that it will only discover those after first requesting and parsing the HTML version. Maybe it will notice those, and maybe it will figure out the pattern for follow-up page requests, but there's no guarantee and it won't help the first request.
  - 9dev 21d ago
    Not necessarily. They could also send a HEAD request to the URL first, to see the headers only and decide on the available alternates. I am well aware that few sites are taking that much care of their API in terms of HTTP features, but all of the problems discussed here have solid and battle-tested answers.
    - _heimdall 20d ago
      The parent was referring to adding `link` tags in the `<head>`, not in HTTP headers.
      - 9dev 20d ago
        I realised that after posting, but then I suggested sending them as header in the first place.

_heimdall 21d ago

The ideas is that the URL references the resource and the content type requested is only asking for that content in a different projection or representation. The content at a URL should always match, the format in which its represented can be different based on the request. Its a bit like buying a book in hard copy or paperback, same book different format.