Polymorphic Website

Problem

A website is read by browsers, terminals, scripts, crawlers and language-model agents. The usual approach serves HTML and adds exports such as RSS or an API. Each export is a copy of the content, and copies drift.

Approach

Content is there to be accessed, and there is no reason for it not to be available everywhere. Each page is one markdown file, and each interface is a renderer over that file.

flowchart LR
  F[(about.md)] --> N{negotiate}
  N -->|browser| P[Page, HTML, 65,517 bytes]
  N -->|view=canvas| C[Canvas, a world over the page]
  N -->|curl or .txt| T[Terminal, 80 columns, 533 bytes]
  N -->|.md or Accept: text/markdown| A[Agent, markdown, 519 bytes]
  F --> L[llms.txt index]

Solution

One function selects the renderer from the request.

flowchart TB
  R[request] --> S{ends in .md or .txt?}
  S -->|yes| SA[agent or terminal]
  S -->|no| Q{view= in the URL?}
  Q -->|yes| QV[that view, and remember it]
  Q -->|no| K{a remembered view?}
  K -->|yes| KV[that view]
  K -->|no| AC{asks for text/markdown?}
  AC -->|yes| AG[agent]
  AC -->|no| UA{a terminal client?}
  UA -->|yes| TE[terminal]
  UA -->|no| PG[page]

A .md or .txt suffix is checked first. A view chosen with view= is kept in a cookie for a year. The User-Agent check runs last, so curl aesv.io returns text with no flag.

const BY_SUFFIX = { md: 'agent', txt: 'terminal' }
function pick({ suffix, query, cookies, accept, ua }) {
  return BY_SUFFIX[suffix]
      ?? (VIEWS.includes(query.view) && query.view)
      || (VIEWS.includes(cookies.view) && cookies.view)
      || (/text\/markdown/i.test(accept) && 'agent')
      || (TERMINAL_UA.test(ua) && 'terminal')
      || 'page'
}
reply.header('vary', 'Accept, User-Agent, Cookie').send(RENDER[pick(req)](file))

Every response sends Vary: Accept, User-Agent, Cookie, so a shared cache never serves curl's text to a browser.

$ curl aesv.io/about          # terminal: wrapped text, links as numbered notes
$ curl aesv.io/about.md       # agent: the file, with a small YAML head
$ open https://aesv.io/about  # page

Every device offers all four interfaces, except the watch, which offers the page and the canvas. Phones also get the terminal view. It costs nothing to have it. On a first visit, the welcome and the interface chooser open as two dialogs over the home page, so that crawlers and Google see the homepage data.

Outcome

Response sizes for /about by client:

The home page is 6,821 bytes to curl and 93,920 bytes to a browser. Usage per interface was not measured.

Applicability

The design fits any site that people and programs both read, such as documentation, public services and timetables. HTTP content negotiation already serves several representations at one URL.