ADR-0022 Syntax highlighting

Context

Code shows up in three places: fenced blocks inside rendered Markdown (ADR-0021) — mostly model replies (ADR-0025) and user text — and standalone snippets such as copyable commands or JSON samples. All of it should be colored, on the server as well as the client.

Surrounding decisions:

• SSR: TanStack Start (ADR-0002) sends HTML first. Colored code should be in that HTML, with no flash of plain text, and hydrate to the same markup
• Styling: colors come from CSS variables with light and dark values, the way shadcn/ui tokens work (ADR-0011); no runtime style injection on the render path
• Markdown: TanStack Markdown takes the highlighter as a synchronous callback and inserts its return value into <code> as trusted HTML — it cannot tell token markup from an injection (ADR-0021)

What is open: which highlighter, which languages, how colors map to our tokens, and what happens with huge or unknown code.

Constraints:

• The output is trusted HTML. A highlighter that fails to escape source text turns every code block into an XSS hole — and the code comes from models and users
• Synchronous. The callback runs during render, on the server and again on the client. An async highlighter has no place there
• Bundle: the browser should receive only the languages we use
• Theming: light/dark from our variables, one markup tree, no inline colors
• Streaming: an open fence is re-highlighted on every update (ADR-0021), so cost must stay small
• Unknown input: an unknown language or an invalid snippet must degrade to plain text, not throw

Options considered

Highlighter

1. TanStack Highlight

@tanstack/highlight 1.1.0 (2026-10-08), MIT, ESM, no dependencies, Node 18+.

Pros

• Synchronous, no initialization. No WASM, no worker, no promise; the same registrations return the same output on the server and in the browser (we ran it twice: identical)
• Selective by construction. @tanstack/highlight/core ships no languages; each is its own import. Maintainers’ gzip numbers: core 1.82 KB, core + TSX 4.26 KB, a nine-language docs set 6.20 KB, all 39 languages 17.73 KB
• Escaped output, no inline styles. One <pre><code> tree with stable th-* token classes and no inline colors. We ran a fence containing <img src=x onerror=…>, one with a hostile language string, and an unknown language: all came out escaped, with unknown languages as plain text
• Theming is CSS variables on one markup tree. Colors are --th-* variables, so light and dark is a CSS switch with no second markup and no second highlight pass
• First-party Markdown adapter (createTanStackMarkdownHighlighter), plus remark, rehype, and fence metadata: title, line numbers, {2,4-6}, ins and del, line and range decorations
• Fast on docs-sized code. Maintainers measured 4.6 ms for 334 docs fixtures warmed, against 182 ms for Shiki (taken before 1.0, and not equal work: Shiki parses far deeper). Our own check (Node 26, one laptop, not a benchmark): about 0.14 ms per KiB
• Compatibility policy for 1.x: entry points, the meaning of the th-* classes, and the wrapper structure stay stable; new languages and themes are minor releases. 1.1.0 added Java, Kotlin, Rust, Ruby, C#, Dart, Lua, Perl, and Swift
• Same organization as Router, Query, and Start

Cons

• A small docs highlighter, not a grammar engine. It is built for valid code found in documentation. No TextMate or VS Code grammars, no automatic language detection, no semantic tokens; unusual or invalid code can be colored wrong
• 39 languages. No GraphQL, PowerShell, or Terraform/HCL, and c is not an alias of cpp — a c fence is plain text. An unregistered language, or an alias whose target is not registered, falls back to plain text
• Young. First npm release 2026-06-21, 1.0.0 on 2026-09-30. Parser corrections ship as patch releases and may change which class a token gets, so exact HTML is not frozen
• Themes are color objects only: no font weight, italics, or contrast checks. Only 11 ship, and none uses our tokens
• Synchronous cost grows with block size; its docs say unbounded input needs a limit of our own

2. Shiki

shiki 4.5.0 (2026-10-01), MIT, TextMate grammars — the engine behind VS Code highlighting.

Pros

• The most accurate: real grammars, a very large language and theme set, VS Code theme compatibility, transformers, a rehype plugin
• Dual light/dark themes through CSS variables (--shiki-light, --shiki-dark); fine-grained bundles; a JavaScript regex engine as an alternative to WASM
• Active, and the highlighter behind Streamdown’s code plugin

Cons

• Heavy by default. The pre-built bundles are 6.4 MB minified (1.2 MB gzip) for shiki/bundle/full and 3.8 MB (695 KB gzip) for shiki/bundle/web, async chunks included; browser use needs the fine-grained entries
• Async setup. await createHighlighter() with Oniguruma compiled to WASM is the default path. A synchronous core exists (createHighlighterCoreSync, since 1.16) but needs languages and themes as plain objects plus an explicit JavaScript engine. A Markdown callback is synchronous, so this is extra wiring
• Inline styles on every token (style="--shiki-dark:…;--shiki-light:…"), and with the default dual-theme output dark mode needs !important overrides unless defaultColor is false. In the maintainers’ comparison it produced 3.4x more HTML for the same docs
• No first-party TanStack Markdown adapter; we would write the callback and own its escaping

3. Prism / prism-react-renderer

prismjs 1.30.0 (2025-03-10), prism-react-renderer 2.4.1 (2025-08-14), both MIT.

Pros: mature and widely used (Docusaurus uses prism-react-renderer); synchronous; class-based tokens with CSS themes; modular languages. Cons: prismjs releases are slow (1.29.0 in 2022-08, then 1.30.0 in 2025-03); prism-react-renderer bundles a modified Prism with a short default language list and its own theme format; no Markdown adapter for TanStack Markdown, so the callback and its escaping are ours.

4. highlight.js

highlight.js 11.12.0 (2026-08-12), BSD-3-Clause, no dependencies.

Pros: 193 languages and 516 themes; automatic language detection; synchronous; class-based. Cons: the largest core of the small options (29.54 KB minified, 11.12 KB gzip for TypeScript only, in Sugar High’s benchmark). Automatic detection is wasted work — fences name their language — and a wrong guess colors code as the wrong language. No Markdown adapter.

5. Sugar High

sugar-high 2.6.1 (2026-10-19), MIT, no dependencies.

Pros: tiny (4.35 KB gzip for TypeScript in its own benchmark); synchronous; 32 languages in v2; token colors are CSS custom properties; @sugar-high/react and @sugar-high/remark exist. Cons: its documented theme has nine color variables (Highlight has 21 token classes), so coloring is coarser; no first-party TanStack Markdown adapter. TanStack’s comparison measured Sugar High 1.2.1 (so 78 ms against 377 ms and 6.7 MiB against 44.8 MiB of HTML is stale for 2.x). It is the closest alternative and our fallback if Highlight stalls.

Theme source

A. Our own CSS variables, written once in app.css

Highlight reads --th-background, --th-token, and one --th-<class> variable per token class. We define them under :root and .dark, as shadcn/ui does.

Pros

• Colors follow our tokens: dark mode and per-product branding swap variables, not code (ADR-0011)
• No <style> built at render time; the CSS is in the one static stylesheet
• Contrast is checked by us, per mode

Cons

• We write and review 22 variables (21 token classes plus the background) in two modes, and copy the token rules once

B. Packaged themes through createThemeCss

createThemeCss({ light, dark }) with githubLightTheme and githubDarkTheme (11 themes ship).

Pros: nothing to write; GitHub-style palettes. Cons: fixed hex colors unrelated to our tokens, so a brand change means a code change. The React examples in its docs inject the result as a <style> element inside a component, against ADR-0011’s no-runtime-style rule. Themes carry no contrast validation, and the theme objects are trusted configuration, not sanitized CSS.

Decision

TanStack Highlight highlights all code in our apps, with an explicit language list and colors from our own CSS variables. This is the company standard for all new apps.

Concretely:

• Package: @tanstack/highlight 1.x. Import only /core, /languages/<name>, /markdown, and /theme. The root entry’s highlight() and the aggregate /languages entry pull the all-language build (about 18 KB gzip); don’t use them in app code. Upgrade it together with @tanstack/markdown (ADR-0021)

• One module per app, src/lib/highlight.ts, isomorphic with no server-only imports. It registers the languages, builds the highlighter once, wraps it for Markdown, and caps the work:

  src/lib/highlight.ts

  import { createHighlighter, escapeHtml } from "@tanstack/highlight/core";
  import { css } from "@tanstack/highlight/languages/css";
  import { diff } from "@tanstack/highlight/languages/diff";
  import { html } from "@tanstack/highlight/languages/html";
  import { js } from "@tanstack/highlight/languages/js";
  import { json } from "@tanstack/highlight/languages/json";
  import { jsx } from "@tanstack/highlight/languages/jsx";
  import { plaintext } from "@tanstack/highlight/languages/plaintext";
  import { shell } from "@tanstack/highlight/languages/shell";
  import { sql } from "@tanstack/highlight/languages/sql";
  import { ts } from "@tanstack/highlight/languages/ts";
  import { tsx } from "@tanstack/highlight/languages/tsx";
  import { yaml } from "@tanstack/highlight/languages/yaml";
  import { createTanStackMarkdownHighlighter } from "@tanstack/highlight/markdown";
  import type { CodeHighlighter } from "@tanstack/markdown";
  
  export const highlighter = createHighlighter({
    languages: [plaintext, js, jsx, ts, tsx, json, html, css, shell, diff, yaml, sql],
  });
  
  export const MAX_HIGHLIGHT_CHARS = 20_000; // our limit, not the library's
  
  const highlightFence = createTanStackMarkdownHighlighter(highlighter);
  
  export const highlightMarkdownCode: CodeHighlighter = (code, lang, options) =>
    code.length > MAX_HIGHLIGHT_CHARS ? escapeHtml(code) : highlightFence(code, lang, options);

  A module-level highlighter is fine here, unlike a QueryClient (ADR-0012): it holds only the language registry and no per-user data. Adding a language is one import and one array entry; products extend the list, nothing else changes

• Language list. Register html together with js and css: Highlight then colors <script> and <style> bodies, and only registered languages do. Aliases (typescript, sh, jsonc) resolve only when their target is registered. Leave the fallback at plaintext; unknown fences render as escaped text, and there is no auto-detection to turn on

• Escaping is non-negotiable. Markdown gets the highlighter only through createTanStackMarkdownHighlighter, because it inserts the returned string as trusted HTML. Never write a callback that returns source text unescaped or builds markup by string concatenation. Standalone snippets use one CodeBlock component that renders highlighter.highlight(code, { lang }).html — the only place besides Markdown’s callback where highlight output becomes HTML — and falls back to a plain React <pre><code> above MAX_HIGHLIGHT_CHARS

• SSR and hydration: highlighting runs during render, on the server and again on the client, from the same module and the same registrations, so the markup is identical. No useEffect, no lazy or client-only highlighting, no separate server and client language lists

• Theme: our variables, in src/styles/app.css. Define the --th-* variables under :root and .dark, take the neutrals from shadcn/ui tokens, and style pre ourselves. Never call createThemeCss or render a <style> element for this:

  src/styles/app.css

  :root {
    --th-background: var(--muted);
    --th-token: var(--foreground);
    --th-comment: var(--muted-foreground);
    --th-keyword: oklch(…); /* one variable per token class, 21 in all */
  }
  .dark {
    --th-keyword: oklch(…); /* the same names, dark values */
  }
  
  .markdown-renderer pre.tm-code,
  pre.th-code {
    overflow-x: auto;
    padding: 1rem;
    background: var(--th-background);
    color: var(--th-token);
  }
  .th-token {
    color: var(--th-token);
  } /* first: tokens carry th-token plus their own class */
  .th-keyword {
    color: var(--th-keyword);
  } /* one rule per class */
  .th-line--highlighted {
    background: color-mix(in srgb, var(--th-token) 10%, transparent);
  }

  The 21 classes are token, attr, code-inline, command, comment, deleted, function, heading, inserted, keyword, link, literal, meta, number, operator, property, selector, string, tag, type, and variable. Start from githubLightTheme and githubDarkTheme, convert the values once to oklch, and check every color for at least 4.5:1 against --th-background in both modes — Highlight’s themes carry no contrast checks. Dark mode is the .dark class and needs no second highlight pass: light and dark share the same markup. .markdown-renderer is the wrapper class from ADR-0021, and pre.tm-code is the <pre> Markdown owns. Line numbers are off; a product that turns them on adds the rules from createThemeBaseCss()

• Drift guard: a bun test test checks that every name in themeTokenClasses (exported from @tanstack/highlight/theme) has a --th-<name> variable in app.css. A new class would arrive only with a major version; the test makes it a failing check instead of uncolored code

• Streaming: Markdown re-renders the accumulated text and Highlight re-colors the open fence on each update. Cost grows linearly with block size: our local check measured about 2 ms for 16 KiB and 9 ms for 64 KiB, and a 16 KiB fence streamed in 32-character updates averaged 0.9 ms per update including Markdown parsing. Throttle updates (ADR-0023); blocks over MAX_HIGHLIGHT_CHARS stay plain text, because Highlight’s docs say unbounded input needs a limit of our own

• Fence metadata is data: title, {2,4-6}, line annotations, and lineNumbers are read as text and never evaluated. Check the language against our own list before using it for anything else

• Tests (ADR-0014): assert that source text survives, that <img onerror> inside a fence comes out escaped, that an unknown language falls back to plain text, and that a known token gets its th-* class. Don’t snapshot the highlighted HTML — patch releases may change token boundaries

The deciding factors: a synchronous, SSR-identical path that fits a Markdown callback; escaped, class-based output with no inline styles that our CSS variables can color in both modes; per-language imports; and a first-party Markdown adapter from the same team. We accept a heuristic tokenizer and a library that is a few weeks past 1.0.

Consequences

Easier:

• Colored code is in the server HTML and hydrates unchanged: no async init, no WASM, no flash of plain text
• One module lists the languages and one CSS block holds the colors; a brand change is a variable edit, and dark mode costs no extra markup
• The browser receives only the languages we import
• Unknown languages and oversized blocks degrade to escaped text; nothing throws
• Escaping happens in one place, the library’s renderer, that we do not hand-write

Harder:

• Accuracy is lower than Shiki’s: the tokenizer is heuristic, so unusual syntax or invalid code can be mis-colored
• Only 39 languages ship. A fence in an unregistered or missing language (GraphQL, Terraform, plain c) renders as plain text, and users see uncolored code
• We own 22 variables in two modes and their contrast; the drift-guard test covers names, not colors
• The selectors are coupled to TanStack Markdown’s pre.tm-code wrapper, and the library is young — read both changelogs on every upgrade, and don’t snapshot HTML
• Standalone code blocks are the one place besides Markdown where we put highlight output into HTML; the CodeBlock component must stay the only one
• Highlighting is synchronous, so the size cap and update throttling are our job; the cap is a number we chose, not a library limit

Given up: Shiki’s grammar accuracy, VS Code theme compatibility, and its very large language set, plus automatic language detection. Accepted — small, synchronous, SSR-identical output with escaped markup matters more for code in chat and comments than editor-grade coloring; a product that needs exact grammars (for example a code viewer) writes an ADR for Shiki.