Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/swift-powershell.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tanstack/highlight': minor
---

Add isolated Swift and PowerShell language definitions, default registration, and idiomatic documentation showcases. Support Swift nested comments and raw delimiters and PowerShell quoting, here-strings, variables, cmdlets, and operators.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ Available themes: Aurora X, Dracula, GitHub Dark, GitHub Light, Gruvbox Dark, Gr

## Languages

`apache`, `cmake`, `cpp`, `css`, `diff`, `dockerfile`, `ejs`, `env`, `go`, `html`, `http`, `js`, `json`, `jsx`, `markdown`, `mermaid`, `nginx`, `php`, `plaintext`, `python`, `scheme`, `shell`, `sql`, `svelte`, `toml`, `ts`, `tsrx`, `tsx`, `vue`, and `yaml`.
`apache`, `cmake`, `cpp`, `css`, `diff`, `dockerfile`, `ejs`, `env`, `go`, `html`, `http`, `js`, `json`, `jsx`, `markdown`, `mermaid`, `nginx`, `php`, `plaintext`, `powershell`, `python`, `scheme`, `shell`, `sql`, `svelte`, `swift`, `toml`, `ts`, `tsrx`, `tsx`, `vue`, and `yaml`.

Each language is available from `@tanstack/highlight/languages/<name>`. The aggregate `@tanstack/highlight/languages` entry can tree-shake, while direct subpaths make isolation explicit. Importing only core helpers from the root entry also removes unused language registrations in a compatible bundler.

Expand All @@ -215,10 +215,10 @@ Local browser bundles, minified with esbuild and compressed independently. KB us
| Registration | Minified | Gzip | Brotli |
| --- | ---: | ---: | ---: |
| Core, no languages | 3.84 KB | 1.82 KB | 1.66 KB |
| Core + TSX | 10.11 KB | 4.26 KB | 3.90 KB |
| Octane MDX + TypeScript | 13.86 KB | 5.59 KB | 5.17 KB |
| Nine-language docs set | 16.04 KB | 6.20 KB | 5.66 KB |
| All 30 languages | 30.72 KB | 10.77 KB | 9.79 KB |
| Core + TSX | 10.16 KB | 4.29 KB | 3.93 KB |
| Octane MDX + TypeScript | 13.91 KB | 5.62 KB | 5.21 KB |
| Nine-language docs set | 16.09 KB | 6.22 KB | 5.68 KB |
| All 32 languages | 36.17 KB | 12.45 KB | 11.27 KB |

The following comparison was measured before the 1.0 property-context correction. Re-run the comparison commands below for current timings.

Expand Down
Binary file added docs/assets/swift-powershell/powershell-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/swift-powershell/swift-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/swift-powershell/swift-light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/swift-powershell/swift-mobile.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
{ "label": "React Integration", "to": "guides/react" },
{ "label": "Octane Integration", "to": "guides/octane" },
{ "label": "Custom Languages", "to": "guides/custom-languages" },
{ "label": "Swift and PowerShell", "to": "guides/swift-and-powershell" },
{ "label": "Bundle Size and Performance", "to": "guides/performance" }
]
},
Expand Down
20 changes: 12 additions & 8 deletions docs/guides/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,29 @@ CI measures selective browser bundles and highlighting performance on real docum

## Bundle profiles

`pnpm run size` builds seventeen browser profiles with esbuild and measures minified, gzip, and Brotli bytes independently. It also checks that helper, adapter, and selective language imports retain only the requested modules.
`pnpm run size` builds nineteen browser profiles with esbuild and measures minified, gzip, and Brotli bytes independently. It also checks that helper, adapter, and selective language imports retain only the requested modules.

| Profile | Registered languages | Current gzip | CI budget |
| --- | --- | ---: | ---: |
| Core | None | 1.82 KB | 2.0 KB |
| TSX | TSX | 4.26 KB | 4.35 KB |
| Octane | TypeScript plus Octane MDX adapter | 5.59 KB | 5.7 KB |
| Docs | CSS, HTML, JS, JSON, JSX, Markdown, Shell, TS, TSX | 6.20 KB | 6.3 KB |
| All | All 30 definitions | 10.77 KB | 10.9 KB |
| TSX | TSX | 4.29 KB | 4.35 KB |
| Octane | TypeScript plus Octane MDX adapter | 5.62 KB | 5.7 KB |
| Docs | CSS, HTML, JS, JSON, JSX, Markdown, Shell, TS, TSX | 6.22 KB | 6.3 KB |
| Swift | Swift | 3.38 KB | 3.55 KB |
| PowerShell | PowerShell | 3.27 KB | 3.55 KB |
| All | All 32 definitions | 12.45 KB | 12.7 KB |

KB uses 1,000 bytes. Core helpers imported from the root tree-shake to the same engine size. The standalone theme helper is 695 gzip bytes.
KB uses 1,000 bytes. Core helpers imported from the root tree-shake to the same engine size. The standalone theme helper is 691 gzip bytes.

Adding Swift and PowerShell raises the all-language gzip bundle from 10.77 KB to 12.45 KB. Their isolated bundles are 3.38 KB and 3.27 KB gzip including core; existing selective budgets are unchanged.

Selective profiles are the primary metric. The all-language profile exists to prevent convenience-entry growth from becoming invisible.

## Runtime corpus

The committed corpus contains 334 real code fences sampled from TanStack documentation, with up to twenty samples per normalized language.

`pnpm run bench` measures tokenization, HTML, Markdown, HAST, line numbers, long decorated blocks, and dedicated C++, CMake, and PHP samples. Each profile reports the median of three samples after two warmup passes, with a 1.2 second CI budget. The main highlighting profile processes at least 10,000 blocks.
`pnpm run bench` measures tokenization, HTML, Markdown, HAST, line numbers, long decorated blocks, and dedicated C++, CMake, PHP, Swift, and PowerShell samples. Each profile reports the median of three samples after two warmup passes, with a 1.2 second CI budget. The main highlighting profile processes at least 10,000 blocks.

A local before-and-after review used the same minified bundle settings, fixtures, and benchmark harness on macOS arm64 with Node 24.15.0:

Expand Down Expand Up @@ -76,4 +80,4 @@ Context-aware fixes are welcome when they solve common docs code. A change shoul

The correct response to a crossed budget is to inspect the behavior and architecture. Budgets can move when a measured quality improvement justifies the bytes, but the tradeoff must be explicit.

The 1.0 property-context correction adds roughly 230 gzip bytes to the TSX profile without changing core. Local Node 26 gzip results differ slightly from CI compression: the CI docs profile is 6,217 bytes and all languages is 10,838 bytes. Their budgets are 6,300 and 10,900 bytes respectively, retaining a small explicit margin.
The 1.0 property-context correction adds roughly 230 gzip bytes to the TSX profile without changing core. Current local Node 26.10.0 measurements are 6,220 gzip bytes for the docs profile and 12,449 bytes for all languages. Their CI budgets are 6,300 and 12,700 bytes respectively, retaining a small explicit margin. CI measurements may differ slightly with the compression runtime.
140 changes: 140 additions & 0 deletions docs/guides/swift-and-powershell.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
---
title: Swift and PowerShell
---

# Swift and PowerShell

Two views of an observatory: Swift keeps concurrent observations behind an actor; PowerShell turns them into a typed pipeline. The examples deliberately mix declarations, metadata, raw text, numeric literals, and control flow so the shipped themes have a rich palette to work with.

Register only the languages you need:

```ts
import { createHighlighter } from '@tanstack/highlight/core'
import { swift } from '@tanstack/highlight/languages/swift'
import { powershell } from '@tanstack/highlight/languages/powershell'

const highlighter = createHighlighter({ languages: [swift, powershell] })
```

Use `swift`, `powershell`, `pwsh`, or `ps1` as fence tags. The default entry registers both definitions. See [Themes](themes) to switch colors without changing markup.

## Swift: concurrent observations

Nested comments, actor isolation, `async let`, closure parameters, key paths, an extended regex, and exact raw-string delimiters share one block. The example illustrates highlighting; the API URL is a placeholder.

```swift
import Foundation

/* An actor owns the cache.
/* Nested notes stay inside this comment. */
No locks escape into the view. */
struct Reading: Sendable, Codable {
let station: String
let celsius: Double
var fahrenheit: Double { celsius * 1.8 + 32 }
}

actor Observatory {
private var cache: [String: Reading] = [:]

func reading(for station: String) async throws -> Reading {
if let saved = cache[station] { return saved }
let url = URL(string: "https://example.com/observations/\(station)")!
let (data, _) = try await URLSession.shared.data(from: url)
let reading = try JSONDecoder().decode(Reading.self, from: data)
cache[station] = reading
return reading
}
}

@MainActor
func forecast() async throws {
let observatory = Observatory()
async let coast = observatory.reading(for: "coast")
async let ridge = observatory.reading(for: "ridge")
let readings = try await [coast, ridge]
let warm = readings.filter { $0.celsius > 0 }.map(\.station)
let scale = 0x1.fp+2
let mask: UInt8 = 0b1111_0000
let band = 1..<4
let pattern = #/^(?<station>[a-z]+):\s+(?<value>-?\d+\.\d+)$/#
let legend = ##"A literal \n; an interpolated value: \##(scale)"##
let report = """
Stations: \(warm.joined(separator: ", "))
Calibrated: \(scale), mask: \(mask), band: \(band)
"""
print(report, legend, pattern)
}

#if DEBUG
let preview = Reading(station: "coast", celsius: 18.5)
#endif
```

## PowerShell: a report pipeline

An advanced function combines attributes, splatting, scoped variables, typed records, here-strings, null coalescing, and case-insensitive word operators. The API URL is a placeholder.

```powershell
#Requires -Version 7.4
<# Build typed records, then shape a pipeline for display.
Quotes and $variables in this comment stay quiet. #>
function Get-StationReport {
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[ValidateSet('coast', 'ridge')]
[string[]] $Station
)

begin {
$script:Endpoint = $env:STATION_API ?? 'https://example.com'
$headers = @{ Accept = 'application/json' }
$budget = 64MB
}
process {
foreach ($name in $Station) {
$request = @{
Uri = "$script:Endpoint/observations/$name"
Headers = $headers
ErrorAction = 'Stop'
}
try {
$reading = Invoke-RestMethod @request
[pscustomobject]@{
Station = $name
Celsius = [double] $reading.celsius
Online = $true
}
}
catch {
Write-Warning -Message "Station $name unavailable: $_"
}
}
}
end {
$banner = @"
Forecast ready
Budget: $budget
"@
$literal = @'
$HOME is literal; # this is text, not a comment.
'@
Write-Verbose -Message ($banner + $literal)
}
}

${report-title} = 'Today''s observatory'
Get-StationReport -Station coast, ridge |
Where-Object { $_.Online -and $_.Celsius -ge 0 } |
Sort-Object -Property Celsius -Descending |
Select-Object -First 5 -Property Station, Celsius
```

## Preview and scope

Run `pnpm run report:compare` in the repository and open `artifacts/shiki-comparison.html`. These two showcases appear first, in GitHub Light and Aurora X, alongside Shiki's grammar-based reference. Their canonical sources are `test/showcases/Observatory.swift` and `test/showcases/Get-StationReport.ps1`.

Interpolation is kept inside the string token. Bare Swift regex literals, arbitrary custom operators, symbol resolution, and full PowerShell command/argument disambiguation are outside the lightweight tokenizer's scope.

Lexical rules follow [The Swift Programming Language](https://docs.swift.org/swift-book/documentation/the-swift-programming-language/lexicalstructure/) and PowerShell's [quoting rules](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules) and [language specification](https://learn.microsoft.com/en-us/powershell/scripting/lang-spec/chapter-02).
2 changes: 2 additions & 0 deletions docs/language-support.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,13 @@ Every language is an isolated definition imported from `@tanstack/highlight/lang
| Nginx | `nginx` | - | Directives, variables, URLs, comments |
| PHP | `php` | - | PHP tags, attributes, quoted strings, heredoc/nowdoc, optional HTML delegation |
| Plaintext | `plaintext` | `text`, `txt`, `-->` | Escaping only |
| PowerShell | `powershell` | `pwsh`, `ps1` | Here-strings, escaped quotes, variables, cmdlets, type literals, word operators |
| Python | `python` | `py` | Triple strings, prefixes, decorators, comments |
| Scheme | `scheme` | `scm`, `racket` | Comments, strings, forms, literals |
| Shell | `shell` | `bash`, `sh`, `zsh`, `cmd`, `console` | Heredocs, parameter expansion, comment boundaries |
| SQL | `sql` | - | Strings, comments, common SQL clauses |
| Svelte | `svelte` | - | Markup plus optional script/style and expression delegation |
| Swift | `swift` | - | Nested comments, raw/multiline strings, extended regex delimiters, attributes, concurrency keywords |
| TOML | `toml` | - | Strings, comments, tables, properties |
| TypeScript | `ts` | `typescript`, `angular-ts` | JavaScript scanner plus TypeScript keywords/types |
| TSRX | `tsrx` | `octane` | TypeScript, contextual JSX, Octane component shorthand and template directives |
Expand Down
6 changes: 5 additions & 1 deletion docs/reference/languages.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,18 +40,22 @@ const highlighter = createHighlighter({
| `nginx` | `@tanstack/highlight/languages/nginx` | None |
| `php` | `@tanstack/highlight/languages/php` | None |
| `plaintext` | `@tanstack/highlight/languages/plaintext` | `text`, `txt`, `-->` |
| `powershell` | `@tanstack/highlight/languages/powershell` | `pwsh`, `ps1` |
| `python` | `@tanstack/highlight/languages/python` | `py` |
| `scheme` | `@tanstack/highlight/languages/scheme` | `scm`, `racket` |
| `shell` | `@tanstack/highlight/languages/shell` | `bash`, `sh`, `zsh`, `cmd`, `console` |
| `sql` | `@tanstack/highlight/languages/sql` | None |
| `svelte` | `@tanstack/highlight/languages/svelte` | None |
| `swift` | `@tanstack/highlight/languages/swift` | None |
| `toml` | `@tanstack/highlight/languages/toml` | None |
| `ts` | `@tanstack/highlight/languages/ts` | `typescript`, `angular-ts` |
| `tsrx` | `@tanstack/highlight/languages/tsrx` | `octane` |
| `tsx` | `@tanstack/highlight/languages/tsx` | None |
| `vue` | `@tanstack/highlight/languages/vue` | None |
| `yaml` | `@tanstack/highlight/languages/yaml` | `yml` |

`@tanstack/highlight/languages` re-exports `apache`, `cmake`, `cpp`, `css`, `diff`, `dockerfile`, `ejs`, `env`, `go`, `html`, `http`, `js`, `json`, `jsx`, `markdown`, `mermaid`, `nginx`, `php`, `plaintext`, `python`, `scheme`, `shell`, `sql`, `svelte`, `toml`, `ts`, `tsrx`, `tsx`, `vue`, and `yaml`. The barrel is convenient but individual subpaths make bundle intent explicit.
`@tanstack/highlight/languages` re-exports `apache`, `cmake`, `cpp`, `css`, `diff`, `dockerfile`, `ejs`, `env`, `go`, `html`, `http`, `js`, `json`, `jsx`, `markdown`, `mermaid`, `nginx`, `php`, `plaintext`, `powershell`, `python`, `scheme`, `shell`, `sql`, `svelte`, `swift`, `toml`, `ts`, `tsrx`, `tsx`, `vue`, and `yaml`. The barrel is convenient but individual subpaths make bundle intent explicit.

See the [language support matrix](../language-support) for the context-aware behavior and current scope of each registration.

Swift supports nested comments, raw and multiline strings, extended regex literals (`#/…/#`), attributes, concurrency keywords, numeric bases, and common types. PowerShell supports comments, quoted and here-strings, scoped/braced/splat variables, cmdlet names, parameters, type literals, and word operators. String interpolation stays within the string token in both languages; these lightweight definitions do not parse expressions or resolve symbols. Bare Swift regex literals and full PowerShell command/argument context are outside their scope.
2 changes: 1 addition & 1 deletion docs/test-strategy.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ The suite protects the package's actual product boundary: valid code commonly pu
| TSX | TSX | 4.35 KB |
| Octane | TypeScript plus Octane MDX adapter | 5.7 KB |
| Docs | CSS, HTML, JS, JSON, JSX, Markdown, Shell, TS, TSX | 6.3 KB |
| All | All 30 definitions | 10.9 KB |
| All | All 32 definitions | 12.7 KB |

The selective profiles are the primary product metric. The all-language profile protects the convenience entry from unbounded growth. Bundle graphs reject unexpected language or theme code. Package tests repeat isolation checks through public exports after building.

Expand Down
8 changes: 8 additions & 0 deletions scripts/bench.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,14 @@ try {
outputBytes: htmlBytes,
targetBlocks: 10_000,
},
...Object.fromEntries([
['swift', 'Observatory.swift'], ['powershell', 'Get-StationReport.ps1'],
].map(([rawLang, file]) => [rawLang, {
fixtures: [{ rawLang, code: fs.readFileSync(`test/showcases/${file}`, 'utf8') }],
run: (fixture) => highlight(fixture.code, { lang: fixture.rawLang }),
observe: (result) => result.html.length,
targetBlocks: 2_000,
}])),
cpp: {
fixtures: [{ rawLang: 'cpp', code: '#include <vector>\nconstexpr auto text = R"tag(// raw text)tag";\nint main() { std::vector<int> values{1, 2, 3}; return values.size(); }' }],
run: (fixture) => highlight(fixture.code, { lang: fixture.rawLang }),
Expand Down
8 changes: 7 additions & 1 deletion scripts/generate-visual-compare.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,13 @@ import { githubLightTheme } from '../dist/themes/github-light.js'
const fixtureFile = 'test/generated/tanstack-doc-fixtures.json'
const outFile = process.argv[2] || 'artifacts/shiki-comparison.html'
const fixtureData = JSON.parse(fs.readFileSync(fixtureFile, 'utf8'))
const fixtures = selectFixtures(fixtureData.fixtures)
const fixtures = selectFixtures([
...[['swift', 'Observatory.swift'], ['powershell', 'Get-StationReport.ps1']].map(([lang, file]) => ({
lang, rawLang: lang, file: `test/showcases/${file}`, line: 1,
code: fs.readFileSync(`test/showcases/${file}`, 'utf8'),
})),
...fixtureData.fixtures,
])
const shikiHighlighter = await shiki.createHighlighter({
themes: ['github-light', 'aurora-x'],
langs: ['plaintext'],
Expand Down
7 changes: 6 additions & 1 deletion scripts/measure-size.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -98,13 +98,18 @@ const profiles = {
languages: ['cmake'],
limits: { minified: 6_000, gzip: 2_800, brotli: 2_600 },
},
...Object.fromEntries(['swift', 'powershell'].map((language) => [language, {
source: `import { createHighlighter } from './src/core.ts'; import { ${language} } from './src/languages/${language}.ts'; globalThis.highlighter = createHighlighter({ languages: [${language}] })`,
languages: [language],
limits: { minified: 7_700, gzip: 3_550, brotli: 3_300 },
}])),
all: {
source: `
import { defaultHighlighter } from './src/index.ts'
globalThis.highlighter = defaultHighlighter
`,
languages: 'all',
limits: { minified: 30_800, gzip: 10_900, brotli: 9_800 },
limits: { minified: 36_400, gzip: 12_700, brotli: 11_600 },
},
reactAdapter: {
source: `export * from './src/react.ts'`,
Expand Down
7 changes: 7 additions & 0 deletions scripts/test-package.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,13 @@ const isolatedBundles = [
source: `import { createHighlighter } from '@tanstack/highlight/core'; import { tsx } from '${entry}'; globalThis.highlighter = createHighlighter({ languages: [tsx] })`,
languages: ['tsx'],
})),
...['swift', 'powershell'].flatMap((language) => [
`@tanstack/highlight/languages/${language}`, '@tanstack/highlight/languages',
].map((entry) => ({
name: `${entry} ${language}`,
source: `import { createHighlighter } from '@tanstack/highlight/core'; import { ${language} } from '${entry}'; globalThis.highlighter = createHighlighter({ languages: [${language}] })`,
languages: [language],
}))),
...['react', 'markdown', 'remark', 'rehype', 'octane'].map((entry) => ({
name: `${entry} adapter`,
source: `export * from '@tanstack/highlight/${entry}'`,
Expand Down
Loading