Introduction

Note

Please see the GitHub README for steps to install and configure the preprocessor. This book covers usage and examples.

This book exercises mdbook-nvim-treesitter. Every code block in it is highlighted by tree-sitter, using the queries nvim-treesitter ships, with the parsers built from the grammar revisions that project pins.

Highlights become CSS classes rather than inline colours. A @keyword.function capture is rendered as:

<span class="ts-keyword ts-keyword-function">fn</span>

so a stylesheet can target ts-keyword for every keyword, or ts-keyword-function for just this one. Run mdbook-nvim-treesitter css to get a starting theme.

Opting out

A single block can be left to mdBook with a no-treesitter tag, which is what you want for a Rust playground example whose hidden lines mdBook strips:

fn hidden() {}
fn main() {
    println!("this block is highlighted by mdBook, not tree-sitter");
}

Overriding a highlight

In some languages, the exact semantic token cannot be determined from the syntax alone, an example is: function-like macros and functions in C have the same syntax.

mdbook-nvim-treesitter allows an inline comment to override the next highlight if this is important to you

void check(int x) {
    /* @tree-sitter:function.macro */ASSERT(x > 0);
    report(x);
}

Outputs:

void check(int x) {
    ASSERT(x > 0);
    report(x);
}

Note the default theme colors them the same way, but you can inspect the class list on the span and verify it is ts-function-macro instead of ts-function-call, compared to the one below which does not have a comment

void check(int x) {
    ASSERT(x > 0);
    report(x);
}

Where code can live

Markdown lets code appear in several places, and each needs slightly different handling to splice highlighted HTML back in without breaking the surrounding structure.

At the top level

The simple case: a fence at column 0.

fn main() {
    let greeting = "hello";
    println!("{greeting}, world");
}

Inside a list

Indented fenced blocks such as the ones inside a list also works:

First, define the function:

def area(radius: float) -> float:
    return 3.14159 * radius ** 2

Then call it:

print(area(2.0))

Even nested two levels deep:
```
int main(void) { return 0; }
```

Inside a block quote

A quoted example:
#[derive(Debug)]
struct Point { x: f64, y: f64 }
Deeply nested block quotes also work
#[derive(Debug)]
struct Point { x: f64, y: f64 }
…and the quote continues afterwards.

Inline

Markdown has no syntax for tagging inline code blocks. We parse each inline block to inspect a language tag before the first backtick (`):

```rust`let x = 1;``` # use triple-backtick to quote the backtick, tagging as rust

This gives: let x = 1;.

It works for any language, so lambda x: x + 1 and static const int N = 8; are highlighted too. Ordinary inline code such as --flag or book.toml is left exactly as it was.

Mixed into a table

Language	Declaration
Rust	`let x: u8 = 1;`
C	`uint8_t x = 1;`
Python	`x: int = 1`

Languages

The pages under this one are a sampler: one per language family, with enough syntax in each to show what the queries pick out. Nothing here is meant to be runnable.

Rust

use std::collections::HashMap;

/// Counts how often each word appears.
///
/// ```
/// assert_eq!(count("a a b")["a"], 2);
/// ```
pub fn count(text: &str) -> HashMap<&str, usize> {
    let mut counts = HashMap::new();
    for word in text.split_whitespace() {
        *counts.entry(word).or_insert(0) += 1;
    }
    counts
}

#[derive(Debug, Clone, PartialEq)]
pub enum Shape {
    Circle { radius: f64 },
    Rect(f64, f64),
}

impl Shape {
    const UNIT: f64 = 1.0;

    pub fn area(&self) -> f64 {
        match self {
            Shape::Circle { radius } => std::f64::consts::PI * radius * radius,
            Shape::Rect(w, h) => w * h,
        }
    }
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let shapes = vec![Shape::Circle { radius: 2.0 }, Shape::Rect(3.0, 4.0)];
    let total: f64 = shapes.iter().map(Shape::area).sum();
    println!("total area: {total:.2}, unit {}", Shape::UNIT);

    // A raw string, an escape, and a byte literal.
    let path = r"C:\Users\example";
    let tab = "a\tb\u{1F600}";
    let byte = b'\n';
    assert!(!path.is_empty() && tab.len() > 2 && byte == 10);

    Ok(())
}

Macros, lifetimes and generics:

macro_rules! square {
    ($x:expr) => {
        $x * $x
    };
}

pub trait Store<'a, T: Clone + 'a> {
    type Error;
    fn get(&'a self, key: &str) -> Result<Option<T>, Self::Error>;
}

pub async fn fetch<S>(store: &S) -> u32
where
    S: for<'a> Store<'a, u32, Error = ()>,
{
    square!(store.get("n").ok().flatten().unwrap_or(0))
}

C and C++

C++’s queries begin with ; inherits: c, so the C patterns have to be loaded and prepended before anything C-shaped in a C++ sample is highlighted.

C

#include <stdio.h>
#include <stdlib.h>

#define MAX_ITEMS 64
#define SQUARE(x) ((x) * (x))

typedef struct Point {
    double x, y;
} Point;

enum Status { OK = 0, FAILED = 1 };

static enum Status sum(const int *values, size_t count, long *out) {
    if (values == NULL || out == NULL) {
        return FAILED;
    }
    long total = 0L;
    for (size_t i = 0; i < count; ++i) {
        total += values[i];
    }
    *out = total;
    return OK;
}

int main(void) {
    int values[] = {1, 2, 3};
    long total = 0;
    /* A block comment, and a character literal. */
    char sep = '\t';
    if (sum(values, 3, &total) != OK) {
        return EXIT_FAILURE;
    }
    printf("total%c%ld squared%c%d\n", sep, total, sep, SQUARE(3));
    return EXIT_SUCCESS;
}

C++

#include <memory>
#include <string>
#include <vector>

namespace geometry {

template <typename T>
class Buffer {
  public:
    explicit Buffer(std::size_t capacity) : m_data(capacity) {}

    [[nodiscard]] auto size() const noexcept -> std::size_t { return m_data.size(); }

    void push(T value) { m_data.push_back(std::move(value)); }

  private:
    std::vector<T> m_data;
};

struct Point final {
    double x{0.0};
    double y{0.0};

    constexpr auto norm2() const -> double { return x * x + y * y; }
};

}  // namespace geometry

int main() {
    using namespace geometry;
    auto buffer = std::make_unique<Buffer<Point>>(4);
    buffer->push(Point{.x = 3.0, .y = 4.0});
    const auto& label = "done";
    return buffer->size() == 1 && label != nullptr ? 0 : 1;
}

Python

"""Word counting, with a docstring the queries pick out separately."""

from __future__ import annotations

import re
from dataclasses import dataclass, field
from typing import Iterable

WORD = re.compile(r"[A-Za-z']+")


@dataclass(frozen=True)
class Counter:
    """Counts words, ignoring case."""

    counts: dict[str, int] = field(default_factory=dict)

    def add(self, text: str) -> None:
        for match in WORD.finditer(text.lower()):
            word = match.group(0)
            self.counts[word] = self.counts.get(word, 0) + 1

    @property
    def total(self) -> int:
        return sum(self.counts.values())


async def gather(sources: Iterable[str]) -> Counter:
    counter = Counter()
    for source in sources:
        counter.add(source)
    return counter


if __name__ == "__main__":
    # f-strings, numeric literals and a walrus.
    counter = Counter()
    counter.add("the quick brown fox")
    if (total := counter.total) > 0:
        print(f"{total=} {0x1F:d} {1_000_000} {3.14e-2!r}")
    else:
        raise SystemExit("nothing counted")

JavaScript and TypeScript

JavaScript’s queries inherit from ecma and jsx; TypeScript’s from ecma. Neither of those is a grammar in its own right – they exist only to be inherited from.

JavaScript

import { readFile } from "node:fs/promises";

/**
 * Loads a config file.
 * @param {string} path
 * @returns {Promise<Record<string, unknown>>}
 */
export async function loadConfig(path) {
  const text = await readFile(path, "utf8");
  return JSON.parse(text);
}

export class Cache extends Map {
  #hits = 0;

  static from(entries) {
    return new Cache(entries);
  }

  get(key) {
    const value = super.get(key);
    if (value !== undefined) this.#hits += 1;
    return value ?? null;
  }

  get hits() {
    return this.#hits;
  }
}

const NAMES = /^[a-z][\w-]*$/iu;
const tagged = (strings, ...values) => String.raw({ raw: strings }, ...values);

for (const [key, value] of Object.entries({ a: 1, b: 2 })) {
  if (!NAMES.test(key)) continue;
  console.log(tagged`${key} => ${value}`);
}

TypeScript

type Result<T, E = Error> = { ok: true; value: T } | { ok: false; error: E };

interface Repository<T> {
  readonly name: string;
  find(id: string): Promise<T | undefined>;
}

export enum Level {
  Debug = "debug",
  Info = "info",
}

export abstract class Base<T extends { id: string }> implements Repository<T> {
  protected constructor(public readonly name: string) {}

  abstract find(id: string): Promise<T | undefined>;

  async findOrFail(id: string): Promise<Result<T>> {
    const found = await this.find(id);
    return found
      ? { ok: true, value: found }
      : { ok: false, error: new Error(`missing ${id}`) };
  }
}

export function isLevel(value: unknown): value is Level {
  return typeof value === "string" && Object.values(Level).includes(value as Level);
}

JSX

jsx and ecma are query-only languages in nvim-treesitter – they have no grammar of their own and exist so that javascript, typescript and tsx can inherit their queries. They are also Neovim filetype aliases for JavaScript, so a fence tagged with either resolves to JavaScript, which does have a parser and which pulls those queries in anyway.

export function Card({ title, children }) {
  const [open, setOpen] = useState(false);
  return (
    <article className="card" data-open={open}>
      <h2 onClick={() => setOpen(!open)}>{title}</h2>
      {open && <div className="body">{children}</div>}
    </article>
  );
}

The web stack

The interesting part here is injection: an HTML document’s <script> and <style> contents are separate languages, and the queries say so. Highlighting this page needed the JavaScript and CSS parsers even though no block is tagged with either.

HTML, with JavaScript and CSS inside it

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>Injected languages</title>
    <style>
      :root {
        --accent: #4ec9b0;
      }

      .card > h1::after {
        content: "!";
        color: var(--accent);
      }

      @media (max-width: 40rem) {
        .card {
          padding: 0.5rem 1rem;
        }
      }
    </style>
  </head>
  <body>
    <!-- A comment, then a script. -->
    <div class="card" data-id="1">
      <h1>Hello</h1>
    </div>
    <script type="module">
      const card = document.querySelector(".card");
      card?.addEventListener("click", (event) => {
        event.preventDefault();
        console.log(`clicked ${card.dataset.id}`);
      });
    </script>
  </body>
</html>

CSS on its own

@import url("reset.css");

:root {
  --spacing: 8px;
  --font: "Inter", system-ui, sans-serif;
}

.grid {
  display: grid;
  grid-template-columns: repeat(auto-fill, minmax(12rem, 1fr));
  gap: calc(var(--spacing) * 2);
}

.grid:hover > .cell:not(.disabled)::before {
  content: "→";
  transform: translateX(-100%) rotate(45deg);
  transition: transform 150ms ease-in-out;
}

@supports (backdrop-filter: blur(4px)) {
  .overlay {
    backdrop-filter: blur(4px);
  }
}

Markdown

A markdown block injects whatever its own fenced blocks are tagged with, so the sample below needs the Rust and Python parsers on top of markdown’s own two grammars (markdown for block structure, markdown_inline for the rest).

# A document

Some **bold** text, some *emphasis*, a [link](https://example.com) and
`inline code`.

- a list item
- another, with a nested fence:

  ```rust
  fn nested() -> u8 { 1 }
  ```

> A quote with a fence in it:
>
> ```python
> def nested():
>     return 1
> ```

| Column | Column |
| ------ | ------ |
| a      | b      |

Shell and configuration

Bash

#!/usr/bin/env bash
set -euo pipefail

readonly ROOT="${1:-$(pwd)}"
declare -a targets=()

usage() {
  cat <<'USAGE'
usage: build.sh [root]
USAGE
}

for file in "$ROOT"/*.toml; do
  [[ -f "$file" ]] || continue
  targets+=("$(basename "$file" .toml)")
done

if (( ${#targets[@]} == 0 )); then
  usage >&2
  exit 1
fi

printf '%s\n' "${targets[@]}" | sort -u

PowerShell

Note

ps1 also works as an alias: Get-ChildItem -Recurse | Measure-Object.

#Requires -Version 7.0
using namespace System.Collections.Generic

Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'

<#
.SYNOPSIS
    Counts files by extension under a root.
.EXAMPLE
    Measure-Extension -Root . -Minimum 2
#>
function Measure-Extension {
    [CmdletBinding()]
    [OutputType([hashtable])]
    param(
        [Parameter(Mandatory, ValueFromPipeline)]
        [ValidateNotNullOrEmpty()]
        [string] $Root,

        [ValidateRange(1, [int]::MaxValue)]
        [int] $Minimum = 1
    )

    begin {
        $counts = [Dictionary[string, int]]::new()
    }

    process {
        Get-ChildItem -Path $Root -File -Recurse | ForEach-Object {
            $key = if ($_.Extension) { $_.Extension } else { '(none)' }
            $counts[$key] = 1 + ($counts[$key] ?? 0)
        }
    }

    end {
        $counts.GetEnumerator() |
            Where-Object { $_.Value -ge $Minimum } |
            Sort-Object -Property Value -Descending
    }
}

TOML

[package]
name = "example"
version = "0.1.0"
edition = "2024"

[dependencies]
serde = { version = "1", features = ["derive"] }

[[bin]]
name = "example"
path = "src/main.rs"

[profile.release]
lto = true
codegen-units = 1

YAML

name: ci
on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build
        run: |
          cargo build --release
          cargo test
    env:
      RUST_BACKTRACE: "1"

JSON

{
  "name": "example",
  "version": "0.1.0",
  "private": true,
  "scripts": { "build": "tsc -p ." },
  "numbers": [1, -2, 3.5e10],
  "nested": { "ok": true, "missing": null }
}

Lua

local M = {}

---@param items string[]
---@return table<string, integer>
function M.index(items)
  local out = {}
  for i, item in ipairs(items) do
    out[item] = i
  end
  return out
end

function M.greet(name)
  name = name or "world"
  return ("hello, %s"):format(name)
end

return M

Keyboard shortcuts

mdbook-nvim-treesitter