02 · Architecture

← Docs home · For a new contributor learning how the repo is wired.

The shape in one paragraph

A raw textbook source lands in projects/<Book>/. Claude Code, driving the book-pipeline skill, walks a fixed 15-step list — calling deterministic Python helpers in scripts/workflows/ for the mechanical steps and doing the AI steps (remaster, solutions, math-check, HTML) itself. The result is per-section HTML under projects/<Book>/html/, which publish.py verifies and copies into docs/. docs/ is the deploy root: a wrangler pages deploy ships it to the Cloudflare Pages project bookshelf, where a handful of Pages Functions (functions/) add the dynamic bits (AI chat, slide-deck storage) on top of the static pages. Media enhancement (Manim figures/videos, Slidev decks, EPUB) hangs off the same source tree through dedicated skills.

 projects/<Book>/          .claude/skills/ + scripts/workflows/        docs/                 Cloudflare
 ┌──────────────┐  book-   ┌────────────────────────────────┐ publish ┌───────────┐ deploy ┌──────────────┐
 │ source_files │ pipeline │ scrape→merge→extract→normalize  │  .py    │  *.html   │ wrangler│ Pages: bookshelf│
 │ remastered/  │ ───────▶ │ →remaster→number→solutions→     │ ──────▶ │  images/  │ ──────▶ │ + Functions    │
 │ figure_*.yaml│          │ philosophy→youtube→math→html→   │         │  slides/  │         │ + D1 + R2      │
 │ section_*.yaml│         │ fixup→publish→figure→slides     │         │  ebooks/  │         │                │
 └──────────────┘          └────────────────────────────────┘         └───────────┘         └──────────────┘
        │                                                                     ▲
        └──── manim-videos/ (figures + narrated openers) ─── render/mux ──────┘

Repo map (tracked top-level)

Path What lives here
projects/ Source + intermediate artifacts, one dir per book. The pipeline’s working area. (~4,300 files.)
docs/ The published site — deploy root for Cloudflare Pages. Rendered HTML, images, slides, ebooks, plus this docs/.
scripts/workflows/ The deterministic helpers — every Python/Node script the pipeline calls (numbering, HTML gen, publish, figure render, a11y, EPUB).
.claude/skills/ The procedures — ~30 skills. SKILL.md files are the live spec for each task.
.claude/ Also: agents/, commands/, hooks/, teams/, settings.json — the Claude Code harness config.
manim-videos/ Manim scene sources (figure-*/, narrated openers), render scripts, TTS caches → final MP4s.
functions/ Cloudflare Pages Functions — the dynamic backend (see below).
migrations/ D1 (SQLite) schema migrations for the backend.
prompts/ Prompt files the AI pipeline steps read (remaster-chapter.md, solutions.md, …).
slide-deck/ Slidev project scaffolding / shared deck assets.
tests/ Pytest + node tests for the workflow scripts.
.agents/ Project memory + handoffs (memory/MEMORY.md, active/, long-term/).
graphify-out/ Persistent knowledge graph of the repo (graph.json, GRAPH_REPORT.md).
wrangler.toml Cloudflare Pages runtime bindings (D1, R2, vars).

Stale-doc warning: the root README.md references a .hermes/ agent and an Electron bookshelf-app/ — neither exists in the tree. Ignore those sections; these docs reflect the current reality.

Data flow: a section’s journey

For one section (say Stats §8.4), the artifacts flow through projects/<Book>/:

source_files/<source>/8-4.md            ← raw extracted source (per page)
        │  single_source_extract.py  (or scrape→merge→extract for multi-source)
        ▼
remastered/Chapter_8_Source/8.4_*.md    ← canonical source layout
        │  normalize-tags → remaster (AI) → number
        ▼
remastered/Chapter_8_Numbered/8.4_*.md  ← numbered, then + _Solutions.md (AI splice)
        │  philosophy-reorg → youtube → math → html (AI) → fixup
        ▼
html/8.4_*.html                          ← assembled section page
        │  publish.py  (verify → copy → figure swaps → navbar sync)
        ▼
docs/introduction-to-stats/chapter-8-*/8.4_*.html   ← published

The canonical step list and per-step details live in 03 · Content Pipeline and, exhaustively, in PIPELINE_GUIDE.md.

Per-project config (the manifests)

Each projects/<Book>/ carries YAML manifests the pipeline and preflight read:

File Controls
pipeline_config.yaml Per-project pipeline options (e.g. philosophy: auto|native).
book_manifest.yaml Book/chapter/section structure.
figure_animations.yaml Which figures become animated MP4 / static PNG pairs (per-figure mode + cache_buster).
section_videos.yaml Narrated section-opener video registry (dir, variants, TTS backend, cache buster).
manim_candidates.yaml Candidate figures worth animating.

The book-pipeline-preflight skill validates all of these (plus source-file existence) before any pipeline command runs.

Tech stack

Layer Technology
Orchestration Claude Code skills (in-session) + deterministic Python 3 / Node helpers
Math rendering KaTeX (Calculus) · MathJax (Stats, Finite Math)
Figures & video Manim Community Edition + FFmpeg + NumPy; neural TTS (Orpheus / CosyVoice 2 / Fish Audio) for narration
Slides Slidev (Vue + Vite + UnoCSS) → PPTX via transpile_slidev_to_pptx.py / embed_pptx_videos.py
Accessibility axe-core gate, KaTeX SSR, Speech Rule Engine + kokoro-js read-aloud; EPUB via custom builder + ACE audit
Styling docs/style-bookshelf.css — self-contained, owns layout and dark/light. Chapter pages link only this.
Backend Cloudflare Pages Functions + D1 (SQLite) + R2 (object storage)
Browser automation Playwright (headless) + Playwriter (real Chrome) for recording/verification

Stylesheet ownership: chapter pages load only style-bookshelf.css (self-contained). style.css is not loaded by chapter pages — verify the <link> before editing either.

The Cloudflare backend

The site is mostly static, but functions/ adds a thin dynamic layer as Cloudflare Pages Functions (file-based routing under functions/ → routes):

Route Function Purpose
/api/chat functions/api/chat.js AI study-tutor proxy. Per-day rate limit in D1 (bookshelf-ai, migration 0001). Talks to an Ollama endpoint.
/api/decks/*, /decks/:id/file functions/api/decks/, functions/decks/ Slide-deck upload/list/activate/versions. Files in R2 (bookshelf-decks). Metadata in D1 (migrations 0002–0004).
/api/slides-source functions/api/slides-source.js Slide-editor save-back commit to the GitHub repo (shuff57/bookSHelf, branch desktop) via a GITHUB_TOKEN secret.
/slides/*, /admin/slides functions/slides/, functions/admin/slides.js Slide rendering + admin.
functions/_lib/ — Shared helpers: auth.js, decks.js, html.js, ollama.js.

Bindings are declared in wrangler.toml (DB → D1, DECK_FILES → R2, plus GH_OWNER/GH_REPO/GH_BRANCH vars). Secrets (GITHUB_TOKEN) are set out-of-band with wrangler pages secret put.

Deploy mechanics: 08 · Publishing & Deploy.

Where to look when…