Claude App Bot's Diff Pipeline

by Jared Tan

Two parallel feedback loops keep the Claude app-builder bot learning from the hand-built north-star app. Parity loop: each commit on the real app is re-implemented by the bot in a sandbox, audited for gaps, and patched back via PR. Lessons loop: every reusable rule we discover while building the real app gets absorbed into the bot's scaffolding templates so the next app it generates ships with the lesson baked in.

Parity loop: weresobach → bot → bottest → audit → bot PR ↻ verify
Lessons loop: WSB feedback memory → absorb workflow → template PR → future apps inherit
How the parity loop works (per-commit feature replication)
●
Commit
You tweak or add a feature in the real app — this is what the bot needs to learn to build.
Technically: A new commit lands on weresobach/main (the hand-built north-star app).
●
Phase 2 sync
The system shows the Claude bot your change and asks it to build the same thing on its own.
Technically: path-b-sync workflow sends the commit + diff to the bridge's /prompt API so the bot rebuilds the same feature on bottest.
●
Bottest commit
The bot's attempt lands in a separate "bot-only" copy of the app, so you can compare side by side.
Technically: The bot pushes its output to weresobachbottest/main with a [path-b-sync] tag.
●
Phase 3 audit
An automated checker scans the bot's version for must-have features — maps, login, real-time sync, games, and more.
Technically: gap-audit workflow fingerprints bottest against 7 curated patterns and reports missing features.
●
Gap PR
If anything is missing, a draft pull request opens on the bot's own repo so you can teach it better next time.
Technically: A draft PR lands on discord-claude-app-builder with the findings to improve bot prompts. PR body carries a <!-- path-b-retry --> marker so Phase 4 knows which SHA to retry.
●
Phase 4 verify
When you merge the gap PR, the bot gets another chance to build it — up to 5 tries. Stops when the audit finds zero gaps, or hits the limit.
Technically: path-b-verify workflow parses the PR marker, bumps .path-b-retries.json on bottest, and dispatches path-b-sync.yml on weresobach with the same SHA.
How the lessons loop works (durable rules → bot templates)

Parity proves the bot can rebuild a specific commit. The lessons loop teaches the bot rules that should apply to every future app — so each new project Discord users start inherits the gotchas we already worked through on the real one.

●
Memory captured
While building the real app, a recurring rule comes up — a UI gotcha, a serialization trap, a permission default. It gets written down as a small memory file.
Technically: A feedback_*.md file lands in ~/.claude/projects/<bridge>/memory/ on the Mac mini. Lives outside any git repo — these are Claude Code's auto-memory.
●
Absorb workflow
Once a week (or on demand) the system picks the newest unabsorbed memory and asks the bot to add it to its own scaffolding template.
Technically: path-b-learnings.yml runs on the self-hosted Mac mini runner, reads the memory file, and runs Claude with edit permission against this repo's commands/create.py + helpers/.
●
Template PR
The bot's proposed edit shows up as a draft pull request titled absorb(memory_name). You read it, decide whether the bot understood the lesson, then merge or close.
Technically: Draft PR on discord-claude-app-builder. PR body inlines the full memory body so you can sanity-check the absorption without leaving the page.
●
Future apps inherit
After merge, the next time someone in Discord asks the bot to build an app, the bot writes a CLAUDE.md that already knows the rule.
Technically: The CLAUDE.md template lives as an f-string inside commands/create.py. Each new /create writes the current version of that template into the new project.
●
Skip rule
If a memory already has a PR (open, merged, or closed-as-rejected), the absorb workflow skips it next time.
Technically: The pick step calls gh pr list --search "absorb(<basename>) in:title". Closing a misread absorption rejects it permanently; reopening or deleting the PR makes the memory eligible again.
Status (May 2026) — what was just fixed

Path B's parity loop was silently noop'ing from 2026-05-04 through 2026-05-17. Two-stage diagnosis and fix:

What GitHub permissions does this need?
The pipeline spans three private repos, so it uses fine-grained personal access tokens (PATs) scoped to exactly these repos — not your whole GitHub account. All tokens live as repo / Cloudflare secrets; none are exposed on this page. Three tokens total, two of which can modify code:
Read only Dashboard reader GH_TOKEN
Lets this dashboard list commits, PRs, and workflow runs. Can't change anything — if it ever leaked, nobody could use it to push code, dispatch workflows, or merge PRs.
Contents: read · Actions: read · Pull requests: read
on weresobach, weresobachbottest, discord-claude-app-builder
Write Phase 3 PR opener BRIDGE_AUDIT_PR_TOKEN
Lets the gap auditor open draft PRs on the bot's repo when it spots missing features, and push the branch that backs each PR. Cannot touch the hand-built app's source.
Contents: write · Pull requests: write
on discord-claude-app-builder, weresobachbottest
Write Phase 4 verifier PATH_B_VERIFY_TOKEN
Lets the verify loop bump the retry counter on the bot's sandbox and ask the pipeline to try a SHA again. Can dispatch workflows but doesn't modify code directly.
Actions: write (on weresobach) · Contents: write (on weresobachbottest) · Pull requests: write (on discord-claude-app-builder)
different surface on each of the three repos — principle of least privilege
—
pipelines
—
succeeded
—
running
—
failed
—
skipped
Loading pipelines…