Skip to main content
Grady Gaugler

How I build with AI

I build with AI agents every day. This is my current setup, where it falls short, what I did with AI at GitKraken, and one worked example: this site.

My setup

This is how I work now, in 2026.

  • One interface: T3 Code runs several agents, mainly Claude Code and Codex.
  • Claude Code is usually the primary agent. In my experience it is stronger for coding, UI, and general decisions.
  • GPT reviews changes and investigations for a second perspective. I find it stronger for investigation, review, and architecture. These are my observations, not benchmarks.
  • A few focused skills and a strong AGENTS.md keep output close to the stack's best practices and biased toward simplicity.
  • Agents run manual end-to-end tests before each release and before my own testing. That adds to my review; it doesn't replace it.
How Grady builds with AIHis current setup (2026): one interface for several agents, Claude Code as the primary builder, GPT as reviewer, guardrails in skills and AGENTS.md, agent-run end-to-end tests, and Grady directing product and UI.
  1. Grady: Sets direction and makes the callsleads to T3 Code (directs work)
  2. T3 Code: One interface for several agentsleads to Claude Code (builds)
  3. Skills + AGENTS.md: Stack best practices, bias toward simplicityleads to Claude Code (guides)
  4. Claude Code: Primary agent: code, UI, general decisionsleads to GPT (second opinion); leads to End-to-end tests (runs)
  5. GPT: Reviews changes and investigationsleads to Claude Code (findings)
  6. End-to-end tests: Agents test before release and before Grady doesleads to Grady reviews
  7. Grady reviews: Tests the product and directs UI and UXleads to Grady (decides)
  • His current setup (2026), not a setup used at past employers.
  • Current practice (2026).
  • His own observations, not benchmarks.

Where AI falls short

UI and UX design. Agents miss details and overdesign. A smooth, consistent, trustworthy interface still needs my direction and my own testing.

So I make the design calls, and I cut anything that doesn't earn its place.

AI at GitKraken

Commit-message generation was GitKraken's most popular AI feature, with poor ratings. I started the company's first eval framework: hundreds of real commits, a human benchmark scored by developers, and LLM judges tuned to within 1–5% of the human ratings. Prompt and model changes measured that way took upvotes from 55% to 75%.

I also used agentic workflows for the product work itself: planning, working prototypes, draft implementations before handoff, and analytics.

  • 55→75%

    AI commit-message upvotes

    User thumbs-up rate on generated commit messages. Shipped with engineering teams.

  • ~4×

    faster draft-to-ready planning

    Approximate.

  • ~4×

    less design back-and-forth

    Approximate.

  • Up to 2×

    faster development

    Approximate.

GitKraken commit-message evalsHow Grady built GitKraken's first AI eval framework: real data, a human benchmark, LLM judges tuned to it, then prompt and model iteration that took upvotes from 55% to 75%.
  1. Low ratings: Most popular AI feature, poor upvotesleads to Real commits
  2. Real commits: Hundreds, with diff, branch, and prior messagesleads to Four scores
  3. Four scores: Three-point scale, defined with developersleads to Human benchmark
  4. Human benchmark: Four developers scored 100 examplesleads to LLM judges (calibrate)
  5. LLM judges: One per score, within 1–5% of humansleads to Prompt and model (score each change)
  6. Prompt and model: Prior commits as context; score vs. cost and speedleads to LLM judges (until every score rises); leads to Shipped
  7. Shipped: Upvotes 55% → 75%
The eval storyAI-native product work
  • User thumbs-up rate on generated commit messages. Shipped with engineering teams.
  • Approximate.

AI in my own apps

Well Said finds quotes from a plain-language description, using embeddings, semantic search, and retrieval alongside web searches, with eval sets to measure changes. The hard part is quality: a quote can match the topic and miss the meaning. I weigh quality against cost, and the results are still imperfect.

I also built automated AI error loops that fix errors in Well Said automatically.

Song Sweeper has no AI features. I built it with Claude Code.

Well Said AI searchHow Song Sweeper is built
  • Results are still imperfect. No measured cost savings or eval improvements are published for Well Said.
  • Details of what triggers a loop and how fixes are reviewed and deployed aren't published.

Worked example: this site

Claude Code built this site as the primary agent. Before building, a second model reviewed the plan: GPT, which I call Sol in my setup. It caught real issues, and I decided each one:

  • A kill switch based on an environment variable would need a redeploy to flip. It became a runtime flag.
  • A valid citation id doesn't prove the claim is supported. I added human-reviewed evals.
  • Logging full questions and uploads was a privacy risk. The chat now logs metadata plus short, redacted questions, with a notice.
  • The Apple Retail column had the wrong job title.
  • A headline-number rule would have forced an unverified Well Said metric onto the page.
This site's chatA question goes to /api/chat. The model answers only with typed blocks that cite content ids; the server checks every id, and the site renders its own sections, cards, and metrics.
  1. Visitor: A question, optionally with a documentleads to /api/chat
  2. /api/chat: Limits, kill switch, redacted loggingleads to Model
  3. Published work: Facts, metrics, sections, matrixleads to /api/chat (context)
  4. Model: Returns typed blocks that cite idsleads to Server validation (blocks)
  5. Server validation: Drops blocks with unknown idsleads to Site components
  6. Site components: Sections, cards, metrics, matrixleads to Visitor