Gapfinder: an AI helpdesk that finds the holes in a company's docs
Sebastian BrookesOctober 4, 2026
Overview
Employees at Northwake Therapeutics, a fictional biotech, ask how to get things done in Coupa, Concur, Workday, Ironclad, ServiceNow, and Microsoft 365. The assistant answers only from Northwake's Help articles and names the article it used. When the articles fall short, it doesn't guess. It logs a Gap and groups it with others asking the same thing. The IT team has the AI draft the missing article, then approves it.
In the demo you play both sides, the Employee asking questions and the IT team fixing the docs. Everything is fake, and other visitors never see what you do.
How it works
Swipe sideways to see the whole diagram.
Why every article goes with every question
At 37 articles the whole library fits in one prompt, so the AI sees everything and has to cite the article it used. There's no search step to miss a synonym, like "PTO" for "time off," and log a false Gap. At a real company with thousands of articles, I'd put search in front.
Why the IT team notes never answer Employees
The notes are facts the IT team knows that no article states yet. They speed up the person approving a draft, but they're never a shortcut around approval. If the assistant answered from them, the Gap would be hidden and the docs would never get fixed.
Why it can't overspend
The AI calls run on prepaid credit with automatic top-ups off, and the demo's key has a monthly limit. Each visitor gets 15 AI calls a day, and the whole app gets 250. When the money or a limit runs out, the app saves typed questions and points the visitor to the three Suggested questions, which keep working from saved answers.
How it was tested
I measured the assistant on a fixed Test set of 120 new questions: 74 with a Help article that answers them, 33 about the 11 holes still open in the Help articles (3 phrasings each), and 13 Off topic. All 120 were written after the app was finished: 100 by Claude Code and 20 by a separate AI agent that never saw the Help articles. Nobody tuned the prompts on them. Each run started from a fresh copy of the starting data, on a database kept apart from the live demo. A script scored every answer, with no AI grading.
| Measure | Worst of 3 runs | What counts |
|---|---|---|
| Correct article | 68/74 (91%) | An answerable question cites the right article and logs no Gap. |
| Known gaps flagged | 33/33 (100%) | A question about a known hole in the docs logs a Gap. |
| Correct Gap reason | 30/33 (90%) | That Gap gets the right reason, No match or Not covered. |
| False alarms | 6/74 (8%) | An answerable question logs a Gap. Lower is better. |
| Off topic declined | 13/13 (100%) | Small talk and lab questions get a polite decline. |
| Gaps grouped correctly | 9/11 (81%) | All 3 phrasings of a known gap join its existing Gap group. |
| Fill from IT notes | 3/11 (27%) | Every filled fact uses a note's own words, and facts the notes lack stay blank. |
The first two rows are the numbers on my resume.
Caveats
The data is fake. The company, its 37 Help articles, its IT team notes, and the Test set questions were all written with AI. A real company's docs would be messier.
Didn't help isn't tested. That Gap reason needs a person to click the button, so the measures cover only No match and Not covered. For the same reason, a question about the out-of-date MFA article counts as answered when it cites that article.
Each number is the worst of 3 runs, so one lucky run doesn't set it. For false alarms, worst means the highest.
Fill from IT notes is scored strictly. The script fails any fact the AI reworded, like "completed" for "finished." A separate AI agent read all 85 filled facts against the notes. None contradicted a note, 2 added something the notes don't say, and by that reading the worst run fills 10 of 11 drafts correctly.
Checks on the Test set
A separate AI agent that never saw the answer keys built its own from the Help articles. It disagreed on 2 questions, and both were reworded before any run. Another agent read 20 answers that scored as correct against the articles they cited. 19 matched, one added a minor unsupported detail, and none was wrong. I didn't read the answers myself.
The full results are in the repo: the summary and the files for run 1, run 2, and run 3.
How I built it
I built this with Claude Code in about two days. The AI wrote the code. I decided what to build and checked that it worked.
What I specified
Before any code, I worked through the design in a series of decision sessions with Claude Code. I picked what the assistant may do on its own (answer, log Gaps, and draft only when asked) and where a person has to approve (every article, with no unchecked facts). I chose to send every article with each question instead of searching, to keep the IT team notes out of answers, and to cap spending. I set the testing rules: write the Test set after the app was done, never tune on it, and keep the worst of 3 runs. For the layouts, I chose between throwaway prototypes, like tabs over stacked panels on phones.
What I checked
I tested the full flow by hand on the live site from a phone and a desktop: a Suggested question, a typed question, a Gap, a draft, and an approval. I tried Fill from IT notes on a local copy before it went live, and I ran the Test set.
What the AI wrote
Claude Code wrote the app, its automated tests, and the scripts that generate the data and run the Test set. AI also wrote the fictional company's Help articles, IT team notes, and 8 weeks of question history, plus the Test set questions and answer keys.