Skip to content
CASE STUDY Product Designer & Builder · Omnis AI

Design QA took me a day per release.So I built an engine that takes minutes.

A tool that opens a web app the way a person would, clicks through its menus and pop-ups, and lists each design problem it finds with the exact code fix. It costs about $0.30 a screen.

Role
Solo product designer & builder
Duration
Jun – Jul 2026 · Omnis AI
Owned
Problem, design, architecture, ~27k lines of TypeScript
Skills
Design QAAccessibilityAI pipeline designFull-stack TypeScript
Omnis UX Audit Engine live report preview: CaseNotes dashboard scan with expanded critical contrast findings
01
Overview

I built the QA teammate I didn't have.

I was the only designer across five legal-tech products at Omnis AI. Before each release I checked every screen by hand, and it took hours. So I built a tool that does the checking, and I keep the final say.

How it worksFour steps, no designer needed in the loop
Give it a link and a test login
It clicks through screens, menus, and pop-ups
It checks each state against 229 rules
You get a report with a fix for each problem
02
The problem

Design bugs hide behind clicks.

A screen can pass review when it first loads. The problems show up after someone opens a dropdown, submits a form with an error, or lands on an empty page.

Nobody had time to click through all of those states before each release, so those bugs reached real users.

The same New meeting dialog, closed and open
Left: at first load the form passes review. Right: open the Matter dropdown and three problems appear. The list is cut off by the dialog edge, the options are too faint to read, and the list covers the Schedule button.

I needed a way to check every state of every screen, fast enough to run before each release, and accurate enough that engineers would trust it.

03
Solution

Three parts: a clicker, a checker, and a reporter.

PartWhat it doesWhat you get
CrawlerLogs in and clicks through screens, menus, dialogs, and tabsA record of each state it reached
CheckerRuns 229 fixed rules (contrast, design tokens, accessibility), then an AI pass for layoutProblems, each tied to a real element on the page
ReporterSorts problems into a view for each roleA report with the exact code fix for each problem

Who it's for: product designers, frontend engineers, QA, content designers, and the managers who sign off on releases.

04
Background

One designer, five products, no QA team.

Who this was built for · me, first
Owned five products

Each one shipped on its own schedule.

Checked each release by hand

Screenshots, red boxes in Figma, then tickets.

Repeated it after each fix

A fix meant checking the whole screen again.

A typical manual check meant taking screenshots, drawing red boxes in Figma, listing the design tokens, and writing tickets. When engineers fixed something, I checked everything again. Design QA became the thing holding releases back.

Before: manual screenshot markup in Figma, one note per problem
Before: my manual review in Figma, one note per problem.
05
Design process

Two versions. The first one made things up.

Each version tested the same question: can a tool find design problems that engineers trust enough to fix?

V1Reading screenshots with AI

V1 report flagging low contrast on a button that does not exist
V1's report flagged low contrast on a button beside the search field. The red box marks that spot: there's no button there.
What I built
The tool took screenshots and asked an AI vision model to list problems, each with a location, a severity, and a suggested fix.
What I saw
On a dashboard I knew well, it reported low contrast on "the secondary button beside the search field." That button didn't exist. The AI had guessed from what dashboards usually look like.
Insight
If one finding is made up, you have to double-check all of them, and that cancels out the time saved.

FinalChecking the real page, with proof

Annotated screen: flagged interface elements mapped to numbered finding cards with CSS selectors and diffs
Final: each flagged element maps to a numbered finding with its selector and fix.
What I built
The tool works inside a live browser. It runs 229 fixed rules first and uses AI only for layout questions. Any problem that can't point to a real element on the page gets thrown out.
What I saw
Over 90% of findings point to an exact element in the page code, so an engineer can find each problem in one click.
Insight
A short list people trust beats a long list they have to verify.
Keep or dropHow each finding earns its place in the report
A rule or the AI spots a problem
Does it point to a real element on the page?
Yes: it goes in the report
No: it gets dropped
If a designer rejects a finding, the tool remembers and won't flag it again
The tool reading a live page: styles, positions, and the element tree, straight from the browser.
06
Outcome

What changed between the manual way and the tool.

Before · by hand
After · the tool
Hours of designer time per product
About 3 minutes per screen
Some states checked, when there was time
All reachable controls on the audited screens
A Figma screenshot with a red box
The exact element and a copy-paste fix
Check everything again after a fix
One command re-checks and shows what changed
After: automated report with annotated screen and DOM-anchored finding cards
After: the tool's report, with the screen annotated and each problem tied to its element.
Before · by handEvery release, every screen, every fix
Open each screen
Screenshot and mark it in Figma
Click menus and errors by hand
Answer "which element?" questions
Start over after each fix
After · the toolOne command, proof attached
Run one command
The tool clicks through each state
Each problem points to a real element
Engineers paste the fix
Run it again after a fix to see only what changed
$0.30

per screen, in about 3 minutes

Measured compute and API cost. Manual reviews took 2 to 3 hours per product.

89

problems across 22 states

A 4-screen check of CaseNotes, in about 11 minutes, for about $1.20.

90%+

tied to a real element

Each finding links to the exact spot in the page code.

AutoDoc: 87 findings by severity
Ordered by triage priority
Blocker 4 Major 26 Minor 42 Nit 12 Praise 3
Scope noteThese numbers come from my own test runs on Omnis staging apps. They are not a published benchmark, not checked by a third party, and not measured on live user traffic.
07
Design decisions

Proof first, clicks before judging, fixes ready to paste.

  1. Proof before opinion.

    Fixed rules handle what can be measured: contrast, accessibility attributes, and design tokens. AI handles judgment calls, and only when it can point to the element.

  2. Click before judging.

    The tool opens each dialog, dropdown, drawer, and tab before it reviews a screen.

State explorer screenshot: crawling beyond the default screen to audit dialogs, drawers, and active dropdowns
The tool going past the first screen to check dialogs, drawers, and open dropdowns.
  1. Every problem comes with its fix.

    Each card shows a severity, the rule it broke, a plain-English title, the design principle behind it, a copy-paste CSS fix, and a confidence score.

Anatomy of a finding card: verified DET-CONTRAST-001 finding with rule IDs, selector paths, and token fixes
One finding card, from severity to the fix an engineer pastes.
A finding inside a GitHub pull request, with selector, measured colors and a suggested fix
What the engineer receives: the rule, the exact element, the measured colors, and a fix they can commit in one click.
  1. Each role sees its own list.

    One run, six readers, each filtered to the work they can act on:

    • Product designer: problems by severity and token mismatches. Rejecting a false alarm stops it coming back.
    • Frontend developer: blockers with the exact element, its styles, and the fix.
    • QA: a log of every dialog, drawer, and dropdown the tool opened.
    • Content designer: a separate list of confusing labels and copy.
    • Engineering manager: a dashboard across all products.
    • Leadership: a record of problems fixed over time.
Dashboard showing findings across four products, open blockers and a release gate
One run across four products: finding counts per product, open blockers with an owner, fixes per week, and a release gate.
08
Under the hood

Two codebases, one TypeScript stack.

The engine (ux-audit-mcp) captures and checks pages. The service (ux-audit-service) adds the multi-product dashboard and automation.

Terminal output of one audit run, ending with 89 findings and two report files
One command logs in, opens each state, drops findings it can't tie to a real element, and saves an HTML and a Markdown report.
LayerTechnology
runtimeNode + TypeScript, Model Context Protocol (MCP) SDK
capturePlaywright (session management, full-page screenshots, DOM inventory, state crawler)
a11y@axe-core/playwright (live axe engine execution per page)
ai-visionOpenAI-compatible SDK with Gemini Flash and OpenRouter fallbacks
imageSharp (annotation rendering), tesseract.js (OCR coordinate snapping), pixelmatch (diffing)
serviceNext.js 16, React 19, Tailwind 4, Prisma 7 + Postgres, BullMQ + Redis, GitHub Actions, Docker
Reports

Reports open anywhere.

Each audit saves as a standalone HTML page and a Markdown file ready for GitHub or Linear.

09
Agent Skill & Bundle

Download the UX Audit Assistant Skill.

I packaged the heuristics engine, WCAG 2.2 accessibility rules, DOM state exploration prompts, and report markdown templates into a drop-in Claude and AI agent skill bundle. Download the package to run automated design reviews in your own workflow.

Agent Skill Bundle · v1.2

UX Audit Assistant Agent Skill

Drop-in skill for Claude Code, Cursor, and custom agent loops. Automates design QA and generates grounded finding cards from real DOM states.

229+ Deterministic UX & a11y heuristics
DOM state explorer prompts & rules
Self-contained HTML + Markdown report templates
Developer-ready copy-paste fix generator
Get the skill package
ai-ux-audit.skill SKILL.md README.md

Direct download. Compatible with Claude Code, Cursor, Windsurf, and custom agent workflows.

10
Looking back

What building it taught me.

  • Trust comes before speed. A tool that reports 50 problems with 15 made up costs more time than checking by hand.
  • Real checks need clicks. Looking only at first loads misses overflowing dropdowns, error messages, and empty pages.
  • Engineers fix what they can paste. A selector and a code fix get merged. A vague comment gets skipped.
Where it standsA working prototype with 187 automated tests and about 27,000 lines of TypeScript. After a company restructuring changed the staging setup, I parked it. Testing at a larger scale is still open.
If I pick it back up
  1. Remember a starting point for each product, so later runs flag only new problems.
  2. Run on every staging deploy through GitHub Actions and flag new blockers on pull requests.
  3. Re-check after a fix is merged to confirm it worked.
  4. Send verified notes back into Figma files through the Figma API.
11
Questions

Questions people ask.

Who approved giving a product designer live credentials to staging environments?
No new grant was needed: it reused the same staging access I already held as the product designer auditing those apps by hand. The engine logs in with my credentials, scoped to the same environments I already reviewed, and never touched production or anyone else's access.
Are you a designer or an engineer?
Product design is my core foundation: research, UX architecture, interaction design, and design systems. I also write production TypeScript to ship working prototypes and developer tooling end-to-end.
What was your contribution compared to engineering?
I conceived, designed, and built the entire engine myself, covering the MCP architecture, Playwright crawlers, axe integrations, and Next.js reporting dashboard.
How do you prevent model hallucinations?
Deterministic checks evaluate objective rules (contrast math, aria attributes). AI heuristic findings are rejected unless they map to a valid DOM node and pass structured false-positive validation.
How does this differ from axe-core or Lighthouse?
Lighthouse and axe evaluate a single static URL. This engine authenticates, crawls interactive application states, pairs axe-core with semantic layout heuristics, and exports diff-ready CSS token patches.
How did you measure audit effectiveness?
By tracking verified findings per screen and measuring false-positive rates against real product staging environments.
Can this deploy to production CI pipelines?
Not yet. Running audits on every staging deploy and flagging new blockers on pull requests is the next step listed above.
NEXT CASE STUDY

CasePro Matter Redesign: Splitting Intake from Case Creation

I split a merged intake-and-matter form into two linked systems: an AI-assisted path and a manual path into each new matter.