83 Rounds: What AI-Assisted Accessibility Testing Actually Looks Like in Claude Design and Code

83 Rounds: What AI-Assisted Accessibility Testing Actually Looks Like

In my last newsletter I mentioned that the demo for the Accessible and Inclusive Design Conference turned into a meta experience: the artifact meant to teach accessible testing became a live example of it. What I didn't share was what those rounds actually looked like, or the screen reader testing demo that came out of it. So here's the follow-up.

One bug, six rounds, and a lesson about AI's blind spot

The auto-play toggle on a tips carousel took six separate rounds to get right, and the back-and-forth is the clearest picture I have of why AI can't test its own output.

Round 42: The toggle announced its state twice, once from a live region and once from the button relabeling itself. Fixed by dropping the live-region message.

Round 43: That fix broke something else. The toggle stopped announcing its state at all, because a renamed button reads as a different button, not a changed state.

Round 55: Froze the label and used the live region as the only channel. It announced correctly, but now the visible label never showed "Pause."

Round 59: Tried combining a changing label and a live region, on the theory that they'd read as one announcement. They didn't.

Round 60: Went back through every combination tried across the entire history of this one button, laid them out side by side, and confirmed which single combination had actually tested clean twice.

Round 63–64: Even that broke again in real Safari testing, because VoiceOver doesn't reliably re-announce a focused control's own name change. Restored the live region as the actual spoken channel.

Every fix was locally correct and globally wrong until it was tested with an actual screen reader, in the browser, by a human listening to what it said.

The bugs that only exist because of manual testing

Some of the most useful rounds weren't AI mistakes. They were environment quirks that no amount of clean code fixes:

Round 65–67: Tabbing through the page in Safari skipped nearly every control. Not a code bug. Safari's default behavior only lets Tab reach text fields unless a system setting (Full Keyboard Access) is turned on, and links are gated by a second, separate setting entirely. No markup change fixes that. Only knowing the setting exists does.

Round 80: Two accessibility scanners reported "no heading structure" against the published link, even though the page had a full, verified heading hierarchy. The scanners were reading an empty iframe wrapper, not the actual content. Testing a standalone copy of the same HTML came back clean. The lesson: when a scanner contradicts what you can verify directly, test the raw file before trusting the scanner.

Round 82: A native date field looked accessible by every structural check: correctly labeled, focusable, semantically sound. Then a real VoiceOver test found that focusing it announced "in stepper," and arrow keys moved nothing. Screen reader users need a specific, non-obvious command just to interact with a native date input's segments. It got rebuilt as a plain text field with a format hint, trading native UI for something with no hidden interaction model to learn.

None of that shows up in a code review. It only shows up when you actually put a screen reader on the thing. Rounds 80 and 82 are the same lesson twice: passing a scan or looking structurally correct is not the same thing as being accessible. Compliant and accessible are two different bars, and the gap between them is exactly what manual testing exists to catch.

Fixes that quietly undid earlier fixes

Two rounds got reopened by later ones, which is its own lesson:

Round 51 reopened rounds 1 and 2. The tab order had technically been fixed with arrow-key navigation, which does satisfy keyboard access, but it wasn't the tabs pattern the design called for. It took knowing the pattern, not just testing "can I reach it by keyboard," to catch that.

Round 58 reopened round 57. A fix for a double-announced quiz question replaced a native <fieldset>/<legend> pairing with a hand-built ARIA group, which promptly caused the answer choices to be read twice instead. The fix reverted to native markup and kept only the one piece that had actually been broken.

AI is very good at fixing the thing you point at. It's much less good at noticing what that fix touches on the way past.

The part worth sitting with

None of this is a case against using AI to build accessible courses. It's the opposite, with a caveat that matters. Eighty-three rounds sounds like a lot until you consider what it replaced: an instructional designer without a development background, building an interactive, screen-reader-tested course with working tabs, an accessible quiz, a modal, and an accordion, and catching problems that would normally require a specialist to even name.

But that only works because the expertise was already there. Every round in this log got caught because I knew what "correct" was supposed to sound like on a screen reader, knew the tabs pattern the design called for, knew that a native date field's stepper behavior was a real problem and not a false positive. AI didn't supply any of that. It executed fixes fast once it was told what was wrong and why, and it could not have told me on its own.

So the shift isn't that AI removes the need for WCAG and accessibility knowledge. It's that someone who has that knowledge but isn't a developer can now act on it directly, instead of writing a ticket and waiting for someone else to build it. The tool is powerful. It is not a substitute for knowing what you're looking at. What changes is who gets to build the course themselves, not what you need to know to do it well.

What I'd do differently next time

I started this design in Claude Design and exported to Claude Code (I honestly don't remember which round anymore). My recommendation for the next project: keep a standing reference document instead of re-deriving fixes from scratch every time, and test each interactive component against a keyboard and screen reader individually, before it ever lands on the full page, rather than testing everything together only once the whole thing exists.

That's exactly what the accompanying markdown file is: accessible-elearning-patterns.md, the settled, tested code for every pattern this project needed (tabs, a toggle button, a native radio quiz, a modal, an accordion) plus the wrong turns each one took before landing on the version that held up. The point isn't to hand you a shortcut past testing. It's to hand you a better starting point than round 1.

I'd also start in Design and export to Code sooner rather than later.

Hearing it, not just reading about it

Numbers and a round log only go so far, so Jacob Wood ran a full screen reader walkthrough of the remediated course using NVDA: skip links, semantic headings, labeled buttons, descriptive alt text, and the ARIA techniques that make the tab experience make sense. He narrated what a screen reader user actually hears at each step.

His takeaway lines up with mine. Claude Design is built to get a working skeleton in front of you fast. It isn't built to be accessible by default. Jacob's and Mark Spermon's practical recommendation is to move into Claude Code earlier rather than later, and once you're there, give it a standing markdown file with your WCAG standards and ARIA expectations written into it. Vibe coding without that tends to drift or regress. Fix a table to be semantic in one round, and ten rounds later it can quietly turn back into plain formatted text, because nothing told the model to hold the line.

One correction worth flagging: the quiz's "Submit quiz" button is correct, but only because the demo has a single question (for demo purposes only). If the quiz actually had more than one question, that same button should read "Next" until the final question, then "Submit." This surfaced during Jacob's review and will be updated for future demos.

Want to go deeper?

The full round-by-round log, the reusable prompts, and the "Where ARIA was used, and why" breakdown all live in the demo itself, under the "How this was built" tab. And the pattern file mentioned above, accessible-elearning-patterns.md, is yours to download and build from directly. I'll share the recording from the session when it becomes available.

If you want to go further than the demo and the file, into the testing methodology, the WCAG grounding, and how to run this process on your own content, that's what Making Online Content Accessible for All is for. A new module is coming for Fall 2026.