My AI Said “Fixed.” The Problem Was Still There.

In short: An AI coding tool reporting “fixed” only confirms that the implementation changed and the build passed. It does not confirm that the user’s problem was solved. Verification needs three separate checks: did the implementation change, did the visible behavior change, and did the user’s actual problem improve?

The response looked reassuring.

The component had been updated. The build passed. No errors. The AI even gave me a tidy summary of what it had changed.

Then I opened the page.

The problem was still there.

Not exactly as before. The code had changed. On one screen size, the layout even looked better. But the thing I had actually asked to fix was not fixed.

The CTA was still easy to miss.

That was when I learned to stop treating “code changed” as evidence that a user problem had been solved.

The AI wasn’t lying

It is tempting to blame the tool when this happens.

I don’t think that is quite fair.

The AI had completed the task it understood. It found the component I described, changed the requested properties, and confirmed that the application still compiled.

Its definition of done was roughly:

The requested code changed without breaking the build.

Mine was:

A person arriving on this page can now recognize the primary action without stopping to figure out what to do next.

Those are not the same test.

One checks implementation. The other checks the experience.

I had written the prompt as if the gap between them would somehow close itself.

A correct change can still be the wrong fix

This happens often when building with Lovable, Cursor, Bolt, v0, or any other AI coding tool.

You ask for a button to stand out, so the AI changes its color.

But the button still competes with two nearby actions.

You ask it to improve the mobile layout, so it stops one element from overflowing.

But the tap target is still too small.

You ask it to fix an accessibility issue, so it adds an ARIA label.

But keyboard focus is still invisible.

You ask it to simplify a signup form, so it removes a field.

But the user still does not know why the remaining information is required.

Every change can be technically real. The original problem can survive all of them.

That is especially dangerous on AI-built websites because shipping the change is so fast. A successful build produces a feeling of closure before anyone has checked the result in context.

I was prompting with solutions instead of outcomes

As a product designer, I should have recognized this sooner.

Teams get into the same trouble without AI. Someone reports that a button is hard to find, and the request becomes “make the button green” before anyone asks why it is being missed.

AI makes that shortcut easier.

If I tell it to increase the font size, add contrast, or move a component, it can usually do exactly that. The prompt sounds precise. The output is easy to inspect in a diff.

But precision is not the same as correctness.

The better request starts with the outcome:

On the pricing page, the Start Free Audit action should be the clear next step after a visitor compares the plans. Keep secondary actions available, but do not let them compete with the primary conversion action. Verify the result on desktop and mobile.

Now the AI has a reason for the change, a hierarchy to preserve, and two states to check.

It still may not produce the right answer. But at least we are evaluating the same problem.

Build passing is only the first gate

I now separate verification into three questions.

1. Did the implementation change?

This is the part AI coding tools are good at reporting.

• Was the intended component updated?
• Does the application build?
• Did the change introduce a console or type error?
• Were unrelated files changed?

This gate matters. It just does not prove much about the experience.

2. Did the visible behavior change?

Open the actual page. Use the actual flow.

• Is the change visible at the live route, not just in an isolated component?
• Does it work on desktop and mobile?
• What happens when the text is longer?
• What happens in loading, empty, error, hover, and focus states?
• Did a global style override the local fix?

This is where many of my “fixed” issues turned out to be partially fixed.

3. Did the user problem improve?

This is the gate I used to skip.

If the problem was weak CTA hierarchy, compare the primary action with every competing action around it. If the problem was form friction, complete the form. If it was accessibility, use the keyboard and inspect the focus order. If it was an inconsistent claim, read the homepage and pricing page as one journey.

A UX audit or website conversion audit should ask this question. A code diff cannot.

Screenshots helped, but they were not enough

For a while, I asked AI to take a screenshot after every visual change.

That caught obvious failures. It also gave me false confidence.

A screenshot can show that a page rendered. It cannot prove that a menu opens, that focus moves correctly, that validation helps the user recover, or that the next step remains clear after an error.

It also captures one viewport and one moment.

So I started matching evidence to the claim:

• A visual hierarchy change needs before-and-after screenshots at relevant breakpoints.
• An interaction change needs the flow to be exercised.
• An accessibility fix needs keyboard and semantic checks, not only a clean screenshot.
• A conversion fix needs the full path checked, not just the edited component.
• A cross-page consistency fix needs both pages compared.

This sounds slower than asking AI to “make it better.”

It is faster than discovering three releases later that the original problem never left.

The audit report can make the same mistake

There is another version of this problem that matters to UXtion.

An audit can identify a weak CTA, generate a paste-ready fix, and mark the finding resolved after the code changes.

That is convenient. It may also be wrong.

If the system does not verify the rendered result, “resolved” only means the suggested change was applied. It does not mean the page now communicates a clearer action.

The same applies to an AI SEO audit, an accessibility audit, or an AI search optimization recommendation. A heading can be added in the code but hidden by the rendered structure. Metadata can be generated but never appear on the deployed route. A fix can exist in one component while an older pattern still controls another state.

This is why I have become cautious about products that move too quickly from finding to green checkmark.

The checkmark feels like the end of the work.

Sometimes it is only proof that work happened.

The verification note I use now

For any meaningful fix, I try to save four things:

Problem
What was failing for the user?

Change
What did we alter?

Evidence
What did we check to confirm the result?

Boundary
What did we not verify?

For example:

Problem: The primary CTA disappeared into the dark hero and competed with the secondary demo link.
Change: Increased contrast, reduced the visual weight of the secondary action, and kept the primary label consistent with pricing.
Evidence: Checked the live homepage at desktop and mobile widths, including hover and keyboard focus.
Boundary: Conversion impact is not yet confirmed because traffic is too low.

The last line is important.

Low-traffic founders often cannot prove that a UI change increased conversion. That does not mean they should pretend the fix is verified, or do nothing until they have statistical significance.

They can verify implementation, behavior, clarity, accessibility, and consistency. They just need to label those as evidence, not revenue proof.

“Fixed” should be a claim with evidence

I still use AI to make changes quickly. That part has not become less useful.

What changed is the meaning I assign to its completion message.

When the tool says it updated the component and the build passed, I treat that as a handoff, not a verdict.

The code changed.

Now the page has to make its case.

That distinction is shaping how I build UXtion. Findings should not move from open to resolved because a patch exists. The system should check the surface it can actually observe, preserve the evidence, and be honest about what remains unverified.

I learned that from a green “fixed” message attached to a problem I could still see on the screen.