Skip to content
All posts
Custom Software

AI Wrote the Code. How Do You Know the Software Works?

How to check AI-written software when you cannot read code: write the decisions down first, then test each one. A plain guide for business owners.

You have been shown a demo. The screens look right, the developer clicked through a booking from start to finish, and it worked. Most of the code was written by an AI tool, quickly. You cannot read code. How do you know the software works?

This is the practical question behind custom software now. Writing the code stopped being the slow part. The slow parts are deciding what the software should do (design) and proving that it does (verification), and a business owner can take part in both without reading a line of code.

Short answer: You cannot check AI-written software by looking at it, and you cannot check it against a vague idea. Write down what it must do as plain statements you could mark true or false, including the awkward cases. Then check each statement on a test copy, ask what is tested automatically, and get the result in writing before launch.

Why a good demo proves very little

A demo shows the path where everything goes right: the correct details, entered in the expected order, by the person who built it. Software earns its keep on the other paths.

AI-written code is particularly good at the first path. The tools are built to make the thing you described appear on screen, and they do. What they do not do unprompted is ask what should happen when the customer double-clicks "Pay", when two staff edit the same job, or when a user who should see only their own records types a different number into the address bar.

So "it works" can mean two things: it ran once in front of you, or it does the right thing in every situation you care about. Only the second is worth paying for, and it has to be defined before it can be checked.

Step one: design is deciding, in writing

You cannot verify software against "a system for managing our jobs". There is nothing in that sentence to mark as passed or failed. The design work in a project is turning it into decisions.

A useful decision has three parts: a situation, who is involved, and what must happen.

Vague A decision you can check
"Customers can cancel bookings" "A customer can cancel up to 24 hours before the session and is refunded in full. Inside 24 hours they can cancel but are not refunded."
"Staff can see jobs" "A technician sees only jobs assigned to them. An office manager sees every job at their branch. Nobody sees another branch."
"It sends invoices" "When a job is marked complete, an invoice is created in the accounting system as a draft. It is never sent to the customer without a person approving it."
"It handles double bookings" "If two people try to book the last place at once, the first payment to succeed gets it and the other person is told it is full before they are charged."

The 24 hours in that table is an example. The point is that somebody chose a number.

Writing these is the part of the project that most needs you. A developer brings the list of situations that always come up (cancellations, permissions, failures halfway through, people leaving, data being wrong). You bring the answers, because they are rules about your business. Expect a good developer to ask more questions than you thought there were, and to hand the answers back to you in writing before the build starts.

Three kinds of decision are worth extra time, because they are where an AI tool's silent guesses do the most damage:

  • Who can see and change what. Every type of user, every type of record.
  • Anything involving money. Refunds, rounding, tax, partial payments, what happens when a payment fails.
  • What happens when something goes wrong halfway. The payment succeeded but the email did not. The upload stopped at 90 per cent.

Step two: verification is checking each decision

Once the decisions are written, checking stops being a matter of opinion. Each statement is true of the software or it is not. The checking happens in layers, and you should know which ones your developer is doing.

Automated tests. Small programs that run the decisions against the software every time anything changes. "A technician cannot open another technician's job" becomes a test that tries it and fails loudly if it ever succeeds. These matter more with AI-written code, because a change in one place can quietly break another, and the tests are what notice.

A second reader. Code read by someone other than whoever (or whatever) wrote it. For AI-written code this is a person reading what the tool produced before it is accepted, with most attention on permissions, money and data.

A test copy. A full working copy of the system, at a private address, with made-up data. This is where you come in.

Monitoring after launch. Something that tells a person when the software fails, and backups that have been restored at least once to prove they work.

What you can check yourself, without reading code

On the test copy, with your list of decisions beside you:

  1. Do each thing the wrong way. Leave fields blank. Enter a quantity of zero, a date in the past, a name with an apostrophe. Click the button twice.
  2. Log in as each kind of user. Check what each can see. Then, logged in as an ordinary user, open one of your own records, change the number in the address bar and reload. You should be refused.
  3. Follow the money. Run a payment, a refund and a failed payment on test details, and check the totals against a calculation you did by hand.
  4. Break it halfway. Close the browser in the middle of a booking. Go back. What state is it in?
  5. Use your worst real example. Every business has the customer or the job that never fits the normal process. Put that one through.
  6. Tick the list. Every decision gets a pass or a fail, in writing. Anything that fails is fixed and tried again before launch.

You will find things. That is the purpose of the exercise, and it is far cheaper for you to find them on a test copy than for a customer to find them on the real one.

Questions to ask a developer about verification

  • Will I get the decisions in writing before the build starts, and can I change them?
  • Which of those decisions are covered by automated tests?
  • Who reads AI-written code before it goes live?
  • Is there a test copy I can use while it is being built, and from when?
  • How will you know if it fails after launch, and who is told?
  • Have the backups been restored, and when?
  • If it does not do what we agreed, who pays to fix it?

Vague answers to these are information. So is impatience with the questions.

How Rootech handles design and verification

Rootech is a Melbourne software studio working Australia-wide, with delivery outside Melbourne done remotely. The promises we make on every build are the two steps above:

  • Design: scope and price are agreed in writing before you sign, so the decisions are on paper before any code is.
  • Verification: the build runs on a live address you can test as it goes, and bug fixes on anything we built are included after launch. If it does not work the way we agreed, that is ours to fix at no cost.

You own the code, the accounts stay in your name, and launch comes with a handover of the code, documentation and credentials.

If you already have software an AI tool built and nobody has checked it, a vibe code cleanup starts with a review of exactly that. If you are starting fresh, tell us what you need at /#contact. For the bigger picture of why the work has shifted this way, read Custom Software Development in the Age of AI: What Changed.

Frequently asked questions

Can I ask an AI tool to check the code an AI tool wrote?

It helps, and developers do use AI tools to review code and write tests. It is not enough alone, because the checker can share the writer's blind spots, and neither knows your business rules unless someone wrote them down. A person still has to decide what "correct" means and confirm the result.

How many decisions does a typical project have?

It depends on the software, and there is no useful average. A simple internal tool might have a couple of dozen. A booking or payments platform has many more. If a brief for a system that handles money and several kinds of user fits on one page, decisions are missing.

Do I have to do the testing myself?

No, the developer does most of it. Your part is the checks only you can do: whether the rules match how your business really runs, and whether your worst real example goes through. An hour or two on the test copy before launch is usually well spent.

What if I find a problem after launch?

Report it with what you did and what you expected. If the software does not do what was agreed, a developer who stands behind the work fixes it at their cost. If it does what was agreed and you now want something different, that is a change, and it should be quoted as one.

We already launched without any of this. What now?

Write the decisions down now, starting with permissions and money, and check the live system against them carefully, using test accounts. If real customer data is in it, start with Is Your Vibe-Coded App Safe With Real Customer Data?.

Free AI workflow audit

Wondering what this looks like for your business?

We’ll map where your hours and leads are leaking, and hand you a prioritised plan with dollar figures attached. The audit fee comes off your first project.

Get in touch