Data readiness: why the model is rarely the bottleneck
Your AI project is behind. Someone on the team says the next model release will fix it. That is a comfortable thing to believe, because it means the fix is on its way and nobody has to do anything hard.
Look at what the team is waiting for this week, though. It is usually not a smarter model. It is an access request that has sat in a queue for a month. It is a question nobody can answer: which of these three customer lists is the correct one? It is a folder of scanned forms that no system can read.
The model is the easy part. You can rent a good one this afternoon. You cannot rent your own records in a usable state. That work is yours, and it is where most of the calendar goes.
A quick test: is it the model or the data?
Before you blame the model, run one check. Take ten real cases. For each one, collect everything a person would need to do the job well: the right records, the current policy, the history of the case. Paste all of it into the model by hand and ask for the answer.
If the answers are good, the model is fine. Your problem is getting that same information to it automatically, every time. That is a data problem.
If the answers are still poor with everything in front of it, then you may have a model problem, or a task that is not well defined. In practice the first result is far more common. And notice how long it took you to collect the information for those ten cases. If it took a day and four phone calls, you have just measured your bottleneck.
The three things your project is really waiting on
1. Access
The data exists, but your team cannot reach it. It sits in a system owned by another department. Getting to it needs a security review, a change to a vendor contract, or an integration that someone has to build and maintain.
None of that is unreasonable. It is slow, and it often starts late, because the pilot ran on an export that someone emailed over. The export hid the problem. The real request gets filed the week you planned to go live.
What to do: list every system the project needs to read from and write to. File every access request in week one, before any build work. Ask each system owner one question: "What do you need from us to approve this?" Then track those requests the way you track the build, with a name and a date on each one.
2. Ownership
Every useful piece of data raises questions that only a person with authority can settle. Which system is the source of truth for a customer's address? What does "active customer" mean? Is the team allowed to use support transcripts for this purpose? Who fixes a record when it is wrong?
If nobody owns the data, nobody can answer. The questions go around in email. Each one is small. Together they can hold a project for a quarter.
What to do: for each data source, write down one name. Not a department, a person. That person decides what the fields mean, approves how the data is used, and gets the call when records are wrong. If you cannot find a name for a source, raise that with your sponsor now. A source with no owner is a risk to the project, and it is better on the risk list in week one than discovered in month four.
3. Messy records
Then there are the records themselves. You will find some mix of these:
- Duplicates. The same supplier entered three times with three spellings.
- Empty or misused fields. The "reason" field is blank in half the records, and the real reason is in a free-text note.
- Stale documents. Four versions of the same policy in a shared drive, with nothing to mark which one is current.
- Unreadable formats. Scans, photos of forms, handwriting, tables saved as images.
- Knowledge that is not written down. The rule everyone follows lives in one experienced person's head.
A model does not fix any of this. It reads what you give it. Give it two conflicting policies and it will pick one, or blend them, and sound confident either way. A stronger model gives you a more fluent wrong answer.
What to do: pull a random sample from the live system, not a selected one. Fifty to a hundred records is enough to start. Sit down with someone who does the work and go through them one by one. For each record ask: could a new hire handle this case with only what is on the screen? Mark each one yes or no, and write down why for each no.
What "ready" means
You do not need clean data across the company. That project never ends, and waiting for it is another way to stall. You need data that is good enough for one use case.
A simple definition: your data is ready when a competent new hire, given the same records and documents the model will get, could do the task correctly most of the time. If a person would need to ask a colleague, the model needs that answer written down too.
This keeps the cleanup small. You are not fixing the customer database. You are fixing the six fields this workflow reads, for the records this workflow touches.
A two-week readiness check
Run this before you commit a build budget. It takes one or two people and no new tools.
- Days 1 to 2: map the inputs. List every system, document set, and field the task uses. Include the unofficial ones, like the spreadsheet the team keeps on the side.
- Days 2 to 3: file access requests and name owners. One request per system. One name per source.
- Days 4 to 8: review the sample. Pull the random records. Score each one as usable or not. Group the reasons for the failures.
- Days 9 to 10: sort the problems. Put each problem in one of three piles: fix now, route to a person, or leave out of scope.
Here is an illustrative example, with made-up numbers. Say you want help answering warranty claims. You review 100 past claims. In 60, everything needed is in the record. In 25, the purchase date is missing and the clerk found it in an email. In 15, the claim refers to a product line whose warranty terms exist only as a scanned booklet.
Now you have a plan. Launch on the 60. Make the purchase date a required field on the claim form from today, so the 25 shrink over time. Get the booklet typed up once, which is a few days of work. Until then, send those 15 to a person. None of these steps involves changing the model.
Fix the source, not the copy
When records are messy, the tempting move is a one-time cleanup: export the data, tidy it in a spreadsheet, and feed the tidy copy to the system. It works for the demo. A month later the live data has drifted and the copy is wrong.
Fix the problem where the data is created instead. Make the field required. Add a dropdown where people type free text. Agree one home for the current policy and archive the old versions. These are small changes to forms and habits, and they keep paying off whether or not the AI project continues.
Budget for this honestly. In your plan, give data work its own line, with its own people and dates. If the plan shows twelve weeks of build and zero weeks of data work, the data work will still happen. It will just show up as delay.
Before you ask for a better model
Ask three questions first. Can the team reach the real data today? Is there a named person who can settle questions about it? Have you looked at a random sample of records with someone who does the job?
If any answer is no, that is your bottleneck, and no model release will move it. If all three are yes and the results are still poor, you have earned the right to look at the model. Most projects never get that far, because fixing the first three was enough.
If you want more writing like this, the Hyperautomation AI Report is a free newsletter, three times a week, with practical notes on building and using AI. You can subscribe on the newsletter page.
Talk to me about AI in your company
If you're working through an AI rollout, tell me what you're trying to do. I read every message and reply by email.