Building an assistant that isn't allowed to be wrong

At the market, nobody likes the one who makes things up

You arrive at the big market looking for a particular trader. You ask for directions. Two possible answers.

The first: “go straight, turn left after the basins”. You walk for twenty minutes. It wasn't there. The person didn't know, but preferred saying something to saying nothing.

The second: “I don't know, ask that lady over there”. You lose ten seconds. You find it.

The second answer is worth more than the first. That is the whole difference between a useful assistant and a dangerous one — and it's the sentence that decided the architecture that follows.

Now replace the market with a health mutual, and the directions with a reimbursement rate. Being wrong no longer costs someone a twenty-minute walk: it becomes a dispute. That is the kind of assistant I build, and that sentence decided its entire architecture.

I can't name the real project: the setting is moved, the technical decisions are exactly the ones I made.

The instruction you forget, the padlock you can't get around

You can tell your children: “don't go out without telling me”. That's an instruction. It holds as long as everyone remembers it.

Or you can put a padlock on the gate. Then the question doesn't arise.

The program answering members is the instruction. The database is the padlock. And I put the rule in the padlock.

Concretely, the database physically refuses to store a published answer without three things: the document it comes from, the date it was checked, and the confidence level. It isn't a test a developer might forget to run. It's a wall.

Second padlock: a published answer can never be edited. Ever. If the information changes, a new one is published and the old withdrawn — but the old one stays, dated.

Why so rigid? Because if a member disputes in October an answer received in March, you must be able to show exactly what the system said in March. A row that can be quietly edited makes that proof impossible.

A developer would say: “the invariants are in the schema, not in the application”. Same thing, said in jargon.

Finding an address in Cotonou

In Cotonou nobody searches by street name. People say: “it's past the junction, next to the pharmacy, opposite the blue bar”.

These are two ways of searching, and they don't find the same things.

By exact name. If you know the name, you land on it directly. One letter wrong and you find nothing.

By landmarks. You describe what's around it. It works even if you don't know the name — but you may land on the pharmacy next door, which looks very similar.

A member writes “glasses”. The contract says “medical optical equipment”. Exact-name search finds nothing. They write “I just gave birth”; the contract says “maternity benefit”. Again, not a single word in common.

So we keep both methods running in parallel. One looks for exact words, the other looks for meaning. Each catches what the other misses.

The technical name for the second is “vector” search: each sentence is turned into a series of numbers representing its meaning, and the numbers are compared. Two sentences meaning the same thing end up close together, even without a single shared word.

Kossi counts the visits, Fifamè marks out of ten

So we have two engines, each answering on its own side. Now they must agree. And that's where it gets tricky.

Take an ordinary situation: you're looking for a good local restaurant, so you ask two friends.

Kossi tells you how many times he has eaten there. It can climb to forty-seven. Fifamè marks out of ten.

If you add their numbers, you're only trusting Kossi. His forty-seven crush a mark that will never exceed ten — even when Fifamè gave two out of ten to the place he loves.

The fix is one sentence: don't add their numbers, add their rankings. First, second, third. Now each person's way of counting no longer matters, and the place both rate highly wins.

Reciprocal rank fusion, a simple method for combining the rankings from multiple systems, consistently yields better results than any individual system.

Cormack, Clarke & Büttcher — SIGIR 2009 ↗

The two search engines have exactly this problem. One returns scores with no upper limit, the other scores between zero and one. So we compare positions, not marks. It has a fancy name — reciprocal rank fusion — for an idea everyone already applies without knowing it.

Knowing how to say “I don't know”

We set a minimum level. Below it, the assistant doesn't answer: it hands over to a human advisor.

It's the motorbike taxi who tells you “I don't know that neighbourhood, take that guy, he goes there every day” instead of circling for an hour with the meter running.

A mistake I nearly made: applying that threshold after blending the two engines. It makes no sense — once blended, the resulting number no longer measures anything real. The threshold applies to each engine separately, before the blend, while its numbers still mean something.

And we count how often the assistant stays quiet. If that number climbs, it isn't malfunctioning: people are asking questions whose answer hasn't been written yet. That counter tells you what to write next.

The transfer went through, the message didn't

You send someone money by mobile money. Then you call to tell them. Two actions.

If the money leaves but the call doesn't connect, they don't know it arrived. If you call but the transfer fails, they wait for nothing. Two actions means two chances to fail.

The system has the same problem. When an answer is published, it must be stored *and* the rest of the system must be told. Two operations. Two ways to half-fail.

The fix is simple once you see it: the message to send is written into the database, at the same moment as the data, in a single action. A small program then comes along, picks the messages up and delivers them.

Either both succeed or neither does. The half-failure disappears. Developers call it the outbox.

The service that sends the message first stores it in the database as part of the transaction that updates the data; a separate process then publishes the messages.

Chris Richardson — Pattern: Transactional outbox ↗

Three tricycles or one articulated lorry?

You have to deliver twenty bags of rice a day across the city.

The lorry carries it all in one go. But you need a heavy-goods driver, a garage that can fix it, somewhere to park it, and it doesn't fit down small streets.

Three tricycles do the same job. You fix them yourself, they go everywhere, and if one breaks down the other two keep going.

That's exactly the choice between the two tools that run this kind of system. The first — Kubernetes — is the lorry: very powerful, used by the web giants, and it needs a team to look after it. The second — Docker Swarm — is the tricycles.

For three or four machines and a dozen programs, the lorry becomes a project of its own. I took the tricycles.

In practice: programs are spread across machines according to their role, passwords never sit in configuration files, and traffic between machines is encrypted. All without one person losing their weekend to it.

The day the volume outgrows this, the question comes back. Not before. You pick the tool sized for the problem, not for your CV.

The generator you start before the cut

The power is about to go. You're going to start the generator.

If you wait for the cut to start it, you get two minutes of darkness. The fridge stops, the computer reboots, the connection drops.

If the generator is already running when you switch over, nobody notices a thing.

Updating a running program is the same. The bad method stops the old version then starts the new one — and in between, nothing answers. The good method starts the new one first, checks it works, and only then shuts the old one down.

And if the new version misbehaves? The system goes back to the previous one by itself. Not a procedure written in a document nobody re-reads at 3am: an automatic rollback.

The night guard who writes “nothing to report”

A guard can write “all fine” all night long. It doesn't tell you whether the gate was locked.

That's the trap of computer monitoring. We measure memory, processor, speed. All those numbers can be green while the system tells members complete nonsense.

So the most important alert in this whole system watches no machine at all. It counts answers sent without a source document. Its normal value is zero. At the first deviation, immediate alert — because a promise has just been broken.

Alongside it: how often the assistant stayed quiet, response time, and requests passed to an advisor still sitting there after two days. Every alert maps to a promise made to someone.

One last point, and it matters: activity logs are stripped of personal data before being stored, not after. Cleaning afterwards means you cleaned nothing.

What I'd do differently

Three things, honestly.

I spent too long tuning details before having any way to measure whether my tuning improved anything. Without measurement you optimise blind and call it intuition.

The “I don't know” threshold should have been adjustable question by question from the start. Being wrong about opening hours and being wrong about a reimbursement rate are not the same mistake.

And I'd have installed the monitoring on day one, not day thirty. The first three weeks of a live system are the ones that teach you the most — I watched them through a dirty window.

What to take away

None of these decisions is spectacular. None uses the newest technology. They share one thing: they all answer the same sentence, set down before the first line of code.

An architecture isn't a stack of good technical choices. It's one constraint held all the way through, including where it hurts — in the database, in the alerts, in how you deploy.

If you're building something that isn't allowed to be wrong, the real question isn't “which technology?”. It's: what must this system never do — and what stops it when everyone has forgotten the rule?