Recently my team took an AI assistant for a government agency from a working prototype to something ready for real use. People ask it questions in plain language, and it answers from the agency’s own data: programs, regions, budgets, progress over the years.

The prototype was impressive within minutes. Getting it ready for production took a lot longer, and taught me more than I expected. Here’s what stayed with me.

The model is the easy part

When people picture an AI assistant, they picture the model. In practice, the model was the most stable part of the whole system. Most of the work was everywhere around it.

Before the assistant could answer anything, the data had to arrive, be complete and make sense. We built a pipeline to pull data from the agency’s existing systems, page by page, and store it in a form the assistant could use. Along the way we found endpoints that returned errors, records with gaps, and fields that meant slightly different things in different places.

None of that is visible in a demo. All of it decides whether the answers are right.

Never give the AI more power than a junior analyst

Part of the assistant answers questions by writing its own database queries. That’s powerful, and it’s also exactly where things can go wrong.

So we treated it like a new analyst on their first day. It gets a read-only database account. It can only see the tables it actually needs. It can’t change anything, and it can’t reach anything outside its own area. If the model ever writes a bad query, the worst outcome is a failed answer, not damaged data.

The rule I keep coming back to: design as if the AI will eventually do something wrong, because one day it will.

Specialists beat one big brain

Instead of one assistant trying to do everything, we split the work across several specialized agents: one to understand the question and route it, one to look up numbers, one to explain, and so on. Each one has a narrow job and a clear set of tools.

Narrow agents are easier to test, easier to fix, and easier to trust. When something goes wrong, you know where to look.

In government, correct beats fast

There’s always pressure to make AI feel instant. We had the option to make the model think less so it would answer faster. We decided not to.

For a public institution, a wrong number delivered quickly is worse than a right number delivered a few seconds later. Someone might put that number in a report or a presentation to leadership. So we kept the reasoning. If we need more speed, the place to find it is elsewhere: running independent steps in parallel, and skipping agents a question doesn’t need.

Where the data lives matters

The prototype ran on convenient managed cloud services. For production, we moved everything to infrastructure we control: our own database, inside the agency’s environment, with access that we can see and audit.

Government environments are their own world. Some servers have no direct internet access at all, so even installing basic tools takes planning. Changes go through people and processes, not just code. That isn’t a problem to work around. It’s part of the job, and the system has to be designed for it from day one.

Public means probed

As soon as the assistant was reachable from the internet, the scanners arrived, testing for known vulnerabilities in the frameworks we used. That’s normal now, and it’s a good reminder that “launch” is the beginning of maintenance, not the end of the project.

Rate limits, patched dependencies, locked-down servers, daily backups. None of it is exciting. All of it is part of what makes a system trustworthy.

What I’d tell anyone starting one

• Start with the data, not the model. Know where every answer comes from.
• Limit what the AI can touch. Read-only by default, and only what it needs.
• Keep the pieces small. Several focused agents beat one that does everything.
• Choose accuracy over speed when people will act on the answer.
• Plan for the environment you’re deploying into, not the one you developed in.

AI makes the first version of a system faster to build than ever. Making it something an institution can rely on is still the same old work: careful data, clear boundaries, and steady maintenance. That part hasn’t changed, and I don’t think it should.