I was recently invited to take part in an evaluation of the AI rollout at a city government’s research and evaluation commission. I’d like to share what I saw and what I suggested (just one person’s view, discussion welcome). I wrote about a similar review in Civil Servants Aren’t Behind on AI.
What I saw:
- Internal stats show 89% of staff used AI regularly as of end of July, and 81% as of end of August
- About 60% of staff save 1–2 hours a week, and about 20% save 3–5 hours
- The order made sense: start with a small local version, get it running, then expand step by step
- They counted the “invisible time invested” too, and estimated roughly 8 months to pay back against the time saved
Whether it’s 89% or 81%, I think these are impressive results.
My suggestions:
1. People are accountable. AI is not. AI is producing output almost faster than humans can review and verify it. Whatever permissions you hand to AI, you have to find the key release checkpoints in the end, and build people’s ability to approve and verify.
2. Use risk to decide the order of adoption: reversible or not, easy to verify or not. I suggested starting with what is “reversible and easy to verify”, and leaving “irreversible and hard to verify” for last.
3. To save more time, the bottleneck is mostly not AI but the surroundings. Data governance, security and permissions, and the business logic itself. Rather than piling more on top of old processes, subtract first and tidy things up once. (I wrote about this separately: Most of the Benefit from AI Adoption Has Little to Do with AI.)
4. Judge quota by efficiency, not just usage. Staff asking for “higher quota” instead of “a unified tool” means people really are using it, and that’s good. But many companies abroad found that part of their quota was being burned for nothing. This is called tokenmaxing. Model routing matters as much as work routing; not everything should go to the LLM.
The last point I wrote more bluntly in my comments: management in an organization usually cares most about efficiency gains, but frontline staff usually care most about “not making mistakes.” How to handle this “principal-agent problem” is also something to watch as adoption continues.