Many organisations have enabled generative AI. The hard part now is turning it into measurable ROI in software delivery without trading away quality, security, or delivery confidence.
At the same time, the market conversation is still skewed toward tools and individual productivity: prompts, demos, “10x” stories, and personal workflows. Useful, sometimes impressive, but it’s not where outcomes get decided.
Outcomes get decided one level up: in team and project-level choices. Shared quality bars. Consistent ways of working. How you review, test, ship, and learn when AI becomes part of the pipeline.
I think we’re stuck in tool-talk for a familiar reason. Generative AI is general-purpose and shows up everywhere across delivery: code, documentation, testing, design, ops. When a technology touches everything, it’s easy to treat it like a special case and forget we’ve been through adoption curves before. The playbook hasn’t changed: make deliberate choices, measure what matters, embed the habits.
AI works throughout the entire delivery lifecycle; from idea generation, guidance and direction as people work, all the way through to an autonomous contributor. It amplifies engineering discipline, it doesn’t replace it. Which means the real question becomes: what system of work are you trying to improve?
So I sat down with Serge Matheson and Warner Godfrey, two of our most experienced engineering leaders at DiUS, to ask: what does it take to turn generative AI from uneven individual productivity into repeatable delivery outcomes without increasing risk?
Here’s the framework we keep coming back to: build consistency, measure what matters, then embed the practice until it sticks.
First, build consistency
The thing that has to be consistent is the way teams work, not the tool choices people make on their own. Consistency is how you get repeatability, always-on guardrails, and compounding learning across squads.
In practical terms, we see consistency come down to team-level decisions:
- Where AI fits in the delivery workflow (and where it doesn’t). Example: AI for test scaffolding and refactors, not for approving production changes.
- What “good” looks like for AI-assisted work. A clear quality bar: readability, tests, security checks, documentation, and review expectations.
- Where a human stays in the loop. Reviews, releases, and high-risk decisions stay owned by accountable humans.
- What traceability looks like when things go wrong. You can explain what changed, why it changed, and how it was checked.
Serge’s point is that “human in the loop” isn’t just a process step. It’s ownership, individual and team-level, and the ability to explain what you’re putting into production. “With all this generated code, the most important responsibility for the individual engineer is to understand it. Which means, of course, reading code is an even more important skill than writing it.”
And there’s a second point Serge was keen to emphasise: accountability doesn’t move just because AI is involved. “What AI creates for us as engineers is our responsibility. It’s a tool that we use, not an autonomous team member.”
Finally, consistency only sticks when it’s embedded into day-to-day delivery. The safe way has to be the easy way; otherwise you get speed, plus shadow AI, plus surprises later.
Second, measure value
Warner’s test is pragmatic: are we shipping better outcomes, faster without adding fragility or risk?
That means measurement has to start with the desired outcomes, not what your tools make easy to measure. If it feels like policing, people will game it. And if you only measure what’s easy, you’ll miss what matters: whether you’re solving real customer problems, minimising technical debt and operating sustainably.
A practical approach we see working:
- Measure team outcomes, not individual usage.
- Start small with signals you trust.
- Pair quantitative signals with short qualitative check-ins.
- Use measurement to learn and improve, not to enforce compliance.
One useful line to keep in mind: adoption is a leading indicator. Delivery and quality are the proof.
Practically, both Serge and Warner recommend resisting heavy instrumentation early. Start with a lightweight baseline, then add depth where it actually helps you make decisions.
Warner emphasises, “Tools that automate measurement often skew toward engineering activity rather than delivering customer value. If you want a holistic view, you need to intentionally measure outcomes, not just output.”
Above all, you’re aiming to answer four defensible questions:
- Are we actually using it? Is AI part of how teams work, or a pilot on the side?
- Does work move faster? Are feedback loops shortening and throughput improving where it matters?
- Are we building the right way? Are we moving faster without creating fragile code, extra incidents, or hidden rework?
- Is it worth it? Are the costs justified against time saved, delivery impact, and team experience?
That combination gives leaders both a credible ROI story and a clear basis for what to scale, stop, or fix next — without turning measurement into surveillance.
Third, embed the practice
Embedding is where practice becomes real. It’s the side-by-side work that takes the messiness of real delivery, constraints, governance, legacy, all of it, and turns it into repeatable patterns.
This is the step most organisations skip. They enable the tools, but they don’t change the system. So the gains stay uneven: a few power users get faster, but the team doesn’t.
Warner’s view is blunt: “AI won’t fix broken delivery practice. You still need to diagnose the pain points, such as fragmented organisational knowledge, interdependent work, rubber-stamp reviews, lengthy pre-release verification, and then apply AI where it genuinely helps.”
A simple example of what “embed” looks like in practice: pick one workflow that matters (a tricky legacy change, a high-friction release, a slow review loop) and run it end-to-end with an exemplar team. Capture the patterns, make the decisions explicit, then roll it out as the default way of working.
Which is exactly the point. ROI doesn’t live in a coding tool. It lives in how teams deliver value.
This is the capability uplift playbook we use in delivery: build alongside teams, reduce risk, and leave behind repeatable AI practices.
