In June 2025, I wrote that the AI boom felt like the dotcom era all over again. The chaotic demos, the investment and the excitement of discovering what we could build all felt familiar. I was optimistic about what engineers who understood systems and cared about outcomes could do with it. I still am.
Now everyone is asking where the return on investment is. That’s a reasonable question. I want everyone across Engineering, Product, Design and go-to-market, including leadership, to be revolutionary here, not evolutionary. We have an opportunity to reconsider how a customer problem gets solved, which work needs doing at all and where people should spend their attention.
Laurie Voss returned to the dotcom comparison in October, with Apollo says AI hasn’t paid off outside tech. That’s because it’s 1997. He argues that uneven early returns don’t settle the technology’s eventual value, and that automated evaluation can make AI useful at a different scale. I agree. My interest is how we change the way we develop, sell and support software to realise those gains.
Faster implementation doesn’t guarantee a business result
I’ve written about individual gains failing to become team gains. An engineer can produce changes much faster while delivery still waits for the same reviews, decisions and commitments. We can also ship sooner and discover that customers don’t need what we built.
The ROI question should make us examine the whole route from a customer need to a useful result. Using AI to write the specification, produce the code, review it and update the project tracker may make each stage cheaper. We should also ask which stages, handoffs and approvals still need to exist.
That is the argument behind Don’t Automate the SDLC. Removing a handoff matters if a customer gets a problem solved sooner, a service becomes affordable or the company can take on valuable work it previously couldn’t do. Generating more artefacts doesn’t establish any of those things.
Redesign the work across development and go-to-market
Suppose people sign up, but too few reach a useful first result and become paying customers. The familiar response might be to commission a new onboarding experience, then move through requirements, design, implementation, review and release. AI could make each stage faster while leaving the chosen solution and delivery process largely intact.
In the model I described in What Replaces the SDLC?, a team would work continuously around the customer and business outcome. Agents could help investigate where customers struggle, produce working alternatives, evaluate them and observe their effects within agreed permissions. The intervention might change the product, remove a step, improve information or involve a person. People would make the consequential choices and resolve questions that the checks couldn’t settle.
This is close to what my growth team is doing now. When the data shows meaningful impact, we productise those changes. The experiment gives us evidence to inform that investment. Productising the change means taking responsibility for keeping it working for customers.
Product, Design, Engineering and go-to-market need to work around the same customer and business outcome. In the growth example, the problem may be in the product, the promise made before signup, the price or the help available afterwards. Marketing and Sales need to help shape positioning, pricing and customer commitments while options are still being explored. Customer Success and Support can contribute what they see customers struggling with and help decide how a change should be introduced and supported.
Leadership needs to examine the higher-level choices. Is activation the right outcome to invest in? Which customers and revenue opportunities matter? What else could the investment achieve? What would it cost to productise, operate and support the change? Leaders across those functions need to reconsider the reporting boundaries, handoffs, approvals and incentives that could prevent this way of working.
Leaders need to ask these questions of their own plans too. Some haven’t kept up with what is possible, are too risk-averse to reconsider the process, or have never been good at examining total cost of ownership and business impact. Commercial judgement is often a particular weakness for technically focused leaders. Technical ability and business acumen are both requirements at senior engineering levels.
The ambition of the redesign and the size of the first deployment are separate decisions. A team can test a substantially different approach with a small group, observe what happens and expand it as the evidence improves. Customer promises and operational responsibilities still need owners throughout.
What evaluations can establish, and what we still need to judge
Working continuously around an outcome means deciding which changes to keep, which to stop and how much authority to give the agents doing the work. Laurie’s recommendation is to define good in a form software can check. Reliable evaluations let us test more options and move routine work past repeated manual review.
Laurie also argues that tasks needing judgement are ones where we haven’t yet written down a concrete, repeatable definition of good. I don’t think that covers every decision we make. Sometimes we lack a useful rule; sometimes people disagree about which outcome matters, which customer promise takes priority or what risk the business should accept. The people responsible for the product and business need to make those choices and revisit them when conditions change.
Once those decisions are reflected in evaluations and permissions, routine work can proceed without somebody approving each action. We still need to compare the results with what happens to customers and the business. If the checks pass while customers remain stuck, or a conversion gain depends on a promise the business can’t afford, we need to revisit the evaluations and permissions.
I’m an Octopus Energy customer, and the human touch is one reason why. In its July trial report, Octopus reported customer satisfaction of 76% for its Arlo assistant’s routine email replies and 72% for comparable human replies. Sensitive cases, vulnerable customers and complex complaints went to people; AI emails were labelled and customers could ask for a human.
Those are company-reported results, without disclosed full costs or profit impact. I want a straightforward question answered promptly, and I also want to reach somebody when my circumstances need understanding. A satisfaction score for routine answers doesn’t tell us whether the harder conversations got better.
Define the return you expect from the change
For each investment, we need a clear outcome, a baseline and someone responsible for following the result. In the growth example, are more customers reaching a useful result and becoming paying customers? Does the effect last? What does the change cost to build and run? The answer should include the tooling, integration, evaluation, supervision, support and operating costs.
A company earning more, serving customers better or reducing its operating costs is a business result. A person finishing a task faster may contribute to it. We need to follow the effect far enough to establish the connection.
Some gains will reasonably be reinvested. A team might use the capacity to improve reliability, handle more customers or do work that had never been affordable. That can be valuable without producing an immediate headcount reduction. Somebody still needs to say what improved and show the evidence for it.
An experiment can be worth doing because it resolves uncertainty. We need to agree in advance when we’ll review it, what would justify expanding it and what would make us change the approach or stop.
Leaders across development and go-to-market need to change how their functions work together. I argued in Organising for a Reimagined SDLC that reducing coordination overhead should be an explicit leadership responsibility. That includes agreeing who makes which decisions, who can commit the business and who takes responsibility for the result. Teams need time, support and authority to investigate the whole customer and business problem. Leaders need to own the investment decisions, the way their functions work together and the business result.
Start with a customer outcome that matters to the business, whether that’s improving conversion, retaining customers or making a new service affordable. Rethink how it could be achieved with the capabilities available now, agree what would make the investment worthwhile and test that approach.