I'm lucky enough to serve the technology teams that build and run some of the most important systems in the world. These systems process more than $5 trillion dollars in payments every day. They keep the nation's - and by extension the world's - financial systems moving. I work with great people on real problems, with real impact on people's lives - whether those people know it or not. If we do our job, then it's "not". "Not" is good. Contemplating that responsibility has really informed my perspective.
In that context, engineering isn't just building systems: it's building systems that are sustainable, resilient, high performing, and reliable. I think not only of development, but operations. Building systems to handle outliers and failures, happy paths, and unexpected or unknowable paths that lead only into the Swamp of Sadness*.
As professional engineering organizations, we need to not only launch, but land our systems and features, even as the world is moving and evolving faster than ever; things will never move this slowly again, and the pressure to accelerate isn't going away.
AI, the great accelerator. Maybe, kinda, sorta, it depends?
I've written before about the research that shows that AI can cause the perception of increased productivity, even while the effect is the opposite. Nevertheless, it does seem fairly undeniable that AI is getting better at producing plausible code. Messing about with Fable I found it to produce notably better output than longer-in-the-tooth Opus models for me.
Shifting the bottleneck. Changing the role.
So as an industry, production of code is now "cheap", but our build processes are not designed for cheap, high volume code. In this view, verification and deployment are the new bottlenecks. I'm inclined to believe this, given VC investments in this space (Coderabbit, Blacksmith, Greptile etc). The basic idea is we can produce code faster, but now review, verification, validation, and moving it to production are what's holding us back.
That's probably solvable, but it requires a material change in the role of engineers, from coders and writers, to specifiers and reviewers. Technology - AI, predominantly, but also tooling redesign - will help, but ultimately humans still need to be in the lead. Bob Martin notably disagrees here and has said he no longer reviews code at all.
In either case, AI-led engineering is requiring engineers to move up the abstraction stack, from crafters of clean, understandable, reusable code, to architecture, design or other specialisations: arguably a good thing, but also with a much heavier emphasis on review than production and problem solving. The role is also changing from engineering to meta-engineering: producing the harnesses, mediating work product, orchestrations, agents etc. that actually do the engineering>
Anecdotally there's muttering coming from the field of burnout, and frustration with the changes in the nature of the role. It is concerning that the alleged increase in productivity might be coming at the cost of some of our strongest talent stepping away from the role.
Risk as the new bottleneck
Remember the principles behind DevOps, Agile, CI/CD? Smaller more frequent deployments reduce complexity and risk, and decrease change failure. So what happens when we're deploying product with an order of magnitude more changes (presupposing we effectively manage the verification problem)? If we're deploying with the same frequency, we're back to waterfall-style mega-deployments - just much faster. Even solving the validation problem, that's a big risk bubble.
So if those principles are still something we aspire to, we have to deploy more frequently - or continuously. In most organizations that's a pretty significant governance change, and material shift in risk posture. Without that intentional re-evaluation, organizations are unintentionally accepting risk by assuming that a two weekly deployment of AI code is the same as a two weekly deployment of human originated code. It's not.
OK, so AI might help us produce code faster, but also likely shifts the bottleneck from typing to tooling, and introduces new degrees of risk. Once the parameters are evaluated, that's a tradeoff that can be made based on our tolerances.
But...
The atrophy problem
There's research that the cognitive offload of work to machines leads to skill atrophy or non-formation (for example: here and here). I've seen this happen myself: folks who were once extremely capable, struggling in the absence of AI, to complete tasks they formerly found easy. The production of code that the engineers no longer understand is cognitive debt, and like all debt, it comes with an interest rate.
This has a significant impact on validation - the exact stage when humans are most necessary, even as simple as failing to identify a missed null check (here). This kind of atrophy has the net effect of reducing the derisking effect of the whole validation and verification process.
What really scares me though is what happens when something novel goes wrong in production, and the engineers responsible neither understand what happened, how the code works, or how to figure it out - in part due to cognitive debt, and in part to skill atrophy. I've seen a growing number of stories describing this exact problem when a production failure was effectively met with blank stares, and inability to troubleshoot (for example: here and here).
The AI that wrote the code can't always help, and in an incident, may not even be available right when you need it most. A startup can probably stumble through this kind of mess and be OK. Critical systems on which millions or billions of people depend? Not so much.
I'm not going to pretend I have the answers. I have hypotheses, but not answers. A lot I think, depends on how we lead, enable and incentivise our teams. I'm a big fan of letting others make my mistakes for me, so gathering data from observation, conducting experiments, reading research, will all continue to help shape my thinking here.
If we're really lucky, someone will figure all of this out. More likely, after enough time has passed, and atrophy set in, someone will have a catastrophic failure and struggle to recover, and we'll face a reckoning with the issue - and maybe a complete realignment. Ultimately, in my view this is a management and leadership challenge, and it's complicated not by the need to evolve our ways of working, but also the one constant, that humans are going to behave in human ways. Complacency, the human desire to take the path of least resistance, and plain old laziness are real.
In the meantime, I'm going to keep experimenting, and regardles of how AI progresses (or doesn't), and how engineering methodologies evolve, my key focus is on cultivating a world class engineering organization that's vibrant, and continues to be expert at landing planes, not just launching them and hoping autopilot can take care of getting everyone safely to the gate.
* See The Neverending Story (1984), a film that probably contributed more to Gen X's emotional cynicism and generational trauma than anything.