VP & Client Engagement Manager at Genpact, leads enterprise technology delivery for Tier 1 banks across the UK and Europe.
For the past two years, software engineering has been flooded with claims about AI-driven productivity. One enterprise may report that AI reduces development time by a third, while another may say it doubles output. At the same time, technology vendors often promote even more ambitious results. Each organization presents a different figure, yet the numbers are typically described with the same level of confidence, making it difficult to determine what the gains actually represent or how they were measured.
Every CIO is under pressure to accelerate delivery, and every CFO is looking for lower technology costs. However, the wide variation in these claims should be an early warning that the industry may be measuring engineering productivity incorrectly. Faster code generation, for example, does not necessarily translate into faster delivery if developers spend additional time reviewing, testing, debugging, integrating, or maintaining AI-generated code.
One of the most rigorous studies conducted to date produced a notably different result. In 2025, the AI safety research nonprofit METR conducted a randomized controlled trial involving 16 experienced open-source developers who were working on mature codebases they already knew well. Before using AI, participants expected the tools to make them approximately 25% faster, and they continued to believe that AI had improved their performance after completing the tasks. Objective measurement, however, showed that they were about 19% slower.
A subsequent 2026 METR study suggested that newer AI tools were likely producing larger productivity gains. However, selection effects made the size of that improvement difficult to measure reliably, so the findings did not provide a definitive estimate of the overall benefit.
The difference between perceived and measured productivity is the most important insight. Engineers may feel more productive because AI reduces certain forms of effort, speeds up individual tasks, or makes it easier to generate possible solutions. Yet those benefits may not appear in end-to-end delivery metrics, particularly when the work involves complex systems, established codebases, coordination, quality assurance, and long-term maintenance. Any serious assessment of AI productivity therefore needs to examine the complete development process rather than relying solely on self-reported impressions or isolated claims about output. The important question is not which productivity number is right. It is what each number actually captures, how productivity is defined and measured, and which baseline provides the comparison. Two organizations can both report accurate results while reaching different conclusions because they are evaluating different parts of the work, using different metrics or comparing performance with different starting points.
Software engineering is a system, not merely a coding task.
The industry has gradually, and often unintentionally, narrowed the definition of software engineering until it is treated as synonymous with writing code.
In practice, software engineering is the broader discipline of translating business intent into secure, dependable and compliant software that operates in production. Coding is only one component of that process. The full life cycle also includes requirements engineering, architecture, testing, security, governance, deployment, operations and continuous improvement. Each activity affects the quality, speed, cost and business value of the final result, and weaknesses in any one area can limit the performance of the entire system.
AI has made substantial progress in speeding up code generation, but making one activity faster does not automatically improve the engineering system as a whole. If developers produce code more quickly while testing, review, deployment or operations remain unchanged, the limiting factor simply shifts to another stage. More code may then accumulate in queues, require additional rework or increase the burden on the teams responsible for validating and running it.
Google’s “2025 DORA State of AI-Assisted Software Development” report found that AI adoption was associated with greater software delivery throughput but lower delivery stability. Without similarly mature testing practices, code review, release engineering and operational discipline, organizations may end up accelerating the production of defects, failed releases or operational incidents instead of accelerating meaningful business outcomes.
Improving a single stage is not the same as improving the system. Sustainable productivity gains require the connected engineering process to improve together.
Many organizations still assess AI through narrow, local indicators, including coding speed, the volume of code generated, AI suggestion acceptance rates and the output of individual developers. These measures can provide useful signals about tool adoption and activity, but they do not, by themselves, measure software engineering productivity. They say little about whether teams are delivering valuable capabilities, reducing risk or moving reliable changes into production faster.
Enterprise software delivery operates as a complex system with interdependent stages. Improving a single step rarely improves the performance of the system as a whole. Faster implementation will have limited business impact if requirements take weeks to clarify, testing remains largely manual, development and production environments are difficult to provision or governance processes delay releases. A team may generate code more quickly while the organization’s overall delivery flow remains unchanged.
This is a pattern many banking technology leaders report. AI increases the productivity of individual developers, and dashboards show encouraging gains, yet the release cadence changes little because the primary constraints remain elsewhere. Change approvals, regression testing, compliance reviews, environment provisioning and operational readiness can continue to determine how quickly a completed feature reaches customers. Unless those constraints are addressed, local improvements in development activity may accumulate without producing a corresponding improvement in end-to-end delivery.
Organizations should therefore measure software engineering by the speed, quality and reliability with which it delivers business outcomes. More meaningful measures include the time required to move an approved idea into production, the stability of releases, the frequency and size of successful changes, the effort needed to resolve defects and the value delivered to customers and the business—not simply how quickly code is produced.
AI’s greatest opportunities extend beyond coding. It can also help clarify requirements, generate and maintain tests, identify risks, streamline documentation, support compliance evidence, improve environment management and strengthen operational readiness. The largest gains will come when AI is applied across the delivery system rather than treated only as a tool for accelerating implementation.
Some of AI’s most valuable applications arise before any code is written and in the less visible work that surrounds software delivery. These applications include analyzing and reconciling business requirements, exploring architectural alternatives, evaluating regulatory and compliance implications, estimating delivery effort, tracing dependencies across applications and services, maintaining documentation, generating automated tests, and investigating incidents in production. By handling and accelerating this work, AI can help teams clarify decisions earlier, expose risks sooner and reduce the manual effort required to keep systems reliable.
These activities often offer greater practical value than code generation alone. They can improve delivery speed, consistency and quality while identifying defects and misunderstandings earlier, when they are substantially less expensive to correct. For example, AI can help detect conflicting requirements before they become design problems, identify missing test coverage before a release and summarize incident evidence so engineers can focus on diagnosis and remediation.
A DORA report observed that the benefits of AI increase with the maturity of the engineering system around it. Teams with comprehensive automated testing, disciplined development practices, dependable deployment processes and rapid feedback loops are better positioned to convert AI-assisted throughput into improved business outcomes. Teams that lack those foundations may simply produce more code without improving customer value—and may generate more defects, operational risk and instability in the process.
AI amplifies engineering discipline; it does not replace it.
Productivity does not automatically mean fewer developers.
One of the boldest predictions about AI is that organizations will eventually need dramatically fewer software engineers. Although some organizations may have identified AI-assisted development as one factor in workforce reductions, evidence that coding assistants alone will support large-scale headcount reductions remains limited. In many cases, the time saved on implementation is redirected toward higher-value work, including improving reliability, strengthening security, refining products, modernizing legacy systems and addressing the growing complexity of software estates.
Coding represents only one part of the broader software engineering discipline. Unless AI can also reduce bottlenecks in requirements definition, architecture, testing, security, governance, deployment and release management, it is unrealistic to expect coding assistants by themselves to eliminate large numbers of engineering positions. They may accelerate implementation, but the rest of the life cycle can continue to require substantial human judgment, coordination and oversight.
Jevons Paradox suggests that making a resource cheaper or more efficient often increases demand for it instead of reducing overall consumption. In software engineering, productivity improvements have historically allowed organizations to develop more products and features, address technical debt, strengthen security, modernize aging systems and pursue initiatives that were previously too expensive to justify. As a result, efficiency gains have often expanded the range of work organizations undertake rather than simply reducing the size of engineering teams.
Whether AI functions mainly as a cost-reduction mechanism or as a way to expand organizational capacity will depend largely on leadership decisions and business strategy. The technology can support either outcome. Leaders may use higher productivity to reduce staffing needs, or they may reinvest the additional capacity in new products, faster delivery, improved resilience and broader transformation efforts.
The software engineer’s role is changing.
As AI becomes integrated throughout the software engineering life cycle, developers will spend less time producing routine code and more time directing, evaluating and governing how software is created. Their responsibilities will increasingly include system design, architecture, AI orchestration, security, quality engineering and the creation of guardrails that ensure AI-generated software is secure, compliant, reliable and consistent with business objectives. They will also need to validate AI outputs, manage trade-offs, maintain accountability for technical decisions and ensure that automated development processes meet organizational and regulatory requirements.
Software engineering is increasingly shifting away from the narrow task of writing code and toward the broader responsibility of governing intelligent systems that generate software in a responsible, secure and effective manner.
Measure what actually matters.
Which productivity metric is the right one? The answer is probably all of them—and, in isolation, none of them.
Each metric captures a localized view of a different engineering system, such as developer activity, delivery speed, deployment frequency, defect rates or business impact. Organizations that expect coding assistants alone to transform software delivery are likely to be disappointed. Meaningful gains become more plausible when AI is integrated with modern engineering practices, automated platforms, DevSecOps, effective governance and the organizational changes needed to adopt new ways of working. In that environment, AI can support faster development while also improving consistency, security, quality and operational reliability.
The deeper transformation is that software engineering is becoming a discipline in which humans design systems, establish controls for AI, assure quality and apply technology to solve business problems. The question is no longer “How much faster can developers write code?” It is “How much faster—and more reliably—can our engineering system deliver meaningful business outcomes?” Until organizations answer that question using system-level metrics rather than isolated coding benchmarks, every headline productivity figure—however impressive—will remain accurate for some teams while providing an incomplete picture for everyone else.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
Discover more from Headline Hub
Subscribe to get the latest posts sent to your email.
