When Coding Is Not the Bottleneck: Three Levels of Software Autonomy

Toward autonomous software development: when coding is no longer the bottleneck, how will software engineering be reshaped?

Imagine this: you no longer write code line by line, no longer debug late into the night, and no longer spend weeks agonizing over refactoring legacy modules. You simply describe your intent in natural language—autonomous AI agents analyze the codebase, design a solution, write the implementation, generate tests, and trigger deployment. All of this can happen in just a few minutes.

This is not a sci-fi scenario ten years away. Today, sixteen parallel Claude agents built a working C compiler for less than $20,000. They can reason across a codebase, write tests, spot vulnerabilities, and coordinate multi-stage tasks.

However, when the benchmark shifts from “completing an isolated task” to “continuously evolving a system,” these agents’ performance falls off a cliff: they can add features, but struggle to preserve correctness and architectural consistency across successive changes.

This points to a key fact: AI is already powerful enough to take on meaningful “code autonomy,” but it still has a long way to go before “full autonomy” can be safe, reliable, and accountable.

This article is adapted from the core insights of the position paper “Toward Autonomous Software Development” authored by nearly twenty leading researchers from UC Berkeley and other institutions. Original link: https://rdi.berkeley.edu/blog/auto-software-dev/

For most of software engineering’s history, every organizing principle revolved around one scarce resource: people who can write and review code. Programming languages, frameworks, testing systems, and organizational processes—these best practices exist essentially to help humans translate requirements into reliable software while coping with limited time, attention, and cognitive bandwidth.

With AI, that constraint is beginning to loosen.

State-of-the-art coding agents can already reason autonomously across an entire codebase, write and execute tests, identify vulnerabilities, and coordinate multi-stage work. The most compelling example: sixteen parallel Claude agents jointly built a usable C compiler for under $20,000. Yet on benchmarks that specifically measure “continuous evolution” rather than “isolated tasks,” frontier agents still degrade sharply: they can add features, but struggle to maintain correctness and architectural consistency change after change.

These two realities must be held together: coding agents are already strong enough to take on meaningful “ownership,” but they are nowhere near stable enough for us to treat “autonomy” as a single, undifferentiated capability.

So the core question is no longer “How can AI help developers complete a given task?” but “Across every stage of the software development lifecycle, how can responsibility be transferred from humans to AI—and what preconditions must be met for that transfer to be safe, reliable, and accountable?”

The early lesson from autonomous driving research is that a field cannot discuss safety, capability, and accountability clearly without a shared vocabulary. The SAE levels of driving automation distinguish “driver assistance,” “conditional automation,” and “full driving automation,” and clarify where responsibility lies at each level.

Software development currently has no such framework. The term “autonomous coding agent” can mean anything from a tool that suggests a few lines of code to an agent that files PRs, a system that tests and deploys itself, or an agent that decides what should be built. These systems are fundamentally different, and their failure modes are just as divergent.

The authors borrow from the SAE automation levels and propose three levels of software autonomy, defined by which stages of the software development lifecycle have shifted from full human responsibility to full AI control:

1. Level 1: Code Autonomy

AI is fully responsible for system design and code implementation, with no need for line-by-line human review. Agents can independently produce complete merge requests, accompanied by design rationale, full code, project documentation, and everything else.

Humans retain core decision rights: they define requirements, review AI output at the merge-request level, oversee testing and security audits, and hold the final authority over deployment. Today’s mainstream AI-assisted coding tools are only precursors to this level.

2. Level 2: Pipeline Autonomy

AI takes over the entire software development pipeline—from requirements design and code implementation to testing, security auditing, and production deployment—while humans no longer participate in writing or reviewing code.

Humans only need to provide high-level business requirements and accept the final delivered software. This level represents a qualitative leap for the industry. It depends on two critical preconditions: human requirements can be completely and precisely translated into standardized technical specifications, and the automated verification system is reliable enough that no human checks of intermediate artifacts are needed. Today, neither precondition holds at scale in real business scenarios.

3. Level 3: Demand Autonomy

AI not only completes the full software development, testing, and deployment workflow, but also decides on its own what to build. It can proactively discover and identify potential development needs from multiple dimensions: system runtime data, user behavior traces, security advisories, dependency updates, and dynamic system changes.

In the normal development loop, humans are no longer part of concrete development decisions, but all AI actions must remain within the core business mission set by humans. The biggest challenge at this level: how to ensure that AI-generated development requirements stay aligned with the original business goals and do not silently alter the system’s positioning and core value.

Systems in different industries and scenarios will progress through these levels at different speeds: safety-critical, high-reliability domains will remain at Level 1 or below for a long time, while internal enterprise tools and one-off lightweight applications will adopt Level 2 autonomy sooner. This taxonomy makes it possible to define AI capability, make adoption decisions, and assign responsibilities in a clearer, more actionable, and auditable way.

It is important to note: most AI tools available today only produce functional output without any official correctness guarantee. They do not meet the stability, consistency, and reusability standards required by any autonomy level. The industry has not truly entered Level 1 or Level 2 autonomous development.

The level answers one major question—which stages of the software development lifecycle does AI own?—but it is not enough to fully characterize a system’s autonomy. Two systems at the same level can look very different along three other orthogonal dimensions:

1. Specification Granularity

A bug report with a reproduction test dramatically narrows the agent’s interpretive space, while a vague instruction like “add multi-tenant support” forces the agent to infer scope, architecture, trade-offs, and success criteria on its own. The weaker the specification, the more a nominally lower-level system is forced to confront problems that belong to higher levels.

2. Temporal Autonomy

An agent may work at the scale of a ticket, an iteration, a release, or continuously for months or years. Long-horizon operation brings problems that one-shot evaluations cannot reveal: memory, provenance, regression, and architectural consistency.

3. Oversight Mode

Human involvement can take the form of co-writing specifications, action-level approvals, PR review, policy guardrails, pure monitoring, or automatic rollback. The right mode depends heavily on domain risk and reversibility.

Across all three levels, the same challenge recurs in different forms: as direct human control recedes, how do we preserve and faithfully execute human intent?

At Level 1, humans can correct deviations and restore original intent through review.

At Level 2, human intent must be fully encoded into standardized specifications, with no human checking the entire pipeline as a safety net.

At Level 3, AI must proactively organize, maintain, and update requirement specifications over long-term system evolution, continuously staying aligned with the core mission set by humans.

The seemingly isolated failures in the industry are, at bottom, intent-implementation deviations: specification drift (the code gradually moves away from the original human intent), reward hacking (AI satisfies quantitative metrics while violating core business goals), multi-agent cognitive conflicts (different AIs interpret the same requirement inconsistently and fail to surface the contradiction), and test-code collusion distortion (AI writes code and tests together; they are internally consistent with each other but collectively diverge from real business needs).

A critical warning: if the same AI agent both develops code and writes the test cases, passing tests only proves that the code and tests match each other’s logic. It cannot prove that the code meets the actual business requirements.

This also means that higher-level autonomous development fundamentally resets the target of software security validation. Checking the final software artifact is no longer enough; we must also audit the AI agents that produce the software—covering their requirement specifications, capability boundaries, memory data, decision provenance, collaboration mechanisms, and execution traces.

The most urgent risk today is not immature fully autonomous AI systems; it is the widespread practice of skipping levels across enterprises and teams.

Teams claim to follow Level 1 autonomy—keeping human review and release approval—but in practice merge AI-generated code directly without meaningful verification. Some blindly apply Level 2 end-to-end autonomous development without the validation system, governance rules, and accountability mechanisms that Level 2 requires. Still others let AI initiate development changes based on system data and external information, without any corresponding risk-control chain.

The paper therefore proposes a grade-based admission mechanism: an AI system must fully overcome the technical and governance gaps at its current level before being promoted to a higher autonomy level. The higher the autonomy level, the longer the operating horizon, and the greater the business risk, the stricter the verification threshold must be.

When coding is no longer the development bottleneck, software engineering will not simply speed up existing workflows. It will undergo a comprehensive, deep structural reconstruction. This transformation spans three dimensions—the form of software artifacts, development processes, and the industry-wide ecosystem—and manifests as six core trends.

1. Requirement specifications become the core development artifact

As humans write and modify code far less often, requirement specifications will become the central interface between human business intent and AI machine execution. Specifications will no longer be limited to basic functional requirements; they will also cover security constraints, maintainability requirements, architectural invariants, product design rules, testing strategy, and more.

In the future, the cost of manually maintaining complete specifications will keep rising. Eventually the industry will develop a specification-extraction mechanism: humans collaborate with AI through dialogue, examples, and corrective feedback, and AI continuously distills fragmented interactions into structured, iterative, actionable standards. In the long run, requirement specifications will be bound one-to-one with software systems, and the entire codebase could be reconstructed solely from the specification.

2. Abstraction boundaries shift from hard constraints to flexible rules

Today’s layered abstractions—functions, modules, libraries, interfaces, frameworks—exist largely to accommodate human cognitive limits. AI agents can reason globally across many layers of the codebase. The existing abstraction hierarchy still has value for organizing and structuring a project, but it is no longer an unbreakable constraint.

AI can refactor, merge, split, inline, or bypass existing abstractions on its own in service of overall system optimization. The role of abstraction design changes from “a hard framework developers must obey” to “a flexible norm AI can adapt.” Future software sharing will also shift from traditional code reuse to reusing protocol specifications, behavioral standards, interface contracts, and reference implementations—giving AI more room to optimize while preserving interoperability.

3. Software evolves from static artifacts to dynamic evolving systems

Autonomous development fundamentally changes the form and iteration model of software: traditional versioned, statically frozen artifacts will gradually be replaced by dynamic evolving systems. The new software has four core characteristics: neuro-symbolic fusion (combining deterministic safety guarantees with flexible adaptation), autonomous iteration (continuously optimizing based on runtime data), interactive modifiability (blurring the line between using and re-developing software), and generation on demand (temporarily generating a program for a single task and discarding it when reuse costs more than regeneration). Mature systems typically blend multiple characteristics: core safety modules stay deterministic and stable, edge businesses adapt flexibly, and online systems keep iterating end to end.

4. Validation shifts from checking the artifact to checking the process and the agents

Even when following the same requirement specification, AI-generated code can differ enormously. Traditional validation methods that rely on fixed code patterns and stable artifacts will stop working. More importantly, a single AI can produce code, tests, documentation, and design rationale in one pass, making it easy for systematic correlated failures to emerge across the entire chain.

Relying on independent verification AI cannot fully eliminate the risk. The verification agent must have its own independent evaluation goals, trustworthy validation mechanisms, and standardized conflict-resolution procedures. The core of software security validation moves from “verifying the final software artifact” to “verifying the capabilities of AI agents and the end-to-end development system.”

5. Multi-agent collaboration breaks the human organizational pattern

Most current multi-AI collaboration systems copy human team management: human delegation, division of labor, human review, and natural-language communication. This model matches the limits of human cognition, communication, and management—not the optimal logic for software development.

AI agents have natural advantages: they can split and merge execution state, share complete project context, and coordinate efficiently through structured protocols, enabling massive parallel collaboration at the scale of tens of thousands or millions of agents. In the future, native AI collaboration models fully adapted to machine capabilities will emerge—along with new collaboration failures, security risks, and accountability challenges.

6. The entire software engineering ecosystem is upgraded

Once AI handles the vast majority of coding, companies can build and maintain more complex software systems with leaner engineering teams. The core value of human developers will shift entirely toward high-level activities: product definition, architecture design, requirement specification, outcome evaluation, security control, governance and compliance, and incident response.

At the same time, a brand-new supporting ecosystem will emerge: independent audit firms, certification platforms, trustworthiness assessment services, and compliance governance services will become core infrastructure as important as today’s testing platforms and cloud services. Software engineering education will also change—moving away from hand-coding skills and toward requirement specifications, formal verification, security governance, and system-level judgment.

Based on the six structural transformation trends, we can distill ten actionable and testable predictions for the industry. Different domains will move at different speeds, but together they will shape the trajectory of software engineering over the next decade.

1. Requirement specifications become the “genome” of software

The persistent core of a software system will no longer be code, but the continuously evolving requirement specification—covering business requirements, constraints, architecture decisions, testing strategy, design intent, and other essential information. Code will be just one implementation form of the specification, no longer the only authoritative artifact.

2. Code refactoring is gradually replaced by code regeneration

For software that can be fully reconstructed from a specification, regenerating clean code from the spec will cost far less than iteratively fixing historical cruft and optimizing legacy code year after year. This pattern will not become universal—many systems without complete documentation and with deep legacy adaptations will still need traditional iterative maintenance—but it will fundamentally change the operational cost structure of most applications.

3. Software reuse shifts from code reuse to protocol/spec reuse

Once custom code becomes cheap to produce, the core value of libraries, frameworks, and SaaS platforms will shift from fixed code dependencies to behavioral standards, interface contracts, and reference implementation templates. AI can adapt, optimize, and rewrite the underlying code autonomously while keeping external functionality completely unchanged.

4. Dynamically adaptive software becomes mainstream

Software that combines deterministic safety guarantees, flexible adaptation, continuous iteration, interactive modification, and on-demand generation will become widespread. Most online systems will no longer have a single frozen canonical codebase; instead, they will keep evolving underneath while preserving stable external functionality.

5. Requirement-spec extraction becomes a core engineering skill

Developers will no longer need to write a complete requirements document up front. They can convey business intent through dialogue, examples, outcome review, and corrective feedback. AI will continuously distill these fragmented interactions into structured, maintainable, iterative requirement specifications that stay in sync with the software system.

6. The focus of validation shifts from code review to agent verification

Code review still has auxiliary value, but the center of gravity of trustworthy validation will move decisively. The industry will audit AI agents’ specification configuration, permission scope, memory data, execution traces, decision provenance, evaluation systems, and governance rules. The trustworthiness of the development process will matter as much as the trustworthiness of the final artifact.

7. Native AI collaboration replaces human-style workflows

Current multi-agent collaboration systems all copy human team management logic. Future AI teams will develop their own communication, task delegation, consensus-building, and conflict-resolution mechanisms suited to machine characteristics, achieving collaboration at a scale, speed, and precision that human teams cannot match.

8. Project context becomes core intangible assets

When code can be regenerated cheaply, a company’s core asset is no longer any particular version of the code, but the complete project context accumulated over time: requirement specifications, architecture decisions, development conventions, constraints, evaluation history, and iteration feedback. This portable, platform-agnostic information can support multiple rounds of code regeneration and smooth migration across models and tools.

9. Engineering teams become leaner and governance becomes finer

Heavy coding teams will shrink dramatically, while the value of architecture design, specification, security verification, compliance governance, and incident response roles keeps rising. Independent audit, certification, and trustworthiness assessment organizations will be deeply embedded across the software lifecycle.

10. Code supply becomes abundant; trustworthiness becomes scarce

Autonomous development will dramatically lower the barrier and cost of custom software, broadening who can build software and where. But the flood of AI-generated software will make trustworthiness the scarcest core resource. Provenance of outputs, behavioral verification, agent verification, and trusted governance systems will become the most important competitive capabilities in software engineering.

The level framework and transformation predictions in this paper are not a declaration that the era of fully autonomous software development has arrived. Rather, they clarify the technical capabilities and safety assurance systems that must be in place before higher-level autonomy can be deployed. The current research gaps and priorities are as follows:

1. Automatic synthesis of requirement specifications and behavioral understanding

The industry urgently needs technical approaches that work across the full codebase: automatic reasoning, generation, and iterative maintenance of machine-checkable specifications. At the same time, we need new benchmark infrastructure to detect specification-code consistency drift over multiple iterations. Today, almost no such evaluation infrastructure exists.

2. Highly trustworthy automated verification

In autonomous development, AI must have reliable self-verification capabilities to ensure its output fully matches the original requirement specification. Formal verification is the best technical path, but current techniques have clear limitations in coverage and scalability; they cannot support verification of large, continuously evolving software systems.

3. Security, governance, and accountability systems

Autonomous software development creates entirely new security risks, covering new attack surfaces such as agent memory, tool invocation, external data, multi-machine collaboration, and autonomous execution. We urgently need security architectures, permission systems, monitoring/provenance, and recovery mechanisms adapted to these risks.

Beyond technical security, the industry needs a complete underlying logic for trustworthy governance: clear authorization, oversight, audit, and accountability mechanisms for AI agents, plus trustworthiness criteria, certification processes, and responsibility allocation rules—so that autonomous software can be safely deployed in critical infrastructure.

4. Agent-oriented system and language design

Today’s programming languages, development environments, storage systems, provenance models, and collaboration protocols are all designed around human development scenarios and cannot serve AI-autonomous development. The industry needs to rebuild the technical foundation for an “AI-first, human-supervised” world—optimizing the full development chain while preserving human control, explainability, and the ability to intervene.

5. Human-AI collaboration and a new economic model

Once AI leads development, the industry must solve core human-AI collaboration problems: how to efficiently transfer business intent, project knowledge, and requirement constraints; and how humans can supervise lightly and steer AI precisely without becoming the bottleneck.

At the same time, traditional engineering productivity metrics stop being useful. We need a new evaluation framework that accounts for AI inference cost, human oversight cost, operational overhead, technical debt, system stability, and runtime risk—an economic model suited to autonomous development.

Top research priority: the most urgent task is to build a specification-synthesis benchmark that works across full codebases, plus a long-horizon mechanism for evaluating spec-code divergence. Without this foundation, any claim of Level 2 or Level 3 autonomous capability cannot be verified or deployed.

When coding is no longer the bottleneck of software development, a series of core questions without standard answers await exploration by the entire industry:

Once AI takes over all coding work, what are the high-value jobs that developers should do?

Is shipping software without human code review a mature engineering paradigm, or a reckless experiment whose risks are not yet understood?

Does AI actually lower the barrier to software development and democratize technology, or does it simply shift the industry’s core moat to requirement specifications, system evaluation, architecture design, and governance?

When software built autonomously by AI causes failures or losses, how should ultimate liability be assigned?

Even if AI is capable of autonomous decision-making, which core decisions must remain with humans permanently?

The future of software development will not go to the extreme of “humans fully exiting the scene.” Long-term human-AI collaboration is the most stable form, and the pace of AI adoption will be differentiated and gradual across industries. But the direction is clear: software engineering research must move away from optimizing single coding tasks and toward the long-term evolution of autonomous capabilities.