As autonomous AI development accelerates, traditional engineering infrastructure faces unprecedented strain. At Anthropic, utilizing Claude caused daily code merges to surge from 2,500 to over 30,000 in just four months, turning developer tooling—specifically Git—into their primary bottleneck. Because Git relies on a decentralized architecture optimized for human-scale, offline workflows, maintaining hundreds of internal mirrors to support over 25 million daily checkouts became unsustainable. To solve this, a seven-person team built and migrated to Scion, a proprietary, centralized version control system, in five months with extensive code generation and optimization assistance from Claude.

Core Principles of Agentic Source Control

Scion was designed specifically around workloads where clients are autonomous agents and automated jobs rather than humans working offline:

• Always-Online Assumption: Discards decentralized, offline-first mechanisms in favor of direct server-side data fetching.
• Client-Server Asymmetry: The server handles diffs, merges, rebases, and commit creation. Clients remain thin, maintaining small caches and file trees without performing heavy graph computations.
• Comprehensive API-First Queries: Tools and CI query repository history, blame, and ancestry via API, eliminating the need to clone the monorepo locally.
• Lock-Free Horizontal Scalability: Distributed without single-point packfiles or global ref locks, reducing operational complexity.
• Optional, Sparse Working Copies: Jobs only pay for the specific files and directories they actually read via sparse, virtualized checkouts.

Technical Architecture: How Scion Works

Scion represents units of work as server-side write-ahead log entries called “changesets,” which track content-addressed modifications relative to their parent rather than storing complete tree snapshots. Key structural highlights include:

• Derived Data Generation: History, commit graph skip lists, and blame are computed recursively and cached as derived data, enabling server updates to scale with change size instead of total repository scale.
• Tiered Caching: A regional caching hierarchy—comprising a local node daemon, regional memory cache servers, and backend authoritative servers backed by Spanner and blob storage—yields over 99% hit rates.
• Infrastructure Efficiency: The architecture delivers a 200x reduction in CPU and memory footprint compared to the previous Git mirroring infrastructure, speeding up checkouts from minutes to seconds and returning 25% CPU capacity to CI systems.

Key Lessons for Future Engineering Teams

The experience revealed several emerging dynamics in agentic software engineering:

• Evolving Engineering Roles: Claude wrote the vast majority of the code and optimized complex algorithms, shifting the engineer's role from line-by-line syntax writing to high-level architecture, setting constraints, verifying intent, and preventing agent over-engineering.
• Autonomous Operations: Specialized AI “superfans” continuously monitor individual microservice health, spotting subtle anomalies far beyond static alerting systems.
• Rethinking Software Delivery Paradigms: Traditional practices like global merge queues become non-viable at tens of thousands of merges per day. Instead, teams must pivot to post-merge validation and pre-deploy testing where agents rapidly detect and repair merge conflicts asynchronously.

Mentoring question

As AI agents increasingly automate implementation details and micro-optimizations, how should engineering leaders reshape their team’s skills and code review processes to evaluate architectural intent, system boundaries, and operational constraints rather than syntactic correctness?

Source: https://www.anthropic.com/engineering/redesigning-source-control