Measure what survives, not what ships
Every signal devXOS produces, one slide at a time.
AI made shipping cheap. It did not make proof cheap. devXOS reads your Git and PR history and separates the code that lasted from the code that merely landed.
Velocity metrics were built for a world where humans wrote every line. This deck is what we built to replace them — each metric with its field name, its unit, and what it cannot tell you.
What devXOS will never do
Ten product principles guard the engineering analytics we build. These four matter most at the pitch.
devXOS never ranks or scores developers. Every metric describes repositories, teams, and dynamics — never who wrote what fastest.
No IDE plugin, no proprietary telemetry, no vendor lock-in. We read your Git history and PR data — that's it.
Every metric must hold up in plain language. If an engineering leader can't understand why a score exists, the score doesn't exist.
Engineering analytics can easily become surveillance. devXOS must be safe for teams to adopt. If a feature reduces trust, it doesn't ship.
The agent works inside a harness
Agentic engineering wraps AI in guides and sensors so speed does not outrun control. devXOS does not run inside that loop — it reads what the harness let through.
Which controls does this repo carry?
An inventory of the guides and sensors wrapped around the agents.
Presence and age of sixteen artifacts, split in two: guides that tell an agent what to do (CLAUDE.md, AGENTS.md, .cursorrules) and sensors that catch what it got wrong (types, lint, CI, tests).
It turns every other metric into a conditional question: what happens to durability with an agent guide versus without one? Presence and metadata only — never file content.
Is this codebase legible to an agent?
A structural score for how governable the code itself is.
Six structural sub-signals weighted into one number: type coverage, module boundaries, test presence, lint config, documentation density, and a legacy-ratio penalty.
Typed, bounded, tested code gives an agent the affordances it needs to work safely. This is a property of the code, never of the team — there is no author dimension in the computation.
Is the code staying changeable?
The best-covered dimension: what churns, what duplicates, what moves, and what gets rewritten weeks after it landed.
Does your code survive its first week?
The single most important number in devXOS.
Of every file your team touched, the fraction that was NOT modified again within the churn window. Files touched once count as stabilized.
A stabilization ratio near 1.0 means changes persist — real work. Near 0.0 means rework dressed up as delivery. This is signal vs noise in one number.
The cost of incomplete first tries
Count and weight of files that needed rapid re-editing.
Files modified 2+ times where a consecutive pair of edits falls inside the churn window. Plus the total lines touched across those re-edits.
Churn is the tax your team pays on shaky first implementations. Unlike velocity, it goes up when things go wrong — and it's visible per file.
Name the files that cost you
Top churning files with their chain. And the pairs that move together.
Top 10 churning files with their full chain (e.g., feat → fix → fix → refactor). Plus file couplings: pairs that co-occur in commits with high coupling rate.
Aggregate numbers tell you something is wrong. Churn detail tells you which file, what pattern, and what else changes with it. Now you can fix the root cause, not the symptom.
The 14-day canary
Code that gets re-edited inside two weeks.
Files that received new code and were modified again within 14 or 28 days. Segmented by origin and by AI tool, attributed to the INTRODUCING commit.
Fresh code that gets re-touched within two weeks usually means the first try missed. A 2-week rate trending up is the earliest quality alarm you can wire to a dashboard.
Copy-paste went 8× since AI
GitClear 2025. Measured. Now check yours.
Commits containing 5+ contiguous identical non-trivial lines across multiple files. Rate per commit, median block size, segmented by origin and by AI tool.
Copy-paste is the fast lane to entropy: the same bug, in five places, forever. A rising duplicate rate is the leading indicator of debt you haven't paid yet.
Real refactors look different
And we can tell the difference at the diff level.
Percentage of changed lines that were moved between files in the same commit. Refactoring ratio = moved / (moved + duplicated) — a code-health index.
Moved code dropped from 24% to 9.5% post-AI in industry data. When your refactoring ratio rises, the team is actually extracting and organizing — not just generating more.
Improving mature code, or churning this month's?
The age of the lines your team is rewriting.
Git-blame buckets the age of each line being modified: under 2 weeks, 2–4 weeks, 1–12 months, 1–2 years, 2+ years. Plus percentage revising new code vs mature code.
GitClear found 79% of revised lines in 2024 were less than a month old. If most of your team's effort is re-churning fresh code, you're not improving the codebase — you're spinning.
on code < 1 month old — industry 2024
The mix of how your team writes
Added, deleted, updated, moved, duplicated.
Lightweight five-bucket taxonomy of line operations per commit, built from diff content plus duplicate and move detectors. Overall plus per-origin breakdown.
A team dominated by `added` is growing fast; by `updated`, iterating; by `moved`, refactoring; by `duplicated`, accumulating debt. Shape of work, in one chart.
AI code has a shape. See it.
Focused, spread, bulk, or surgical — by origin.
Median files, lines per file, and directory spread per commit, grouped by origin. Each origin's typical shape emerges: deep, wide, thin, or broad.
AI-generated commits tend to be wide & shallow (spread) — scaffolding, boilerplate. Human commits lean surgical or focused. This pattern is measurable, not anecdotal.
Your repo has zones
Some stable. Some on fire. Name them.
Per-directory rollup (depth 2 by default) of files touched, stabilized, churn events, and stabilization ratio. Directories classified stable ≥ 0.80, volatile < 0.50.
"The backend is a mess" is a feeling. Stability map turns it into `src/payments/` at 0.41 vs `src/shared/` at 0.92. That's something you can fix, staff, or document.
Is correctness holding up?
The dimension the field calls the hardest. Every signal here is a proxy for correctness — never a guarantee.
The bluntest signal something broke
How often does your team un-ship what it shipped?
Commits matching revert patterns, as a fraction of total. Attribution credits the ORIGIN of the reverted code — not who wrote the revert.
Reverts are rare but unambiguous. Segmented by origin and AI tool, they answer: which tool's code gets rolled back? That comparison is hard to argue with.
Does AI code break faster?
Measured in hours — from first commit to rework.
Median time between consecutive modifications of the same file within the churn window. Attribution credits the ORIGINAL commit, not the fix.
Buckets: fast < 72h (probably obvious bugs), medium 72–168h (caught in review/prod), slow > 168h (subtle). Compare AI vs human fast-rework rates side by side.
One bad commit, three follow-ups
Blast radius of code that doesn't quite land.
A trigger commit followed by 1+ FIX commits on shared files within the churn window. Depth = number of follow-up fixes. Attribution credits the trigger's origin.
A 30% cascade rate means almost a third of your trigger commits break something. Segmented by AI tool, this tells you which tool's output carries the highest cleanup cost.
How much AI code survives the quarter
Git blame at HEAD. The ultimate survival test.
For each origin and each AI tool: lines introduced vs lines still present at HEAD. Survival rate. Median age of surviving lines in days.
Our internal benchmark found AI-attributed lines survive at 79% vs human 64% — on primed repos. Durability is the counter-intuitive headline: AI code may last longer when attributed properly.
Whose code attracts the bugs?
Fair share vs disproportionate share.
For each FIX commit's target files, credit the origin of the last non-fix commit. Compute code share vs fix share vs disproportionality (fix/code).
If AI wrote 30% of commits but attracts 50% of fixes, disproportionality = 1.67 — the clearest signal that AI-written code costs more to maintain than it first appears.
Does one regime break more than the other?
Three correctness signals, conditioned on who wrote the code.
Revert rate, hotfix rate and time-to-fix, each computed for AI-attributed and human-attributed commits, reported as a ratio. Above 1.0 means the AI regime behaves worse in this repo.
A raw number tells you nothing without a baseline; the comparison is the metric. It is withheld entirely below 30 commits per side, so a small sample can never read as a verdict.
Who wrote this, really?
Every dimension above can be cut by origin — but only if attribution holds. These metrics measure the measurement.
Human, AI-assisted, or bot
No guessing. We read co-author tags and author patterns.
Co-author matches Copilot, Claude, Cursor, Codeium, Tabnine, Amazon Q, Gemini, or Windsurf → AI_ASSISTED. Known bot names → BOT. Everything else → HUMAN.
Every single other metric in devXOS can be segmented by origin. This is the dimension that unlocks AI impact analysis — without surveys, without self-report.
How much AI work is already visible
The other half of this number is your attribution gap.
AI-attributed commits as a percentage of all non-bot commits. A proxy for how much of the actual AI usage is declared in the git metadata.
Compliance officers, AI governance leads, and skeptical CTOs all ask the same thing: how much AI is in our code? This is the answer you can defend in a meeting.
attributed to AI tools
The AI work nobody tagged
Human-classified commits with AI-shaped velocity patterns.
Flags HUMAN commits hitting 2+ of: 3 commits in 2h, 100+ LOC, < 30min since prev, 5+ files. We never call it AI — we surface the gap for review.
If ai_detection_coverage says 40% and attribution gap flags another 30% of human commits as suspect, your real AI footprint is double what your governance dashboard shows.
Feature, fix, refactor, config
Every commit classified. Deterministically. No ML.
Conventional Commit prefixes first, keywords second, file-type heuristic third. Every commit gets an intent: FEATURE, FIX, REFACTOR, CONFIG, or UNKNOWN.
"We're shipping fast" means nothing if 60% is FIX. Intent distribution turns a flat commit count into a picture of what the team is actually spending time on.
Does AI code pass review?
Single-pass PRs vs rounds of changes-requested.
Per origin and per AI tool: fraction of commits that landed via a PR; of those, fraction merged with zero CHANGES_REQUESTED; median review rounds.
Two different AI tools can produce code that reviews very differently. Acceptance rate quantifies that — it's the missing link between "AI productivity" claims and peer-reviewed outcomes.
The full journey, per origin
Committed → In PR → Stabilized → Still alive.
Four-stage delivery funnel computed per origin, with conversion rates between each step. Composes origin distribution, acceptance, stabilization, and durability.
AI might crush commits and pass review — and still drop off at stabilization. The funnel reveals exactly where each origin wins and where it leaks. One chart, full story.
The day AI changed your metrics
Detected automatically. Pre vs post, side by side.
Finds the inflection point where AI-attributed commits began appearing. Splits history into pre-adoption and post-adoption, each with a full ReportMetrics snapshot.
Before-and-after proof. Stabilization went from 0.71 to 0.84 since the Copilot rollout? That's a number you can put on a slide. Reversed? That's a number you need to look at fast.
How does work actually move?
Where time goes between the first commit and the merge — and whether anyone still looks at the code on the way.
Quantify review friction
Before it becomes a complaint in the retro.
Median time-to-merge, median PR size (files and lines), median review rounds, and single-pass rate — the fraction of PRs merged without a CHANGES_REQUESTED review.
Single-pass rate is the PR metric that correlates most with team satisfaction. Combined with time-to-merge, it tells you whether review is a gate or a bottleneck.
Lifecycle time, split into active and wait
Throughput rose. Lead time didn't. This is why.
Decomposes every merged PR into four phases — coding, awaiting first review, in review, awaiting merge — and classifies each as active or wait. AI compresses coding time; review queues stay flat. The ratio tells you where the time really goes.
Without this, AI looks like a win on commits/week while lead time flatlines. With it, you see exactly which queue is absorbing the speedup — and whether the gain is reaching production.
How much work is in flight, not just shipping
AI expands WIP faster than it expands throughput.
Per ISO week: how many PRs were simultaneously in flight, segmented by intent, alongside the count of distinct commit authors. A system-level WIP signal — never per person.
Healthy throughput with rising WIP and flat author concurrency is congestion in disguise. This is the metric that explains why a team feels busier when the dashboards look fine.
The PRs nobody is looking at
A snapshot of open work that never moved.
Counts non-draft, non-bot PRs still open right now. Reports median age, stale share (no review or commit in 14+ days), and abandonment share (60+ days). Broken down by intent and code origin.
Throughput metrics describe work that moved. This describes work that stopped. When AI generation outruns review capacity, this is the metric that catches it first — and the AI-vs-human gap shows up clearly.
Is anyone still reading the code?
The fraction of merged PRs a human actually reviewed.
Merged PRs carrying at least one human review, over all merged PRs — with the same fraction cut by change intent and by the origin of the PR's commits.
When AI writes more of the code and review is the last human checkpoint, this is the coverage of that checkpoint. A rate falling while AI share rises is the pattern worth catching early.
Every week tells a story
Weekly breakdown + four pattern detectors.
ISO-week rollup: commits, LOC, intent mix, origin mix, stabilization, churn, PRs merged. Patterns auto-detected: burst_then_fix, quiet_period, ai_ramp, intent_shift.
When a metric jumped, you need to know why. The timeline + patterns layer gives you an annotated story — not just numbers, but the moments that made them.
Speed means nothing if durability drops
Commits/week, lines/week, and the correlation with quality.
14-day windows of commits/week and lines/week. Trend classified accelerating, stable, decelerating. Correlated with per-window stabilization — the durability connection.
Accelerating with durability steady = real progress. Accelerating while stabilization drops = you are shipping noise faster. Velocity alone lies. Velocity + durability tells the truth.
Where the code meets reality
The engine sees the code — fast, visible before production. DORA sees production — slow, visible after the damage. These three connect them.
Connect Datadog, see DORA crossed with code origin
Optional integration. Vendor-agnostic by default.
When a team opts in, devXOS pulls Change Failure Rate, MTTR, lead time, deploy frequency and rollback rate from Datadog's DORA events, then joins them by commit SHA against local origin classification. Treats pending evaluations as their own bucket — no fake CFR while Datadog hasn't decided yet.
The closing answer to the question this deck opens with: does AI-assisted code survive in production? With Datadog connected, CFR and rollback rate by origin make the answer numerical — for the teams that choose to provide it.
Does the code signal lead the outage?
Four curated pairs, tested for lead-lag — not a blind scan.
Weekly engine series correlated against weekly DORA series at non-negative lags: cascade rate against change failure rate, stabilization by origin against lead time, new-code churn against deploy frequency.
If a code signal reliably leads a production failure, you get warning weeks earlier. The guardrails matter more than the finding: Šidák correction for the lag scan, tail censoring of look-ahead series, and per-week minimum denominators.
What did the tokens actually buy?
Agent spend, crossed with the survival of the code it produced.
Anonymous token counts priced per model, joined per repository against the same window's stabilization, survival by origin, revert rate and cycle time.
Cost per generated line is a vanity number; cost per feature that lasted is a decision. Models we cannot defensibly price are reported as null and listed — the total under-reports transparently rather than guessing.
Run it on a repo you care about
One command. The engine runs locally — the platform is optional.
Every metric in this deck is a documented field with a name, a unit, and a stated null case. Nothing here is a black box, and nothing here scores a person.
Metrics are hypotheses, not truths. Point the CLI at your repository, read the report, and disagree with it — that is the intended use.