01 / 44
DEVXOS · METRICS DECK

Measure what survives, not what ships

Every signal devXOS produces, one slide at a time.

What it measures

AI made shipping cheap. It did not make proof cheap. devXOS reads your Git and PR history and separates the code that lasted from the code that merely landed.

Why it matters

Velocity metrics were built for a world where humans wrote every line. This deck is what we built to replace them — each metric with its field name, its unit, and what it cannot tell you.

PRINCIPLES · BUILT IN

What devXOS will never do

Ten product principles guard the engineering analytics we build. These four matter most at the pitch.

Systems, never individuals

devXOS never ranks or scores developers. Every metric describes repositories, teams, and dynamics — never who wrote what fastest.

Vendor-agnostic intelligence

No IDE plugin, no proprietary telemetry, no vendor lock-in. We read your Git history and PR data — that's it.

Explainable, or it doesn't ship

Every metric must hold up in plain language. If an engineering leader can't understand why a score exists, the score doesn't exist.

Trust is a product feature

Engineering analytics can easily become surveillance. devXOS must be safe for teams to adopt. If a feature reduces trust, it doesn't ship.

PART 1 · ARCHITECTURE

The agent works inside a harness

Agentic engineering wraps AI in guides and sensors so speed does not outrun control. devXOS does not run inside that loop — it reads what the harness let through.

ARCHITECTURE · CONTROLS

Which controls does this repo carry?

An inventory of the guides and sensors wrapped around the agents.

What it measures

Presence and age of sixteen artifacts, split in two: guides that tell an agent what to do (CLAUDE.md, AGENTS.md, .cursorrules) and sensors that catch what it got wrong (types, lint, CI, tests).

Why it matters

It turns every other metric into a conditional question: what happens to durability with an agent guide versus without one? Presence and metadata only — never file content.

harness_profile.has_agent_guide · has_sensorsbool · bool
example true · true
ARCHITECTURE · READINESS

Is this codebase legible to an agent?

A structural score for how governable the code itself is.

What it measures

Six structural sub-signals weighted into one number: type coverage, module boundaries, test presence, lint config, documentation density, and a legacy-ratio penalty.

Why it matters

Typed, bounded, tested code gives an agent the affordances it needs to work safely. This is a property of the code, never of the team — there is no author dimension in the computation.

harnessability.score0.71
high ≥ 0.66 · moderate ≥ 0.40 · low below
PART 2 · MAINTAINABILITY

Is the code staying changeable?

The best-covered dimension: what churns, what duplicates, what moves, and what gets rewritten weeks after it landed.

MAINTAINABILITY · STABILIZATION

Does your code survive its first week?

The single most important number in devXOS.

What it measures

Of every file your team touched, the fraction that was NOT modified again within the churn window. Files touched once count as stabilized.

Why it matters

A stabilization ratio near 1.0 means changes persist — real work. Near 0.0 means rework dressed up as delivery. This is signal vs noise in one number.

stabilization_ratiofloat 0.0–1.0
example 0.83
MAINTAINABILITY · CHURN

The cost of incomplete first tries

Count and weight of files that needed rapid re-editing.

What it measures

Files modified 2+ times where a consecutive pair of edits falls inside the churn window. Plus the total lines touched across those re-edits.

Why it matters

Churn is the tax your team pays on shaky first implementations. Unlike velocity, it goes up when things go wrong — and it's visible per file.

churn_events · churn_lines_affectedint · int
example 42 · 3 120
MAINTAINABILITY · CHURN DETAIL

Name the files that cost you

Top churning files with their chain. And the pairs that move together.

What it measures

Top 10 churning files with their full chain (e.g., feat → fix → fix → refactor). Plus file couplings: pairs that co-occur in commits with high coupling rate.

Why it matters

Aggregate numbers tell you something is wrong. Churn detail tells you which file, what pattern, and what else changes with it. Now you can fix the root cause, not the symptom.

churn_top_files[0].chainstring
example "feat→fix→fix→fix"
MAINTAINABILITY · NEW-CODE CHURN

The 14-day canary

Code that gets re-edited inside two weeks.

What it measures

Files that received new code and were modified again within 14 or 28 days. Segmented by origin and by AI tool, attributed to the INTRODUCING commit.

Why it matters

Fresh code that gets re-touched within two weeks usually means the first try missed. A 2-week rate trending up is the earliest quality alarm you can wire to a dashboard.

new_code_churn_rate_2wfloat 0.0–1.0
example 0.31
MAINTAINABILITY · DUPLICATES

Copy-paste went 8× since AI

GitClear 2025. Measured. Now check yours.

What it measures

Commits containing 5+ contiguous identical non-trivial lines across multiple files. Rate per commit, median block size, segmented by origin and by AI tool.

Why it matters

Copy-paste is the fast lane to entropy: the same bug, in five places, forever. A rising duplicate rate is the leading indicator of debt you haven't paid yet.

duplicate_block_ratefloat 0.0–1.0
example 0.14
MAINTAINABILITY · MOVES

Real refactors look different

And we can tell the difference at the diff level.

What it measures

Percentage of changed lines that were moved between files in the same commit. Refactoring ratio = moved / (moved + duplicated) — a code-health index.

Why it matters

Moved code dropped from 24% to 9.5% post-AI in industry data. When your refactoring ratio rises, the team is actually extracting and organizing — not just generating more.

refactoring_ratiofloat 0.0–1.0
example 0.62
MAINTAINABILITY · PROVENANCE

Improving mature code, or churning this month's?

The age of the lines your team is rewriting.

What it measures

Git-blame buckets the age of each line being modified: under 2 weeks, 2–4 weeks, 1–12 months, 1–2 years, 2+ years. Plus percentage revising new code vs mature code.

Why it matters

GitClear found 79% of revised lines in 2024 were less than a month old. If most of your team's effort is re-churning fresh code, you're not improving the codebase — you're spinning.

79% of revisions

on code < 1 month old — industry 2024

MAINTAINABILITY · OPERATIONS

The mix of how your team writes

Added, deleted, updated, moved, duplicated.

What it measures

Lightweight five-bucket taxonomy of line operations per commit, built from diff content plus duplicate and move detectors. Overall plus per-origin breakdown.

Why it matters

A team dominated by `added` is growing fast; by `updated`, iterating; by `moved`, refactoring; by `duplicated`, accumulating debt. Shape of work, in one chart.

operation_dominantenum
example "updated"
MAINTAINABILITY · SHAPE

AI code has a shape. See it.

Focused, spread, bulk, or surgical — by origin.

What it measures

Median files, lines per file, and directory spread per commit, grouped by origin. Each origin's typical shape emerges: deep, wide, thin, or broad.

Why it matters

AI-generated commits tend to be wide & shallow (spread) — scaffolding, boilerplate. Human commits lean surgical or focused. This pattern is measurable, not anecdotal.

commit_shape_dominantenum
example "spread" | "focused" | "bulk" | "surgical"
MAINTAINABILITY · STABILITY MAP

Your repo has zones

Some stable. Some on fire. Name them.

What it measures

Per-directory rollup (depth 2 by default) of files touched, stabilized, churn events, and stabilization ratio. Directories classified stable ≥ 0.80, volatile < 0.50.

Why it matters

"The backend is a mess" is a feeling. Stability map turns it into `src/payments/` at 0.41 vs `src/shared/` at 0.92. That's something you can fix, staff, or document.

stability_map[]DirectoryMetrics[]
example src/payments/ — 0.41 · src/shared/ — 0.92
PART 3 · BEHAVIOUR

Is correctness holding up?

The dimension the field calls the hardest. Every signal here is a proxy for correctness — never a guarantee.

BEHAVIOUR · REVERT

The bluntest signal something broke

How often does your team un-ship what it shipped?

What it measures

Commits matching revert patterns, as a fraction of total. Attribution credits the ORIGIN of the reverted code — not who wrote the revert.

Why it matters

Reverts are rare but unambiguous. Segmented by origin and AI tool, they answer: which tool's code gets rolled back? That comparison is hard to argue with.

revert_rate · revert_by_toolfloat · Record
example 0.024 · { claude: 3, copilot: 7 }
BEHAVIOUR · FIX LATENCY

Does AI code break faster?

Measured in hours — from first commit to rework.

What it measures

Median time between consecutive modifications of the same file within the churn window. Attribution credits the ORIGINAL commit, not the fix.

Why it matters

Buckets: fast < 72h (probably obvious bugs), medium 72–168h (caught in review/prod), slow > 168h (subtle). Compare AI vs human fast-rework rates side by side.

fix_latency_median_hoursfloat hours
example 28.4
BEHAVIOUR · CASCADES

One bad commit, three follow-ups

Blast radius of code that doesn't quite land.

What it measures

A trigger commit followed by 1+ FIX commits on shared files within the churn window. Depth = number of follow-up fixes. Attribution credits the trigger's origin.

Why it matters

A 30% cascade rate means almost a third of your trigger commits break something. Segmented by AI tool, this tells you which tool's output carries the highest cleanup cost.

cascade_rate · cascade_median_depthfloat · float
example 0.18 · 2.0
BEHAVIOUR · DURABILITY

How much AI code survives the quarter

Git blame at HEAD. The ultimate survival test.

What it measures

For each origin and each AI tool: lines introduced vs lines still present at HEAD. Survival rate. Median age of surviving lines in days.

Why it matters

Our internal benchmark found AI-attributed lines survive at 79% vs human 64% — on primed repos. Durability is the counter-intuitive headline: AI code may last longer when attributed properly.

survival_rate (AI_ASSISTED)0.79
0.0 never survives · 1.0 all survives
BEHAVIOUR · FIX TARGETING

Whose code attracts the bugs?

Fair share vs disproportionate share.

What it measures

For each FIX commit's target files, credit the origin of the last non-fix commit. Compute code share vs fix share vs disproportionality (fix/code).

Why it matters

If AI wrote 30% of commits but attracts 50% of fixes, disproportionality = 1.67 — the clearest signal that AI-written code costs more to maintain than it first appears.

fix_target_by_origin.AI_ASSISTED.disproportionalityfloat (1.0 = fair share)
example 1.67
BEHAVIOUR · CLUSTER

Does one regime break more than the other?

Three correctness signals, conditioned on who wrote the code.

What it measures

Revert rate, hotfix rate and time-to-fix, each computed for AI-attributed and human-attributed commits, reported as a ratio. Above 1.0 means the AI regime behaves worse in this repo.

Why it matters

A raw number tells you nothing without a baseline; the comparison is the metric. It is withheld entirely below 30 commits per side, so a small sample can never read as a verdict.

behaviour_harness.metrics.revert_rate.relativefloat · > 1 = AI worse
example 1.24
PART 4 · ATTRIBUTION

Who wrote this, really?

Every dimension above can be cut by origin — but only if attribution holds. These metrics measure the measurement.

ATTRIBUTION · ORIGIN

Human, AI-assisted, or bot

No guessing. We read co-author tags and author patterns.

What it measures

Co-author matches Copilot, Claude, Cursor, Codeium, Tabnine, Amazon Q, Gemini, or Windsurf → AI_ASSISTED. Known bot names → BOT. Everything else → HUMAN.

Why it matters

Every single other metric in devXOS can be segmented by origin. This is the dimension that unlocks AI impact analysis — without surveys, without self-report.

commit_origin_distributionRecord<origin, count>
example { HUMAN: 210, AI_ASSISTED: 140, BOT: 28 }
ATTRIBUTION · COVERAGE

How much AI work is already visible

The other half of this number is your attribution gap.

What it measures

AI-attributed commits as a percentage of all non-bot commits. A proxy for how much of the actual AI usage is declared in the git metadata.

Why it matters

Compliance officers, AI governance leads, and skeptical CTOs all ask the same thing: how much AI is in our code? This is the answer you can defend in a meeting.

62.4% of non-bot commits

attributed to AI tools

ATTRIBUTION · ATTRIBUTION

The AI work nobody tagged

Human-classified commits with AI-shaped velocity patterns.

What it measures

Flags HUMAN commits hitting 2+ of: 3 commits in 2h, 100+ LOC, < 30min since prev, 5+ files. We never call it AI — we surface the gap for review.

Why it matters

If ai_detection_coverage says 40% and attribution gap flags another 30% of human commits as suspect, your real AI footprint is double what your governance dashboard shows.

attribution_gap.flagged_pctfloat 0.0–1.0
example 0.286
ATTRIBUTION · INTENT

Feature, fix, refactor, config

Every commit classified. Deterministically. No ML.

What it measures

Conventional Commit prefixes first, keywords second, file-type heuristic third. Every commit gets an intent: FEATURE, FIX, REFACTOR, CONFIG, or UNKNOWN.

Why it matters

"We're shipping fast" means nothing if 60% is FIX. Intent distribution turns a flat commit count into a picture of what the team is actually spending time on.

commit_intent_distributionRecord<intent, count>
example { FEATURE: 45, FIX: 32, REFACTOR: 12, CONFIG: 8 }
ATTRIBUTION · ACCEPTANCE

Does AI code pass review?

Single-pass PRs vs rounds of changes-requested.

What it measures

Per origin and per AI tool: fraction of commits that landed via a PR; of those, fraction merged with zero CHANGES_REQUESTED; median review rounds.

Why it matters

Two different AI tools can produce code that reviews very differently. Acceptance rate quantifies that — it's the missing link between "AI productivity" claims and peer-reviewed outcomes.

acceptance_by_origin.AI_ASSISTED.single_pass_ratefloat 0.0–1.0
example 0.71
ATTRIBUTION · FUNNEL

The full journey, per origin

Committed → In PR → Stabilized → Still alive.

What it measures

Four-stage delivery funnel computed per origin, with conversion rates between each step. Composes origin distribution, acceptance, stabilization, and durability.

Why it matters

AI might crush commits and pass review — and still drop off at stabilization. The funnel reveals exactly where each origin wins and where it leaks. One chart, full story.

origin_funnel.AI_ASSISTEDFunnelStage[]
example [Committed → InPR → Stabilized → Surviving]
ATTRIBUTION · ADOPTION

The day AI changed your metrics

Detected automatically. Pre vs post, side by side.

What it measures

Finds the inflection point where AI-attributed commits began appearing. Splits history into pre-adoption and post-adoption, each with a full ReportMetrics snapshot.

Why it matters

Before-and-after proof. Stabilization went from 0.71 to 0.84 since the Copilot rollout? That's a number you can put on a slide. Reversed? That's a number you need to look at fast.

adoption_timeline.adoption_confidenceenum
example "clear" · pre 0.71 → post 0.84
PART 5 · FLOW

How does work actually move?

Where time goes between the first commit and the merge — and whether anyone still looks at the code on the way.

FLOW · PR LIFECYCLE

Quantify review friction

Before it becomes a complaint in the retro.

What it measures

Median time-to-merge, median PR size (files and lines), median review rounds, and single-pass rate — the fraction of PRs merged without a CHANGES_REQUESTED review.

Why it matters

Single-pass rate is the PR metric that correlates most with team satisfaction. Combined with time-to-merge, it tells you whether review is a gate or a bottleneck.

pr_single_pass_rate · pr_median_time_to_merge_hoursfloat · float
example 0.64 · 18.2
FLOW · FLOW EFFICIENCY

Lifecycle time, split into active and wait

Throughput rose. Lead time didn't. This is why.

What it measures

Decomposes every merged PR into four phases — coding, awaiting first review, in review, awaiting merge — and classifies each as active or wait. AI compresses coding time; review queues stay flat. The ratio tells you where the time really goes.

Why it matters

Without this, AI looks like a win on commits/week while lead time flatlines. With it, you see exactly which queue is absorbing the speedup — and whether the gain is reaching production.

flow_efficiency_median · median_time_to_first_review_hoursfloat · float
example 0.34 · 18.6
FLOW · FLOW LOAD

How much work is in flight, not just shipping

AI expands WIP faster than it expands throughput.

What it measures

Per ISO week: how many PRs were simultaneously in flight, segmented by intent, alongside the count of distinct commit authors. A system-level WIP signal — never per person.

Why it matters

Healthy throughput with rising WIP and flat author concurrency is congestion in disguise. This is the metric that explains why a team feels busier when the dashboards look fine.

flow_load[].wip_total · flow_load[].author_concurrencyint · int
example 12 · 7
FLOW · STUCK INVENTORY

The PRs nobody is looking at

A snapshot of open work that never moved.

What it measures

Counts non-draft, non-bot PRs still open right now. Reports median age, stale share (no review or commit in 14+ days), and abandonment share (60+ days). Broken down by intent and code origin.

Why it matters

Throughput metrics describe work that moved. This describes work that stopped. When AI generation outruns review capacity, this is the metric that catches it first — and the AI-vs-human gap shows up clearly.

stale_open_pr_pct · stale_open_pr_pct_by_originfloat · {origin: float}
example 0.31 · { AI_ASSISTED: 0.48, HUMAN: 0.22 }
FLOW · REVIEW

Is anyone still reading the code?

The fraction of merged PRs a human actually reviewed.

What it measures

Merged PRs carrying at least one human review, over all merged PRs — with the same fraction cut by change intent and by the origin of the PR's commits.

Why it matters

When AI writes more of the code and review is the last human checkpoint, this is the coverage of that checkpoint. A rate falling while AI share rises is the pattern worth catching early.

human_review_coverage_pct · _by_origin_of_prfloat 0.0–1.0
example 0.62
FLOW · TIMELINE

Every week tells a story

Weekly breakdown + four pattern detectors.

What it measures

ISO-week rollup: commits, LOC, intent mix, origin mix, stabilization, churn, PRs merged. Patterns auto-detected: burst_then_fix, quiet_period, ai_ramp, intent_shift.

Why it matters

When a metric jumped, you need to know why. The timeline + patterns layer gives you an annotated story — not just numbers, but the moments that made them.

activity_patterns[].patternenum
example "ai_ramp" · week 10/14
FLOW · VELOCITY

Speed means nothing if durability drops

Commits/week, lines/week, and the correlation with quality.

What it measures

14-day windows of commits/week and lines/week. Trend classified accelerating, stable, decelerating. Correlated with per-window stabilization — the durability connection.

Why it matters

Accelerating with durability steady = real progress. Accelerating while stabilization drops = you are shipping noise faster. Velocity alone lies. Velocity + durability tells the truth.

velocity.trend · velocity.durability_correlationenum · enum
example "accelerating" · "decoupled"
PART 6 · PRODUCTION

Where the code meets reality

The engine sees the code — fast, visible before production. DORA sees production — slow, visible after the damage. These three connect them.

PRODUCTION · DORA × ORIGIN

Connect Datadog, see DORA crossed with code origin

Optional integration. Vendor-agnostic by default.

What it measures

When a team opts in, devXOS pulls Change Failure Rate, MTTR, lead time, deploy frequency and rollback rate from Datadog's DORA events, then joins them by commit SHA against local origin classification. Treats pending evaluations as their own bucket — no fake CFR while Datadog hasn't decided yet.

Why it matters

The closing answer to the question this deck opens with: does AI-assisted code survive in production? With Datadog connected, CFR and rollback rate by origin make the answer numerical — for the teams that choose to provide it.

dora_cfr · dora_cfr_by_originfloat · {origin: {failed, evaluated, cfr}}
example 0.08 · { AI_ASSISTED: 0.11, HUMAN: 0.06 }
PRODUCTION · CROSS-SIGNALS

Does the code signal lead the outage?

Four curated pairs, tested for lead-lag — not a blind scan.

What it measures

Weekly engine series correlated against weekly DORA series at non-negative lags: cascade rate against change failure rate, stabilization by origin against lead time, new-code churn against deploy frequency.

Why it matters

If a code signal reliably leads a production failure, you get warning weeks earlier. The guardrails matter more than the finding: Šidák correction for the lag scan, tail censoring of look-ahead series, and per-week minimum denominators.

cross_correlations.pairs[].spearman · lag_weeksfloat −1…1 · int weeks
example 0.62 · 2
PRODUCTION · ECONOMICS

What did the tokens actually buy?

Agent spend, crossed with the survival of the code it produced.

What it measures

Anonymous token counts priced per model, joined per repository against the same window's stabilization, survival by origin, revert rate and cycle time.

Why it matters

Cost per generated line is a vanity number; cost per feature that lasted is a decision. Models we cannot defensibly price are reported as null and listed — the total under-reports transparently rather than guessing.

estimatedCostUsd × durabilityDeltaUSD · pp signed
example 1 284.00 · +6.1
DEVXOS

Run it on a repo you care about

One command. The engine runs locally — the platform is optional.

What it measures

Every metric in this deck is a documented field with a name, a unit, and a stated null case. Nothing here is a black box, and nothing here scores a person.

Why it matters

Metrics are hypotheses, not truths. Point the CLI at your repository, read the report, and disagree with it — that is the intended use.