↑ ↓ to navigate
Confidential Client · Insurance Quoting Tool · Internal leadership review — confidential

Engineering velocity with the BAL Agentic Harness

Nine weeks of delivery on the SME insurance quoting platform (React 19 + .NET 10 on AKS), measured from the repository's own pull-request record and compared like-for-like — per engineer, per week — against published industry benchmarks. Every figure links to a source that can be opened and checked.

512 merged PRs analyzed 333 classified as real code changes 4-engineer core team 2026-05-04 → 2026-07-03 · 8.7 weeks 15 external sources, adversarially verified
Prepared 2026-07-03 · BCG Platinion for Confidential ClientBAL Agentic Harness
Executive summary

Four engineers delivered 4.8× the elite industry benchmark — with the quality signals of an elite team intact

Counting only real code changes, and the full window including project bootstrap. The steady-state figure is 6.9×.

The team merged 512 pull requests in 8.7 weeks. After excluding every chore, docs and scaffolding PR (35%), 333 real code changes remain — 9.6 per engineer per week against an elite industry tier that starts at 2.0[1].

  • 01Throughput — 4.8–6.9× the elite merge-frequency tier, on the most conservative cut of the data.
  • 02Speed — median PR merges in 37 minutes; industry rates anything under a day as "great"[6].
  • 03Quality — 2.6 test lines per production line, zero reverts in 512 merges, every PR gated by pipelines + SonarCloud[R4].
Code PRs / engineer / week
9.6
Elite tier starts at >2.0[1] · steady state 13.8
Median PR open → merge
37 min
96% inside the <24 h "Great" band[6]
Test : production code
2.6 : 1
112,827 test vs 42,983 production lines[R2]
Reverts in 512 merges
0
Elite change-failure benchmark <1%[1], DORA ~5%[3]
Source: Azure DevOps PR record [R1]; main branch history [R2]; LinearB [1]; Swarmia [6]; DORA 2024 [3] — full references on the Sources slideConfidential Client · Insurance Quoting Tool — confidential
What was counted

Every headline number counts real code only — 35% of merged PRs were excluded as chores, docs and scaffolding

Each of the 512 merged PRs was classified from its actual diff. Documentation, planning artifacts, dependency chores, repo scaffolding and generated files (API clients, lockfiles, EF migrations) count for nothing. Vendor benchmarks count all merged PRs — so the true gap vs industry is larger than shown.

Classification of all 512 merged PRs
By conventional-commit type and diff content · hover any segment
Table view

"Real code change" = type ∈ {feat, fix, refactor, perf, test} and the diff touches hand-written source (.cs / .ts / .tsx / .py / .js / .sql under the backend, frontend or mock roots). The 333 code PRs changed 155,810 hand-written source lines.

Source: Azure DevOps completed-PR record [R1]; first-parent diff of every merge on main [R2]Confidential Client · Insurance Quoting Tool — confidential
Throughput · like-for-like

At 9.6 real code PRs per engineer per week, the team clears the elite tier nearly five-fold

Normalized to merged PRs per developer per week — the unit LinearB publishes from 8.1M+ PRs across 4,800 engineering teams[1], the closest available proxy for a team not using an agentic harness. Counting all PRs the way vendors do, the figures are 14.7 (full window) and 20.8 (steady state).

Table view
Source: this team — 333 code PRs ÷ 4 engineers ÷ 8.7 weeks [R1][R2]; benchmark tiers — LinearB Engineering Benchmarks, 8.1M+ PRs / 4,800 teams / 42 countries [1]Confidential Client · Insurance Quoting Tool — confidential
Delivery cadence

Within three weeks of adopting PR flow, delivery settled at a steady ~83 merged PRs per week

Weeks 20–21 were a direct-push bootstrap phase (81 commits, no PRs). Disciplined PR-based delivery started 2026-05-25 and held its level for six consecutive weeks — this is a sustained pace, not a spike.

Table view
Source: first-parent merge record of main, weekly buckets by ISO week [R2]Confidential Client · Insurance Quoting Tool — confidential
Cycle time

The median PR merges in 37 minutes — industry rates anything under a day as "great"

The harness produces the review evidence — tests, QA passes, verification output — alongside the code, so review starts from proof rather than from scratch. Every merge to main then auto-deploys to the test and int environments[R3] (~12 per workday), matching the DORA Elite on-demand deployment profile[3]; the production gate opens at go-live.

PR cycle time distribution — 333 code PRs, open → merge
Azure DevOps creation → completion timestamps · 61% merge within 1 hour · 96% inside the <24 h "Great" band [6]
Table view
Source: Azure DevOps PR timestamps [R1]; CI/CD trigger definition [R3]; Swarmia cycle-time tiers [6]; LinearB elite commit-to-production <26 h [5]; DORA 2024 tiers [3]Confidential Client · Insurance Quoting Tool — confidential
Quality of the velocity

The speed is not borrowed from quality: 2.6 test lines per production line and zero reverts in 512 merges

Speed is cheap if it ships defects. Three independent signals say it doesn't: the test-to-production ratio, a revert-free merge history, and pipeline + SonarCloud gates on every PR[R4] — the discipline DORA found missing where AI adoption degraded stability[3].

Where the changed source lines went
Hand-written source only — generated code, lockfiles and docs excluded [R2]
Table view
Code-PR size distribution (source lines changed)
Median 284 · 40% ≤ 200 lines (Swarmia "Great" [6]) · sizes include the 2.6:1 test code
Table view
Source: per-PR diff statistics from main [R2]; PR validation & SonarCloud pipelines [R4]; Swarmia batch-size tiers [6]Confidential Client · Insurance Quoting Tool — confidential
Benchmark position

The team sits at or above elite on six of seven published metrics — the seventh is PR size, inflated by shipping the tests

Vendor and DORA tiers as published; this team measured from repository and pipeline records. Definitions differ per row — see the appendix.

MetricThis team (measured)Elite / "Great" tierMedian / "Good" tierSource
Merge frequency (PRs/dev/week)9.6 code-only · 14.7 all PRsabove elite> 2.01.2 – 2.0[1] LinearB
PR cycle time (open → merge)median 37 min · 96% < 24 habove elite< 24 h "Great"< 5 days "Good"[6] Swarmia
Lead time for changes< 1 day (merge → auto-deploy test/int)elite< 1 day1 day – 1 week[3] DORA 2024
Deployment frequency~12 merges/workday, each auto-deployed (pre-prod)eliteOn demandDaily – weekly[3] DORA 2024
Change failure proxy (reverts)0 / 512 mergesabove elite< 1% · DORA 5%1 – 4% · DORA 20%[1] [3]
Issues resolved / engineer / week9.6 code PRs as proxyabove elite3.2+1.6 – 3.1[2] Jellyfish
PR size (lines changed)median 284, incl. tests"Good"< 100 LinearB · < 200 Swarmia< 500 Swarmia[1] [6]

On PR size: harness PRs ship a complete vertical slice — production code, unit/integration tests and QA evidence — in one reviewable unit rather than splitting them into separate small PRs. Two-thirds of the median PR's lines are tests.

Source: LinearB [1]; Jellyfish [2]; DORA 2024 [3]; Swarmia [6]; team measurements from ADO + pipelines [R1–R4]Confidential Client · Insurance Quoting Tool — confidential
External evidence

Published evidence on AI-assisted delivery cuts both ways — the harness is built to counter exactly the documented failure modes

Controlled studies show large individual speed-ups; ecosystem-level studies warn that speed without engineering discipline degrades stability. This team lands on the right side of both: throughput far above elite and the stability signals intact — the argument for the harness itself, not just for "using AI".

Evidence for acceleration

  • 55.8% faster task completion in GitHub's randomized controlled trial of Copilot (95 developers, P=.0017) — arXiv 2302.06590 · github.blogverified 3-0
  • Up to 2× faster on common coding tasks in McKinsey's 2023 lab study — mckinsey.comverified 3-0
  • +8.69% PRs, +15% merge rate for Copilot users in the GitHub/Accenture enterprise field study — github.blogquote-verified
  • +12.9–21.8% weekly PRs at Microsoft in field experiments covering 1,974 developers — MIT GenAIquote-verified

Evidence for caution — what the harness must (and does) counter

  • −1.5% throughput, −7.2% stability per 25% increase in AI adoption, industry-wide — DORA 2024, Fig. 10 p.39 — dora.dev PDF · Google Cloudverified 3-0
  • 19% slower: experienced open-source developers with early-2025 AI tools in METR's RCT (16 devs, 246 tasks) — unstructured AI use is not automatically faster — metr.org · arXiv 2507.09089quote-verified
  • 4× more code cloning; copy/paste exceeded refactored code for the first time in 2024 — GitClear. Counter-signal here: 51 refactor PRs and a 2.6:1 test ratio.gitclear.comquote-verified
  • <10% gains on high-complexity tasks, juniors 7–10% slower — McKinsey's own caveats — mckinsey.comverified 3-0
Source: each study linked inline; verification method in the appendix — 3-vote adversarial pass on primary sources, run 2026-07-03Confidential Client · Insurance Quoting Tool — confidential
Appendix

Methodology, honest limits and sources

The numbers are conservative by construction — and every caveat is stated, not hidden.

Confidential Client · Insurance Quoting Tool — confidential
Appendix A

Methodology & honest limits

Seven things a skeptical reader should know before trusting the headline.

  1. Data. All 512 merged PRs on main (first-parent history [R2]) cross-joined with Azure DevOps PR metadata (creation/completion timestamps, authors) [R1]. Window: 2026-05-04 → 2026-07-03 (61 days = 8.7 weeks). Steady state = W22–W27 (2026-05-25 onward), after a two-week bootstrap in which 81 commits were pushed directly without PRs.
  2. "Real code change" definition. Conventional-commit type ∈ {feat, fix, refactor, perf, test} and the diff touches hand-written source (.cs .ts .tsx .py .js .sql). Excluded from line counts: generated API clients, package-lock.json, EF migrations, docs/index.md, images, data files, all docs/planning.
  3. Like-for-like normalization. Per-engineer figures divide by the 4 core engineers, who authored 100% of the 333 code PRs (six occasional contributors authored 17 non-code/support PRs). Authorship is concentrated — one engineer (the harness operator) merged 63% of code PRs — so per-engineer numbers are team averages, not individual claims.
  4. Benchmark caveats. LinearB/Swarmia/Jellyfish figures are vendor telemetry, not peer-reviewed research; they count all merged PRs (our all-PR figure: 14.7/dev/week) and measure predominantly human-authored PRs — agent-authored PRs merged under human review is a definitional difference we flag rather than hide. Cycle-time definitions differ (open→merge vs commit→production). DORA "lead time" and "deployment frequency" are measured here to pre-production environments; the production gate opens at go-live (2026-10-26).
  1. Change-failure proxy. Zero reverts/hotfixes among 512 merges is a proxy from commit history, not a formal incident-based change-failure rate; the app is pre-go-live, so production incident data does not exist yet.
  2. Fix share. 114 of 333 code PRs (34%) are fix type. Most were caught by the harness's own QA/test stages in the same sprint — visible in each PR's description and linked work items [R1].
  3. External claims verification. Industry numbers were gathered and adversarially verified by a multi-agent research pass: each claim fetched from its primary source and voted on by three independent verifier agents ("verified 3-0"). Claims that failed verification were discarded; two cautionary sources (METR, GitClear) carry verbatim quotes from direct fetches but not the full 3-vote pass ("quote-verified").
Verify it yourself — the per-PR record is reproducible from the repository:
git log --first-parent main --pretty='%H|%ad|%an|%s' --date=iso   # 512 "Merged PR" commits
git diff --numstat -M <merge>^1 <merge>                            # per-PR effective diff
or browse the completed-PR list in Azure DevOps [R1] — any individual PR is at …/pullrequest/<id>.
Source: repository [R1–R4]; analysis scripts preserved at storage/velocity-report/scripts/Confidential Client · Insurance Quoting Tool — confidential
Appendix B

Sources — every claim is one click from its origin

External benchmarks were fetched from their primary URLs and adversarially verified by three independent agents each; the vote is shown. Repository sources require Confidential Client Azure DevOps access.

External
  1. LinearB Engineering Benchmarks (2026)verified 3-0 8.1M+ PRs, 4,800 teams — merge frequency Elite >2.0/dev/week; PR size Elite <100; CFR Elite <1%.
  2. Jellyfish Engineering Benchmarksverified 3-0 78,000 engineers — coding days Elite 2+/wk; issues resolved Elite 3.2+/wk; deploys Elite daily+.
  3. DORA — State of DevOps 2024 (PDF)verified 3-0 Tiers p.13 (Elite: lead <1 day, on-demand deploys, 5% CFR); AI findings p.39 (−1.5% throughput, −7.2% stability per +25% adoption).
  4. DORA — State of DevOps 2023 (PDF)verified 3-0 Tier table p.12; AI capability effects p.21 ("minor decrease" on delivery performance).
  5. LinearB — Software development KPI benchmarksverified 2-0 Elite commit-to-production cycle time <26 h vs >167 h needing improvement.
  6. Swarmia — Benchmarks and comparisonsverified 3-0 PR cycle time Great <24 h / Good <5 days; batch size Great <200 / Good <500 lines; first review ≤4 h.
  7. Peng et al. — Impact of AI on Developer Productivity (arXiv)verified 3-0 RCT, 95 developers: Copilot group 55.8% faster than control.
  8. GitHub — Quantifying Copilot's impactverified 3-0 Same RCT: 1 h 11 m vs 2 h 41 m task time.
 
  1. McKinsey — Unleashing developer productivity with generative AIverified 3-0 Up to 2× on common tasks; <10% on complex; juniors 7–10% slower.
  2. Google Cloud — Announcing the 2024 DORA reportverified 3-0 Official corroboration of the AI-adoption findings.
  3. METR — Early-2025 AI and experienced OSS developersquote-verified RCT, 16 devs, 246 tasks: AI-allowed tasks took 19% longer (CI +2%…+39%). Also arXiv 2507.09089.
  4. GitClear — AI code quality research (2025)quote-verified 4× growth in code cloning; copy/paste exceeded moved code in 2024.
  5. GitHub — Copilot in the enterprise with Accenturequote-verified +8.69% pull requests, +15% PR merge rate.
  6. MIT GenAI — field experiments (1,974 developers)quote-verified +12.92–21.83% weekly PRs at Microsoft with Copilot access.
Repository (Confidential Client ADO access required)
  1. [R1] Completed pull requests The 512 merged PRs with timestamps, authors, linked work items. Individual PRs: …/pullrequest/<id>.
  2. [R2] main branch history First-parent commit record; source of all diff/line statistics.
  3. [R3] CI/CD pipeline definition Every push to main → build + deploy to test and int.
  4. [R4] PR validation & SonarCloud pipelines Backend/frontend gates and SonarCloud analysis on every PR.
Prepared 2026-07-03 from the project repository and Azure DevOps records · external claims verified 2026-07-03Confidential Client · Insurance Quoting Tool — confidential