// engineering metrics

how i read an engineering organization

I look at metrics to understand how work moves through a team, not to decide who deserves a gold star. I don’t use them to grade individual engineers.

A number is useful when it helps us ask a better question, and the question usually leads to a decision: what to stop, what to protect, what to change. This page is about that path, from a signal on a chart to a decision and back to the evidence about whether it worked.

Everything below is simulated: invented teams whose work items have real, checkable histories, so the charts follow from the events. None of it describes a team I’ve managed.

// reading the flow

From a signal to a decision

A cumulative flow diagram stacks every work item by the state it’s in, day by day. The top edge is everything that has entered the system, the bottom band is everything finished, and the thickness of each band is how many items sit in that state.

Each scenario below starts with a shape on that chart and follows it through four steps: what I observe, what I investigate, what I decide, and what the evidence says afterward. Step through them in order. The chart only shows what was knowable at each step, and not every decision works out exactly as planned.

cumulative flow, items simulated scenario
Scenario

// scenario a — accumulating work

When everything is a priority, nothing is.

Cumulative flow diagram for scenario A, Aug 3 to Oct 25: ready, in progress, in review, done, stacked by the number of items in each state at the end of each day.

  • admitted (top edge)
  • ready
  • in progress
  • in review
  • done

Hover, tap or focus the chart to read any visible day.

  1. Aug 10 Product adds the partner API workstream context
  2. Aug 17 Product adds the reporting workstream context
  3. Aug 24 Product adds the mobile parity workstream context
  4. Aug 26 in progress passes 10, two per engineer measured
  5. Sep 7 priorities narrowed; new work starts only below five in progress decision
  6. Sep 22 A-050 enters straight into progress measured
  7. Sep 29 in progress back down to five measured
  8. Oct 6 A-055 enters straight into progress measured
Phase of scenario A

01 / observe

Work is arriving faster than it’s leaving.

The chart shows Aug 3 to Aug 30. In progress rose from 4 to 13 items while done rose by 18. Weeks after Aug 30 are hidden.

Over four weeks the in-progress band grew from 4 items to 13. On a team of five, that’s more than two items per engineer. Throughput didn’t follow: the team finished 5, 5, 4 and 4 items in those weeks while 5, 6, 7 and 8 arrived.

Cycle time is starting to stretch. Items finished in the first two weeks took a median of 6.8 days from first start to done; items finished in the last two took 9.8 days. When more work is open and the finishing rate holds still, each item has to wait longer. That’s the arithmetic of flow, not a judgment about effort.

This is a signal, not a verdict. The chart tells me work is piling up in development. It doesn’t tell me why, and I don’t want to guess.

  • measured work-item history

    Finished per week: 5, 5, 4 and 4. Arrived per week: 5, 6, 7 and 8.

    items finished per week, with items arriving

    • finished
    • arrived
    • finished, weeks in focus

    Hover, tap or focus a mark for its values.

  • measured work-item history

    Median cycle time of items finished Aug 3 – Aug 16: 6.8 days (10 items). Aug 17 – Aug 30: 9.8 days (8 items).

    cycle time of each finished item, by the day it finished

    y: elapsed calendar days from first start to done; x: the day each item finished

    values (18 items)
    • A-007 · onboarding · started Jul 29, done Aug 3 · 5.2 days
    • A-005 · billing · started Jul 27, done Aug 4 · 8 days
    • A-006 · billing · started Jul 28, done Aug 4 · 7.3 days
    • A-008 · billing · started Jul 30, done Aug 6 · 7.2 days
    • A-009 · onboarding · started Jul 31, done Aug 6 · 6.3 days
    • A-011 · billing · started Aug 4, done Aug 10 · 6.2 days
    • A-010 · billing · started Aug 3, done Aug 12 · 9.1 days
    • A-014 · onboarding · started Aug 7, done Aug 13 · 6.1 days
    • A-012 · onboarding · started Aug 5, done Aug 13 · 8.3 days
    • A-015 · billing · started Aug 10, done Aug 14 · 4.3 days
    • A-013 · billing · started Aug 6, done Aug 18 · 12.2 days
    • A-016 · billing · started Aug 11, done Aug 19 · 8.2 days
    • A-017 · partner API · started Aug 12, done Aug 19 · 7.2 days
    • A-018 · onboarding · started Aug 12, done Aug 20 · 8.1 days
    • A-019 · billing · started Aug 13, done Aug 25 · 12.3 days
    • A-021 · billing · started Aug 17, done Aug 26 · 9.3 days
    • A-020 · onboarding · started Aug 14, done Aug 27 · 13.2 days
    • A-022 · billing · started Aug 18, done Aug 28 · 10.3 days

    Hover, tap or focus a mark for its values.

  • hypothesis open question

    Possible causes so far: the work got harder, the team lost capacity, or more of it started at once. The history alone can’t choose between them.

02 / investigate

What is causing the accumulation?

The chart now runs to Sep 6. In progress stands at 17 items across five workstreams.

My first move isn’t to change the process or ask for more output. It’s to find out what’s actually active, where it’s stuck, and what the people doing the work are experiencing.

The history answers part of that. The 17 items in progress now span five workstreams, up from two four weeks ago, and new items were started a median of 1 day after they arrived: only one item waits in Ready, because everything is urgent. Nine of those 17 have already been open longer than 6.8 days, which is how long a whole item took in early August.

The conversations answer the rest. Taken together, the evidence points to the team being spread across more priorities than it can move at once. That’s still a hypothesis, but now it’s one with support from both the data and the people.

what i want to understand
  • Which workstreams are active right now, and who is on each?
  • Where is work getting stuck: in development, review, or waiting on someone?
  • Which of these priorities are genuinely urgent, and which only feel that way?
  • Is the pressure coming from real business commitments?
  • What are the engineers experiencing day to day?

  • measured work-item history

    Items in progress by workstream, Aug 3 and Sep 6.

    items in progress, by workstream

    workstream Aug 3Sep 6
    billing 3 6
    partner API 0 2
    onboarding 1 5
    reporting 0 2
    mobile parity 0 2

  • reported one-on-ones, simulated

    Several engineers describe switching between three or four items a day and being pulled toward whichever priority was raised most recently. Nobody is complaining about workload, which I don’t take as a sign that everything is fine.

  • reported Product, simulated

    Product sees all five workstreams as committed for the quarter. Two, billing and partner API, have external dates. The other three matter, but nothing outside the company is waiting on a date for them.

  • hypothesis my read

    Starting everything at once is costing more in context switching than it gains in progress. Fewer things in motion should finish more of them.

03 / decide

Narrow the active priorities, together.

The decision marker sits at Sep 7, the morning after the last day investigated.

I don’t take this to Product as Engineering overruling the roadmap. Product owns the priorities. My job is to make the cost of running five at once visible, and to help us choose with that cost on the table.

We agree to keep billing and partner API active, because they carry the external dates, and to pause onboarding, reporting and mobile parity. Paused means no new work from those streams for now, re-planned with their stakeholders, not cancelled. Work already in flight on them gets finished: abandoning half-done work wastes what was spent on it, and moving it back to the queue would only hide it.

For now, new work starts only when fewer than five items are in progress, one per engineer. That rule doesn’t shrink anything by itself. Today 17 items are in progress, so nothing new starts until enough of them finish, and new requests will wait in Ready, visibly and in Product’s order, until they do.

the tradeoffs
  • Pausing three workstreams delays real business value. That cost is Product’s to weigh, and they weigh it knowing what running everything was costing.
  • New requests, even for the active priorities, will wait until the work in flight drains. That’s slower to start, and I expect it to be faster to finish.
  • A limit can turn into a rule people work around. It’s a tool for this team, right now, and we’ll revisit it if it stops helping.

  • decision Product and Engineering

    Active priorities: billing and partner API. Paused: onboarding, reporting and mobile parity, with new dates set together; their work in flight is finished. Billing’s intake slows to a pace the team can finish.

  • decision Engineering

    Start new work only when fewer than five items are in progress, in Product’s priority order, oldest first within each workstream.

04 / validate

Did focusing help, or did the pressure just move?

The full chart, Aug 3 to Oct 25. After the decision, in progress fell as items finished, 13, 10, 7, 4, 5, 5 and 5 at week ends, while ready held the new requests.

Work in progress came down only as work finished: 13, 10, 7, 4, 5, 5 and 5 items at the end of each week since the decision. Nothing was moved back to Ready: all 17 items in progress at the decision went on to finish, the last on Oct 6. In progress first fell to five on Sep 29, three weeks after the decision.

For a while, cycle time looked worse, not better. The first items to finish were the oldest ones, so the median by week of finishing climbed to 28.1 days before falling to 7.2 days in the last two weeks. Work started since the decision moves at the old pace: the eleven items that have finished took a median of 7.3 days, 85% within 10.2 days, and five are still open. Counted from when each request arrived, those same items took a median of 21.2 days, most of it waiting in Ready: the decision shortened the time from start to done, not yet the time from request to delivery. Items started during the crowding took 22.1 days, 85% within 28.3 days; before it, 7.2 days.

Throughput didn’t rise: 5, 2, 4, 5, 4, 5 and 4 items a week, against 5, 5, 4, 4 and 3 before. Focus doesn’t create capacity, it stops wasting it, and what improved is how long each new item takes and how predictable that is.

The pressure didn’t disappear. Some of it moved into Ready, where it belongs: new requests waited there, from 1 at the decision to 7 at the peak and 1 at the end. Some of it went around the queue: two requests went straight to an engineer without passing through Ready. That’s my next conversation with Product, because a limit with a side door isn’t a limit.

After seven weeks, the measures moved the way we hoped. That isn’t proof the decision caused all of it, and it isn’t long enough to call it settled. I’ll keep watching the side door and keep asking people how their weeks feel.

  • measured work-item history

    Cycle time by when work started: before the crowding 7.2 days median (20 items); during it 22.1 days (23); after the decision 7.3 days (11 finished, 5 still open). Muted dots started before the decision.

    cycle time of each finished item, by the day it finished

    • started after the decision
    • started before it
    • median

    y: elapsed calendar days from first start to done; x: the day each item finished

    values (50 items)
    • A-007 · onboarding · started Jul 29, done Aug 3 · 5.2 days
    • A-005 · billing · started Jul 27, done Aug 4 · 8 days
    • A-006 · billing · started Jul 28, done Aug 4 · 7.3 days
    • A-008 · billing · started Jul 30, done Aug 6 · 7.2 days
    • A-009 · onboarding · started Jul 31, done Aug 6 · 6.3 days
    • A-011 · billing · started Aug 4, done Aug 10 · 6.2 days
    • A-010 · billing · started Aug 3, done Aug 12 · 9.1 days
    • A-014 · onboarding · started Aug 7, done Aug 13 · 6.1 days
    • A-012 · onboarding · started Aug 5, done Aug 13 · 8.3 days
    • A-015 · billing · started Aug 10, done Aug 14 · 4.3 days
    • A-013 · billing · started Aug 6, done Aug 18 · 12.2 days
    • A-016 · billing · started Aug 11, done Aug 19 · 8.2 days
    • A-017 · partner API · started Aug 12, done Aug 19 · 7.2 days
    • A-018 · onboarding · started Aug 12, done Aug 20 · 8.1 days
    • A-019 · billing · started Aug 13, done Aug 25 · 12.3 days
    • A-021 · billing · started Aug 17, done Aug 26 · 9.3 days
    • A-020 · onboarding · started Aug 14, done Aug 27 · 13.2 days
    • A-022 · billing · started Aug 18, done Aug 28 · 10.3 days
    • A-023 · partner API · started Aug 19, done Sep 1 · 13.3 days
    • A-024 · onboarding · started Aug 19, done Sep 3 · 15.2 days
    • A-025 · reporting · started Aug 19, done Sep 4 · 16.1 days
    • A-026 · billing · started Aug 20, done Sep 8 · 19 days
    • A-027 · onboarding · started Aug 21, done Sep 8 · 18 days
    • A-029 · billing · started Aug 25, done Sep 10 · 16.3 days
    • A-030 · partner API · started Aug 26, done Sep 11 · 16.2 days
    • A-028 · billing · started Aug 24, done Sep 11 · 18.2 days
    • A-031 · onboarding · started Aug 26, done Sep 16 · 21 days
    • A-032 · reporting · started Aug 26, done Sep 17 · 22.1 days
    • A-035 · mobile parity · started Aug 28, done Sep 21 · 24 days
    • A-033 · billing · started Aug 27, done Sep 21 · 25.3 days
    • A-034 · onboarding · started Aug 28, done Sep 22 · 25 days
    • A-036 · billing · started Aug 31, done Sep 23 · 23.1 days
    • A-038 · partner API · started Sep 2, done Sep 28 · 26.2 days
    • A-037 · billing · started Sep 1, done Sep 29 · 28 days
    • A-040 · reporting · started Sep 2, done Sep 30 · 28.1 days
    • A-039 · onboarding · started Sep 2, done Sep 30 · 28.3 days
    • A-041 · billing · started Sep 3, done Oct 2 · 29 days
    • A-043 · mobile parity · started Sep 4, done Oct 5 · 31 days
    • A-042 · onboarding · started Sep 4, done Oct 6 · 32.2 days
    • A-050 · mobile parity · started Sep 22, done Oct 7 · 14.9 days
    • A-044 · billing · started Sep 30, done Oct 7 · 7.3 days
    • A-046 · billing · started Oct 2, done Oct 12 · 10.2 days
    • A-055 · mobile parity · started Oct 6, done Oct 13 · 7.2 days
    • A-051 · billing · started Oct 7, done Oct 14 · 7.1 days
    • A-048 · billing · started Oct 5, done Oct 14 · 9.2 days
    • A-053 · billing · started Oct 7, done Oct 15 · 8.3 days
    • A-056 · billing · started Oct 13, done Oct 19 · 6.3 days
    • A-045 · partner API · started Oct 14, done Oct 20 · 6.2 days
    • A-047 · partner API · started Oct 14, done Oct 20 · 6.3 days
    • A-058 · billing · started Oct 14, done Oct 22 · 8.3 days

    Hover, tap or focus a mark for its values.

  • measured work-item history

    Finished per week since the decision: 5, 2, 4, 5, 4, 5 and 4. Items moved back to Ready: none. In Ready at the end of each week: 3, 5, 7, 7, 6, 3 and 1.

    items finished per week, with items arriving

    • finished
    • arrived
    • finished, weeks in focus

    Hover, tap or focus a mark for its values.

  • measured work-item history

    Two items entered straight into progress after the decision, on Sep 22 and Oct 6, without passing through Ready.

  • reported team retrospective, simulated

    Engineers describe fewer switches and clearer mornings once the in-flight work thinned out, and say the first few weeks felt slower, not faster. One mentions that mobile-parity requests still arrive by direct message, which matches the two items that skipped the queue.

the data behind scenario a (all weeks)

60 simulated work items. The chart leaves out the 4 finished before Aug 3, so its bottom band counts work finished since then, and the bands always add up to the items in scope. Counts are taken at the end of each day, UTC; the team’s working day is 09:00–17:30 UTC. Download the full history as JSON.

Scenario A: items in each state at the end of each Sunday, UTC
week ending readyin progressin reviewdone admitted
Aug 9 1415 11
Aug 16 16010 17
Aug 23 19014 24
Aug 30 113018 32
Sep 6 117121 40
Sep 13 313026 42
Sep 20 510128 44
Sep 27 77132 47
Oct 4 74137 49
Oct 11 65041 52
Oct 18 35046 54
Oct 25 15050 56
Scenario A: every work item's status entries, UTC
itemattributesstatus entries, UTC
A-001 feature · billing · finished before the chart starts ready Jul 20 09:02 → in progress Jul 21 09:25 → in review Jul 27 12:35 → done Jul 28 12:10
A-002 feature · onboarding · finished before the chart starts ready Jul 21 09:02 → in progress Jul 22 09:25 → in review Jul 28 11:45 → done Jul 29 12:28
A-003 feature · billing · finished before the chart starts ready Jul 22 09:02 → in progress Jul 23 09:25 → in review Jul 30 11:34 → done Jul 31 09:29
A-004 feature · onboarding · finished before the chart starts ready Jul 23 09:02 → in progress Jul 24 09:25 → in review Jul 29 15:11 → done Jul 30 16:08
A-005 feature · billing ready Jul 24 09:02 → in progress Jul 27 09:25 → in review Aug 3 14:44 → done Aug 4 10:07
A-006 feature · billing ready Jul 27 09:02 → in progress Jul 28 09:25 → in review Aug 4 09:05 → done Aug 4 15:57
A-007 feature · onboarding ready Jul 28 09:02 → in progress Jul 29 09:25 → in review Aug 3 09:27 → done Aug 3 14:35
A-008 feature · billing ready Jul 29 09:02 → in progress Jul 30 09:25 → in review Aug 6 10:11 → done Aug 6 14:45
A-009 feature · onboarding ready Jul 30 09:02 → in progress Jul 31 09:25 → in review Aug 6 10:40 → done Aug 6 16:18
A-010 feature · billing ready Jul 31 09:02 → in progress Aug 3 09:25 → in review Aug 12 09:42 → done Aug 12 12:43
A-011 feature · billing ready Aug 3 09:02 → in progress Aug 4 09:25 → in review Aug 7 15:36 → done Aug 10 15:07
A-012 feature · onboarding ready Aug 4 09:02 → in progress Aug 5 09:25 → in review Aug 13 09:22 → done Aug 13 16:28
A-013 feature · billing ready Aug 5 09:02 → in progress Aug 6 09:25 → in review Aug 17 13:28 → done Aug 18 13:35
A-014 feature · onboarding ready Aug 6 09:02 → in progress Aug 7 09:25 → in review Aug 12 13:42 → done Aug 13 10:45
A-015 feature · billing ready Aug 7 09:02 → in progress Aug 10 09:25 → in review Aug 14 09:41 → done Aug 14 16:25
A-016 feature · billing ready Aug 10 09:02 → in progress Aug 11 09:25 → in review Aug 18 15:15 → done Aug 19 14:23
A-017 feature · partner API ready Aug 11 09:02 → in progress Aug 12 09:25 → in review Aug 18 15:56 → done Aug 19 14:24
A-018 feature · onboarding ready Aug 11 09:06 → in progress Aug 12 09:28 → in review Aug 19 12:28 → done Aug 20 10:45
A-019 feature · billing ready Aug 12 09:02 → in progress Aug 13 09:25 → in review Aug 25 12:46 → done Aug 25 15:52
A-020 feature · onboarding ready Aug 13 09:02 → in progress Aug 14 09:25 → in review Aug 26 15:08 → done Aug 27 13:24
A-021 feature · billing ready Aug 14 09:02 → in progress Aug 17 09:25 → in review Aug 26 12:46 → done Aug 26 16:36
A-022 feature · billing ready Aug 17 09:02 → in progress Aug 18 09:25 → in review Aug 28 10:48 → done Aug 28 16:21
A-023 feature · partner API ready Aug 18 09:02 → in progress Aug 19 09:25 → in review Aug 31 16:01 → done Sep 1 16:24
A-024 feature · onboarding ready Aug 18 09:06 → in progress Aug 19 09:28 → in review Sep 2 16:53 → done Sep 3 14:14
A-025 feature · reporting ready Aug 18 09:10 → in progress Aug 19 09:31 → in review Sep 3 12:53 → done Sep 4 12:24
A-026 feature · billing ready Aug 19 09:02 → in progress Aug 20 09:25 → in review Sep 4 17:13 → done Sep 8 09:11
A-027 feature · onboarding ready Aug 20 09:02 → in progress Aug 21 09:25 → in review Sep 7 12:30 → done Sep 8 09:31
A-028 feature · billing ready Aug 21 09:02 → in progress Aug 24 09:25 → in review Sep 10 16:42 → done Sep 11 14:20
A-029 feature · billing ready Aug 24 09:02 → in progress Aug 25 09:25 → in review Sep 10 12:42 → done Sep 10 16:08
A-030 feature · partner API ready Aug 25 09:02 → in progress Aug 26 09:25 → in review Sep 10 14:34 → done Sep 11 13:34
A-031 feature · onboarding ready Aug 25 09:06 → in progress Aug 26 09:28 → in review Sep 15 10:36 → done Sep 16 09:03
A-032 feature · reporting ready Aug 25 09:10 → in progress Aug 26 09:31 → in review Sep 16 12:53 → done Sep 17 12:01
A-033 feature · billing ready Aug 26 09:02 → in progress Aug 27 09:25 → in review Sep 21 10:40 → done Sep 21 15:46
A-034 feature · onboarding ready Aug 27 09:02 → in progress Aug 28 09:25 → in review Sep 21 14:05 → done Sep 22 09:06
A-035 feature · mobile parity ready Aug 27 09:06 → in progress Aug 28 09:28 → in review Sep 18 10:10 → done Sep 21 10:21
A-036 feature · billing ready Aug 28 09:02 → in progress Aug 31 09:25 → in review Sep 22 12:00 → done Sep 23 11:36
A-037 feature · billing ready Aug 31 09:02 → in progress Sep 1 09:25 → in review Sep 28 10:01 → done Sep 29 09:27
A-038 feature · partner API ready Sep 1 09:02 → in progress Sep 2 09:25 → in review Sep 25 16:06 → done Sep 28 13:22
A-039 feature · onboarding ready Sep 1 09:06 → in progress Sep 2 09:28 → in review Sep 29 16:04 → done Sep 30 16:25
A-040 feature · reporting ready Sep 1 09:10 → in progress Sep 2 09:31 → in review Sep 29 16:11 → done Sep 30 10:54
A-041 feature · billing ready Sep 2 09:02 → in progress Sep 3 09:25 → in review Oct 1 11:15 → done Oct 2 09:15
A-042 feature · onboarding ready Sep 3 09:02 → in progress Sep 4 09:25 → in review Oct 6 09:50 → done Oct 6 15:20
A-043 feature · mobile parity ready Sep 3 09:06 → in progress Sep 4 09:28 → in review Oct 2 12:33 → done Oct 5 10:25
A-044 feature · billing ready Sep 4 09:02 → in progress Sep 30 09:25 → in review Oct 6 16:00 → done Oct 7 17:11
A-045 feature · partner API ready Sep 8 09:02 → in progress Oct 14 09:28 → in review Oct 19 14:00 → done Oct 20 13:52
A-046 feature · billing ready Sep 9 09:02 → in progress Oct 2 09:25 → in review Oct 12 11:09 → done Oct 12 14:46
A-047 feature · partner API ready Sep 15 09:02 → in progress Oct 14 09:31 → in review Oct 20 09:18 → done Oct 20 16:37
A-048 feature · billing ready Sep 16 09:02 → in progress Oct 5 09:25 → in review Oct 13 14:33 → done Oct 14 14:18
A-049 feature · partner API ready Sep 22 09:02 → in progress Oct 16 09:25
A-050 feature · mobile parity in progress Sep 22 11:20 → in review Oct 6 14:26 → done Oct 7 09:48
A-051 feature · billing ready Sep 23 09:02 → in progress Oct 7 09:25 → in review Oct 13 16:37 → done Oct 14 12:46
A-052 feature · partner API ready Sep 29 09:02 → in progress Oct 20 09:25
A-053 feature · billing ready Sep 30 09:02 → in progress Oct 7 09:28 → in review Oct 15 10:57 → done Oct 15 16:08
A-054 feature · partner API ready Oct 6 09:02 → in progress Oct 20 09:28
A-055 feature · mobile parity in progress Oct 6 11:20 → in review Oct 13 10:40 → done Oct 13 15:26
A-056 feature · billing ready Oct 7 09:02 → in progress Oct 13 09:25 → in review Oct 19 09:26 → done Oct 19 16:04
A-057 feature · partner API ready Oct 13 09:02 → in progress Oct 22 09:25
A-058 feature · billing ready Oct 14 09:02 → in progress Oct 14 09:25 → in review Oct 21 15:11 → done Oct 22 15:43
A-059 feature · partner API ready Oct 20 09:02
A-060 feature · billing ready Oct 21 09:02 → in progress Oct 21 09:25

// scenario b — reviews accelerate

Faster doesn’t always mean better.

Cumulative flow diagram for scenario B, Aug 3 to Oct 11: ready, in progress, in review, done, stacked by the number of items in each state at the end of each day.

  • admitted (top edge)
  • ready
  • in progress
  • in review
  • done

Hover, tap or focus the chart to read any visible day.

  1. Aug 17 AI-assisted review turned on context
  2. Aug 19 the review queue empties for the first time measured
  3. Sep 7 keep AI review; stop the evening checks; human review for shared contracts decision
Phase of scenario B

01 / observe

Reviews are suddenly clearing faster.

The chart shows Aug 3 to Aug 30. In review held 6, 5, 0 and 0 items at successive week ends; the queue first emptied on Aug 19.

The in-review band went from 6 and 5 items at the first two week ends to 0 and 0 after AI-assisted review was turned on, on Aug 17. Reviews that began before then took a median of 5.1 days from entering review to done; reviews since then, 0.3 days.

Completed work jumped to 11 in the week of Aug 17 as the waiting queue emptied, then settled at 6. At first glance, the process got better.

Good news deserves the same curiosity as bad news. A band that narrows this quickly tells me something changed. It doesn’t tell me whether the change is real, or what it cost.

  • measured work-item history

    Median time in review: 5.1 days for reviews begun before Aug 17 (21 items), 0.3 days since (12).

    time in review of each finished item, by the day it finished

    • review began after AI review was turned on
    • review began before

    y: elapsed calendar days in review; x: the day each item finished

    values (29 items)
    • B-005 · accounts · done Aug 3 · 4.9 days in review
    • B-007 · accounts · done Aug 4 · 5 days in review
    • B-009 · shared contracts · done Aug 4 · 4.2 days in review
    • B-006 · shared contracts · done Aug 6 · 6.1 days in review
    • B-008 · checkout · done Aug 7 · 6.8 days in review
    • B-010 · checkout · done Aug 10 · 5.7 days in review
    • B-011 · accounts · done Aug 10 · 5.8 days in review
    • B-016 · checkout · rework of B-004 · done Aug 12 · 6.8 days in review
    • B-012 · shared contracts · done Aug 12 · 6.1 days in review
    • B-014 · checkout · done Aug 12 · 5.3 days in review
    • B-013 · accounts · done Aug 13 · 6 days in review
    • B-015 · shared contracts · done Aug 13 · 2.4 days in review
    • B-017 · checkout · done Aug 17 · 6.8 days in review
    • B-025 · accounts · rework of B-011 · done Aug 17 · 0.2 days in review
    • B-019 · shared contracts · done Aug 18 · 4.8 days in review
    • B-021 · checkout · done Aug 18 · 5.1 days in review
    • B-020 · accounts · done Aug 18 · 4.1 days in review
    • B-023 · checkout · done Aug 18 · 0.1 days in review
    • B-022 · shared contracts · done Aug 18 · 0.4 days in review
    • B-018 · accounts · done Aug 19 · 6 days in review
    • B-024 · accounts · done Aug 20 · 0.2 days in review
    • B-027 · accounts · done Aug 21 · 0.1 days in review
    • B-026 · shared contracts · done Aug 21 · 0.4 days in review
    • B-028 · checkout · done Aug 25 · 0.8 days in review
    • B-030 · checkout · done Aug 25 · 0.1 days in review
    • B-029 · shared contracts · done Aug 26 · 1.2 days in review
    • B-033 · accounts · done Aug 28 · 0.8 days in review
    • B-031 · accounts · done Aug 28 · 0.2 days in review
    • B-032 · shared contracts · done Aug 28 · 0.4 days in review

    Hover, tap or focus a mark for its values.

  • measured work-item history

    Finished per week: 5, 7, 11 and 6. Entered per week: 7, 7, 6 and 6.

    items finished per week, with items entering ready

    • finished
    • entered ready
    • finished, weeks in focus

    Hover, tap or focus a mark for its values.

  • hypothesis open question

    Better tooling, smaller changes, or less careful reviews could each produce this shape. The chart can’t tell them apart.

02 / investigate

What changed, and what did it cost?

The chart now runs to Sep 6. In review stands at 0 items.

I ask what changed before I celebrate it. Several explanations fit the same chart, and some of them are problems.

Quality hasn’t visibly moved. Of the items finished before AI review with two weeks of follow-up behind them, 2 of 15 came back as rework; since then, 0 of 10, with twelve more too recent to judge. Those are small numbers, and all they say is that nothing jumped.

When approvals happen is a different story. Six of the 23 items finished since Aug 17 were approved outside the working day, against zero of 16 before. Every one of them touched the shared contracts between services. That isn’t proof of anything, since people work odd hours for many reasons, but it’s worth a conversation.

possible explanations
  • Better review practices, or smaller pull requests that are simply easier to review. (The history doesn’t record pull-request size, so I can’t check that one from here.)
  • AI-assisted review doing real work.
  • Less rigor: reviews that are faster because they look at less.
  • Someone quietly making up the difference by hand.

  • measured work-item history

    Rework within 14 days of done: 2 of 15 items before Aug 17, 0 of 10 since (twelve pending). After-hours approvals: 0 of 16 before, 6 of 23 since.

  • measured work-item history

    Median time in review since Aug 17: 0.4 days for shared-contract changes (6), 0.2 days for everything else (12). Both look fast.

  • reported one-on-ones, simulated

    One engineer tells me, after some hesitation, that they’ve been checking every shared-contract change by hand in the evening before approving it, because the AI review misses breaking changes between services. They hadn’t raised it because the new numbers looked so good.

  • reported team retrospective, simulated

    The rest of the team likes the faster feedback, and says the AI review catches real problems in most changes.

  • hypothesis my read

    The AI review is genuinely faster for most work. For shared contracts, quality is holding because of one person’s unpaid evenings, not because of the process.

03 / decide

Keep the improvement. Stop the hidden work.

The decision marker sits at Sep 7.

Where the evidence says AI-assisted review is working, I’m comfortable keeping it. We haven’t observed a deterioration in the measured rework, and most of the team says it saves them real effort.

The evening checks stop, starting now. I thank the engineer for telling me, and I’m careful that it lands as a contribution, not a confession, because the next person will watch what happens to them. A process that only looks efficient because someone is quietly working extra hours isn’t efficient.

The weakness they found is real, so we fix it in the open. Shared-contract changes get a human review during the day, alongside the AI review, and the team writes tests for the kinds of contract breaks the engineer kept catching, so the check lives in the pipeline rather than in one person.

the tradeoffs
  • Shared-contract reviews will take longer than they do now. The honest comparison is with the old queue, not with the hidden version.
  • Some of the speed we saw was never real. I’d rather know that than keep a number that depends on someone’s evenings.
  • If rework rises once the evening checks stop, we look again, rather than defending the decision.

  • decision Engineering

    Keep AI-assisted review for everything. Add a daytime human review for shared-contract changes.

  • decision Engineering

    No more evening checks. The team owns contract tests for the breaks that were being caught by hand.

04 / validate

Is it still better without the hidden effort?

The full chart, Aug 3 to Oct 11. After the decision, in review held 2, 2, 1, 2 and 2 items at week ends.

Mostly, yes. Shared-contract changes now take a median of 1.5 days in review, slower than the evening version but far from the 5.1 days reviews took before AI assistance. Everything else still clears in under a day: a median of 0.5 days elapsed, a little above the 0.2 days measured in early September because a review that starts late in the afternoon now and then finishes the next morning. The review band is a little wider than it was in late August, at 2, 2, 1, 2 and 2 items at week ends, and that’s the price of reviews that are real.

None of the 28 items finished since the decision was approved after hours. Rework within two weeks of done: 2 of 15 before AI review, 1 of 22 with it, and 0 of 15 since the decision, with twelve items too recent to judge. Throughput held at 4, 6, 6, 5 and 7 a week.

Five weeks after the decision, only the 15 items finished early enough to have two full weeks of follow-up can be judged for rework, and twelve more are still too recent. A steady rate on that few items is a promising sign, not proof, and defects in shared contracts can take months to surface. I’ll keep watching rework, and I’ll keep asking the team, including the engineer who raised it, how the process feels.

  • measured work-item history

    Median time in review since Sep 7: 1.5 days for shared-contract changes (8 items), 0.5 days for the rest (20). Before AI review: 5.1 days (21).

    time in review of each finished item, by the day it finished

    • shared-contract changes
    • everything else

    y: elapsed calendar days in review; x: the day each item finished

    values (63 items)
    • B-005 · accounts · done Aug 3 · 4.9 days in review
    • B-007 · accounts · done Aug 4 · 5 days in review
    • B-009 · shared contracts · done Aug 4 · 4.2 days in review
    • B-006 · shared contracts · done Aug 6 · 6.1 days in review
    • B-008 · checkout · done Aug 7 · 6.8 days in review
    • B-010 · checkout · done Aug 10 · 5.7 days in review
    • B-011 · accounts · done Aug 10 · 5.8 days in review
    • B-016 · checkout · rework of B-004 · done Aug 12 · 6.8 days in review
    • B-012 · shared contracts · done Aug 12 · 6.1 days in review
    • B-014 · checkout · done Aug 12 · 5.3 days in review
    • B-013 · accounts · done Aug 13 · 6 days in review
    • B-015 · shared contracts · done Aug 13 · 2.4 days in review
    • B-017 · checkout · done Aug 17 · 6.8 days in review
    • B-025 · accounts · rework of B-011 · done Aug 17 · 0.2 days in review
    • B-019 · shared contracts · done Aug 18 · 4.8 days in review
    • B-021 · checkout · done Aug 18 · 5.1 days in review
    • B-020 · accounts · done Aug 18 · 4.1 days in review
    • B-023 · checkout · done Aug 18 · 0.1 days in review
    • B-022 · shared contracts · done Aug 18 · 0.4 days in review
    • B-018 · accounts · done Aug 19 · 6 days in review
    • B-024 · accounts · done Aug 20 · 0.2 days in review
    • B-027 · accounts · done Aug 21 · 0.1 days in review
    • B-026 · shared contracts · done Aug 21 · 0.4 days in review
    • B-028 · checkout · done Aug 25 · 0.8 days in review
    • B-030 · checkout · done Aug 25 · 0.1 days in review
    • B-029 · shared contracts · done Aug 26 · 1.2 days in review
    • B-033 · accounts · done Aug 28 · 0.8 days in review
    • B-031 · accounts · done Aug 28 · 0.2 days in review
    • B-032 · shared contracts · done Aug 28 · 0.4 days in review
    • B-035 · shared contracts · done Aug 31 · 0.4 days in review
    • B-034 · checkout · done Sep 1 · 0.8 days in review
    • B-038 · shared contracts · done Sep 3 · 0.3 days in review
    • B-036 · checkout · done Sep 4 · 0.8 days in review
    • B-037 · accounts · done Sep 4 · 0.1 days in review
    • B-040 · checkout · done Sep 4 · 0.9 days in review
    • B-039 · accounts · done Sep 8 · 0.8 days in review
    • B-043 · accounts · done Sep 8 · 0.1 days in review
    • B-042 · checkout · done Sep 10 · 0.1 days in review
    • B-041 · shared contracts · done Sep 11 · 1.2 days in review
    • B-045 · accounts · done Sep 14 · 2.8 days in review
    • B-049 · checkout · rework of B-034 · done Sep 14 · 0.1 days in review
    • B-044 · shared contracts · done Sep 15 · 4 days in review
    • B-046 · checkout · done Sep 16 · 0.8 days in review
    • B-048 · checkout · done Sep 18 · 0.9 days in review
    • B-047 · shared contracts · done Sep 18 · 1.2 days in review
    • B-050 · accounts · done Sep 21 · 2.8 days in review
    • B-052 · accounts · done Sep 21 · 2.9 days in review
    • B-051 · shared contracts · done Sep 22 · 1.2 days in review
    • B-053 · checkout · done Sep 22 · 0.1 days in review
    • B-054 · shared contracts · done Sep 24 · 1.8 days in review
    • B-056 · accounts · done Sep 25 · 0.8 days in review
    • B-055 · checkout · done Sep 28 · 2.9 days in review
    • B-059 · checkout · done Sep 29 · 0.1 days in review
    • B-057 · shared contracts · done Sep 29 · 1.1 days in review
    • B-058 · accounts · done Sep 30 · 0.8 days in review
    • B-061 · checkout · done Sep 30 · 0.1 days in review
    • B-060 · shared contracts · done Oct 5 · 4 days in review
    • B-062 · accounts · done Oct 5 · 0.2 days in review
    • B-064 · accounts · done Oct 5 · 0.2 days in review
    • B-063 · shared contracts · done Oct 6 · 3.8 days in review
    • B-065 · checkout · done Oct 6 · 0.1 days in review
    • B-068 · accounts · done Oct 9 · 0.8 days in review
    • B-067 · checkout · done Oct 9 · 0.2 days in review

    Hover, tap or focus a mark for its values.

  • measured work-item history

    Rework within two weeks of done: 2 of 15 (before AI review), 1 of 22 (AI review, evening checks), 0 of 15 (since the decision; 12 pending). After-hours approvals since the decision: 0 of 28.

  • reported team retrospective, simulated

    The engineer who had been checking contracts in the evening says their weeks feel normal again. The team says shared-contract reviews are slower now, but nobody is carrying them alone.

the data behind scenario b (all weeks)

75 simulated work items. The chart leaves out the 4 finished before Aug 3, so its bottom band counts work finished since then, and the bands always add up to the items in scope. Counts are taken at the end of each day, UTC; the team’s working day is 09:00–17:30 UTC. Download the full history as JSON.

Scenario B: items in each state at the end of each Sunday, UTC
week ending readyin progressin reviewdone admitted
Aug 9 1365 15
Aug 16 14512 22
Aug 23 23023 28
Aug 30 23029 34
Sep 6 14035 40
Sep 13 33239 47
Sep 20 33245 53
Sep 27 34151 59
Oct 4 34256 65
Oct 11 24263 71
Scenario B: every work item's status entries, UTC
itemattributesstatus entries, UTC
B-001 feature · accounts · finished before the chart starts ready Jul 20 09:02 → in progress Jul 20 09:25 → in review Jul 24 16:37 → done Jul 29 15:45
B-002 feature · checkout · finished before the chart starts ready Jul 20 09:06 → in progress Jul 20 09:28 → in review Jul 24 15:21 → done Jul 29 16:19
B-003 feature · shared contracts · finished before the chart starts ready Jul 21 09:02 → in progress Jul 21 09:25 → in review Jul 27 16:34 → done Jul 31 15:49
B-004 feature · checkout · finished before the chart starts ready Jul 22 09:02 → in progress Jul 22 09:25 → in review Jul 27 09:44 → done Jul 30 14:48
B-005 feature · accounts ready Jul 23 09:02 → in progress Jul 23 09:25 → in review Jul 29 12:37 → done Aug 3 09:28
B-006 feature · shared contracts ready Jul 24 09:02 → in progress Jul 27 09:25 → in review Jul 31 14:12 → done Aug 6 16:14
B-007 feature · accounts ready Jul 27 09:02 → in progress Jul 27 09:28 → in review Jul 30 14:00 → done Aug 4 14:48
B-008 feature · checkout ready Jul 27 09:06 → in progress Jul 28 09:25 → in review Jul 31 15:04 → done Aug 7 10:45
B-009 feature · shared contracts ready Jul 28 09:02 → in progress Jul 28 09:28 → in review Jul 31 11:40 → done Aug 4 17:20
B-010 feature · checkout ready Jul 29 09:02 → in progress Jul 30 09:25 → in review Aug 4 15:23 → done Aug 10 09:07
B-011 feature · accounts ready Jul 30 09:02 → in progress Jul 31 09:25 → in review Aug 4 15:43 → done Aug 10 09:47
B-012 feature · shared contracts ready Jul 31 09:02 → in progress Aug 3 09:25 → in review Aug 6 13:24 → done Aug 12 16:35
B-013 feature · accounts ready Aug 3 09:02 → in progress Aug 3 09:28 → in review Aug 7 10:31 → done Aug 13 10:02
B-014 feature · checkout ready Aug 3 09:06 → in progress Aug 3 09:31 → in review Aug 7 09:50 → done Aug 12 17:04
B-015 feature · shared contracts ready Aug 4 09:02 → in progress Aug 5 09:28 → in review Aug 11 09:02 → done Aug 13 17:27
B-016 bug · checkout · normal · rework of B-004 ready Aug 5 10:41 → in progress Aug 5 10:42 → in review Aug 5 16:52 → done Aug 12 10:54
B-017 feature · checkout ready Aug 5 09:02 → in progress Aug 6 09:25 → in review Aug 10 17:23 → done Aug 17 11:41
B-018 feature · accounts ready Aug 6 09:02 → in progress Aug 7 09:25 → in review Aug 13 13:53 → done Aug 19 13:09
B-019 feature · shared contracts ready Aug 7 09:02 → in progress Aug 10 09:25 → in review Aug 13 13:35 → done Aug 18 09:32
B-020 feature · accounts ready Aug 10 09:02 → in progress Aug 10 09:28 → in review Aug 14 13:09 → done Aug 18 15:54
B-021 feature · checkout ready Aug 10 09:06 → in progress Aug 11 09:25 → in review Aug 13 13:22 → done Aug 18 15:47
B-022 feature · shared contracts ready Aug 11 09:02 → in progress Aug 12 09:25 → in review Aug 18 12:03 → done Aug 18 21:26
B-023 feature · checkout ready Aug 12 09:02 → in progress Aug 14 09:28 → in review Aug 18 14:11 → done Aug 18 16:17
B-024 feature · accounts ready Aug 13 09:02 → in progress Aug 14 09:31 → in review Aug 20 10:46 → done Aug 20 14:34
B-025 bug · accounts · normal · rework of B-011 ready Aug 14 09:50 → in progress Aug 14 09:51 → in review Aug 17 11:51 → done Aug 17 16:48
B-026 feature · shared contracts ready Aug 14 09:02 → in progress Aug 17 09:25 → in review Aug 21 12:48 → done Aug 21 22:16
B-027 feature · accounts ready Aug 17 09:02 → in progress Aug 18 09:25 → in review Aug 21 14:35 → done Aug 21 16:49
B-028 feature · checkout ready Aug 17 09:06 → in progress Aug 19 09:25 → in review Aug 24 16:50 → done Aug 25 11:56
B-029 feature · shared contracts ready Aug 18 09:02 → in progress Aug 19 09:28 → in review Aug 25 16:06 → done Aug 26 21:23
B-030 feature · checkout ready Aug 19 09:02 → in progress Aug 21 09:25 → in review Aug 25 13:52 → done Aug 25 17:12
B-031 feature · accounts ready Aug 20 09:02 → in progress Aug 24 09:25 → in review Aug 28 11:52 → done Aug 28 15:30
B-032 feature · shared contracts ready Aug 21 09:02 → in progress Aug 24 09:28 → in review Aug 28 11:26 → done Aug 28 20:01
B-033 feature · accounts ready Aug 24 09:02 → in progress Aug 25 09:25 → in review Aug 27 15:52 → done Aug 28 11:35
B-034 feature · checkout ready Aug 24 09:06 → in progress Aug 26 09:25 → in review Aug 31 17:08 → done Sep 1 12:50
B-035 feature · shared contracts ready Aug 25 09:02 → in progress Aug 26 09:28 → in review Aug 31 10:34 → done Aug 31 20:27
B-036 feature · checkout ready Aug 26 09:02 → in progress Aug 28 09:25 → in review Sep 3 16:24 → done Sep 4 12:03
B-037 feature · accounts ready Aug 27 09:02 → in progress Aug 31 09:25 → in review Sep 4 10:15 → done Sep 4 13:13
B-038 feature · shared contracts ready Aug 28 09:02 → in progress Aug 31 09:28 → in review Sep 3 13:35 → done Sep 3 21:09
B-039 feature · accounts ready Aug 31 09:02 → in progress Sep 1 09:25 → in review Sep 7 16:45 → done Sep 8 12:06
B-040 feature · checkout ready Aug 31 09:06 → in progress Sep 1 09:28 → in review Sep 3 16:49 → done Sep 4 13:14
B-041 feature · shared contracts ready Sep 1 09:02 → in progress Sep 4 09:25 → in review Sep 10 11:13 → done Sep 11 15:33
B-042 feature · checkout ready Sep 2 09:02 → in progress Sep 4 09:28 → in review Sep 10 13:44 → done Sep 10 16:34
B-043 feature · accounts ready Sep 3 09:02 → in progress Sep 4 09:31 → in review Sep 8 14:52 → done Sep 8 17:14
B-044 feature · shared contracts ready Sep 4 09:02 → in progress Sep 7 09:25 → in review Sep 11 11:20 → done Sep 15 11:05
B-045 feature · accounts ready Sep 7 09:02 → in progress Sep 8 09:25 → in review Sep 11 14:13 → done Sep 14 09:35
B-046 feature · checkout ready Sep 7 09:06 → in progress Sep 9 09:25 → in review Sep 15 14:52 → done Sep 16 09:00
B-047 feature · shared contracts ready Sep 8 09:02 → in progress Sep 11 09:28 → in review Sep 17 09:55 → done Sep 18 14:36
B-048 feature · checkout ready Sep 9 09:02 → in progress Sep 14 09:25 → in review Sep 17 17:02 → done Sep 18 13:31
B-049 bug · checkout · normal · rework of B-034 ready Sep 10 14:05 → in progress Sep 11 09:25 → in review Sep 14 12:19 → done Sep 14 15:01
B-050 feature · accounts ready Sep 10 09:02 → in progress Sep 14 09:28 → in review Sep 18 16:15 → done Sep 21 10:30
B-051 feature · shared contracts ready Sep 11 09:02 → in progress Sep 15 09:25 → in review Sep 21 09:27 → done Sep 22 14:41
B-052 feature · accounts ready Sep 14 09:02 → in progress Sep 16 09:25 → in review Sep 18 14:12 → done Sep 21 10:40
B-053 feature · checkout ready Sep 14 09:06 → in progress Sep 18 09:25 → in review Sep 22 15:49 → done Sep 22 17:23
B-054 feature · shared contracts ready Sep 15 09:02 → in progress Sep 18 09:28 → in review Sep 22 17:06 → done Sep 24 11:28
B-055 feature · checkout ready Sep 16 09:02 → in progress Sep 21 09:25 → in review Sep 25 14:56 → done Sep 28 11:20
B-056 feature · accounts ready Sep 17 09:02 → in progress Sep 21 09:28 → in review Sep 24 17:22 → done Sep 25 11:58
B-057 feature · shared contracts ready Sep 18 09:02 → in progress Sep 22 09:25 → in review Sep 28 10:35 → done Sep 29 14:00
B-058 feature · accounts ready Sep 21 09:02 → in progress Sep 23 09:25 → in review Sep 29 16:10 → done Sep 30 10:31
B-059 feature · checkout ready Sep 21 09:06 → in progress Sep 23 09:28 → in review Sep 29 09:01 → done Sep 29 12:07
B-060 feature · shared contracts ready Sep 22 09:02 → in progress Sep 25 09:25 → in review Oct 1 09:44 → done Oct 5 09:42
B-061 feature · checkout ready Sep 23 09:02 → in progress Sep 28 09:25 → in review Sep 30 13:17 → done Sep 30 14:48
B-062 feature · accounts ready Sep 24 09:02 → in progress Sep 29 09:25 → in review Oct 5 10:22 → done Oct 5 14:08
B-063 feature · shared contracts ready Sep 25 09:02 → in progress Sep 30 09:25 → in review Oct 2 15:00 → done Oct 6 11:00
B-064 feature · accounts ready Sep 28 09:02 → in progress Sep 30 09:28 → in review Oct 5 11:14 → done Oct 5 15:47
B-065 feature · checkout ready Sep 28 09:06 → in progress Oct 1 09:25 → in review Oct 6 12:56 → done Oct 6 14:39
B-066 feature · shared contracts ready Sep 29 09:02 → in progress Oct 2 09:25 → in review Oct 8 16:03
B-067 feature · checkout ready Sep 30 09:02 → in progress Oct 5 09:25 → in review Oct 9 11:34 → done Oct 9 16:26
B-068 feature · accounts ready Oct 1 09:02 → in progress Oct 6 09:25 → in review Oct 8 14:36 → done Oct 9 10:30
B-069 feature · shared contracts ready Oct 2 09:02 → in progress Oct 6 09:28 → in review Oct 8 17:08
B-070 feature · accounts ready Oct 5 09:02 → in progress Oct 7 09:25
B-071 feature · checkout ready Oct 5 09:06 → in progress Oct 9 09:25
B-072 feature · shared contracts ready Oct 6 09:02 → in progress Oct 9 09:28
B-073 feature · checkout ready Oct 7 09:02 → in progress Oct 9 09:31
B-074 feature · accounts ready Oct 8 09:02
B-075 feature · shared contracts ready Oct 9 09:02

// scenario c — quality tradeoffs

Healthy throughput can hide unhealthy tradeoffs.

Cumulative flow diagram for scenario C, Aug 3 to Oct 11: ready, in progress, blocked, in review, done, stacked by the number of items in each state at the end of each day.

  • admitted (top edge)
  • ready
  • in progress
  • blocked
  • in review
  • done

Hover, tap or focus the chart to read any visible day.

  1. Aug 10 launch push: features first context
  2. Aug 26 open bugs reach ten measured
  3. Sep 7 one shared backlog order; minor internal bugs deferred decision
  4. Sep 10 export refactor done; blocked bugs move again measured
Phase of scenario C

01 / observe

Delivery looks healthy.

The chart shows Aug 3 to Aug 30. The ready band widened as bugs waited; open bugs rose to 11.

Since the launch push began on Aug 10, feature delivery is up: 4 features finished the week before, then 6, 5 and 8. Median cycle time moved only a little, from 3.7 days for items finished in the first two weeks to 4.3 days in the last two. By the usual numbers, this team is doing well.

The ready band tells a different story: it’s widening, and bugs explain most of it. Eight of the 11 items waiting in Ready are bugs; the rest are features waiting their turn. Open bugs at the end of each week went 3, 5, 8 and 11, while the team finished 3, 1, 0 and 0 bugs. The oldest bug nobody has started has waited 19 days. Two more were started and then blocked, and have sat there longer still.

Cycle time can’t see this. It only measures work that has finished, and most of these bugs haven’t even started. Healthy throughput can sit on top of an unhealthy tradeoff, and the only way to tell is to look at what isn’t moving.

  • measured work-item history

    Features finished per week: 4, 6, 5 and 8. Bugs finished: 3, 1, 0 and 0.

    features finished per week, with bugs finished

    • features finished
    • bugs finished
    • features finished, weeks in focus

    Hover, tap or focus a mark for its values.

  • measured work-item history

    Open bugs at the end of each week: 3, 5, 8 and 11. Median cycle time of finished items: 3.7 days (14 items), then 4.3 days (13).

    open bugs at the end of each week, with customer-facing ones

    • open bugs
    • customer-facing
    • open bugs, weeks in focus

    Hover, tap or focus a mark for its values.

  • hypothesis open question

    A growing bug list could be a deliberate tradeoff, a blind spot, or a sign of something harder underneath. The count alone doesn’t say which.

02 / investigate

Which of these bugs are a decision, and which are a risk?

The chart now runs to Sep 6. Open bugs: 13, of which 6 are customer-facing and 2 blocked.

I don’t treat 13 open bugs as one number. A minor internal bug we chose to leave is a decision. A customer-facing bug nobody has looked at in weeks is a risk. The history records severity and whether customers are affected, so I can separate them.

Six of the open bugs affect customers, and the oldest of those has been open 47 days. Eleven are waiting in Ready without being started. Two are blocked, both on the export pipeline, and have been since Jul 23 and Jul 30 respectively: that isn’t a prioritization problem, it’s a technical one.

I prefer handling bugs continuously, as part of the normal flow of work, rather than saving them for a cleanup sprint. Kanban makes that natural, as long as the order of the backlog reflects what actually matters. Right now it reflects the launch and nothing else.

  • measured work-item history

    Open bugs by impact, Aug 3 and Sep 6. Eleven waiting in Ready, two blocked, none in progress or review.

    open bugs, by customer impact and severity

    workstream Aug 3Sep 6
    customer-facing, major or critical 3 4
    customer-facing, normal or minor 0 2
    internal, normal or worse 1 2
    internal, minor 0 5

  • measured work-item history

    Bugs started during the push that weren’t critical: C-024 (Aug 26).

  • reported Product, simulated

    Product says the launch features were the priority, and nobody decided to leave customer-facing bugs behind. It happened by default, because the ordering only said “features first.” Support has escalated the export problem twice.

  • reported one-on-ones, simulated

    One engineer says they started a customer-facing bug on their own because leaving it felt wrong. I appreciate the instinct, and I tell them so. I also ask them to raise it next time, because changing the team’s priorities should be a shared decision, not a quiet one.

03 / decide

Order the backlog together.

The decision marker sits at Sep 7.

Product and Engineering sit down and order the backlog together. Customer-facing major and critical bugs go to the top. Features and the other customer-facing bugs follow in the order they arrived. Minor internal bugs are deliberately deferred, written down as deferred, with a date to look at them again.

The two blocked bugs need a different fix. A small refactor of the export pipeline goes first, because until it lands, nobody can fix them no matter how high they sit in the list.

Engineers keep pulling the highest-priority ready item, and they keep the autonomy to say when the order looks wrong. What changes is that the order now says what we actually believe, so nobody has to quietly work around it.

the tradeoffs
  • Feature delivery will slow. Product chooses which launch features move, knowing why.
  • Deferred bugs are still bugs. Deferring them is a bet that they stay low-impact, which is why they get a review date instead of a shrug.
  • Not every defect needs fixing right away. The point is that the ones we leave, we leave on purpose.

  • decision Product and Engineering

    One ordered backlog: refactor first, then customer-facing major and critical bugs, then features and other bugs by arrival. Minor internal bugs deferred, with a review date.

  • decision Product

    Feature intake returns to the pace before the push. Launch scope gets re-planned against that pace.

04 / validate

Is the risk going down, at a pace we can keep?

The full chart, Aug 3 to Oct 11. Customer-facing open bugs fell from 6 to 1; blocked bugs cleared after the refactor.

The risky part of the backlog came down. Customer-facing open bugs went from 6 to 1, and the oldest of them is now 5 days old, down from 47 days. Five major or critical customer-facing bugs were fixed after the decision. The export refactor finished on Sep 10, and two of the blocked bugs were fixed after it.

The total didn’t come down: open bugs rose from 13 to 15, at 12, 11, 13, 13 and 15 across the week ends. That’s mostly the deferred minor internal bugs, 5 before the decision and 11 now. Deferring them was deliberate, but severity and customer-facing labels alone can’t show that risk isn’t accumulating, so we’ll reassess their impact at the review date.

Features kept moving at 2, 4, 7, 4 and 6 a week, down from 6, 5, 8 and 8 during the push. That’s the honest cost of the change, and Product chose it. Median cycle time for items finished in the last three weeks is 4.2 days, about where it was all along, which is a reminder that cycle time was never the number that showed the problem.

  • measured work-item history

    Open bugs on Oct 11 by age: 3 under a week, 2 one to two weeks, 3 two to four weeks, 7 older, every one of the oldest a minor internal bug. An open bug is a bug reported and not yet done, whatever its state; its age counts from when it was reported.

    open bugs, by severity

    severity open Sep 6open Oct 11customer-facingmedian ageoldest
    critical 000——
    major 400——
    normal 4416d10d
    minor 511033d59d
    all open bugs 1315126d59d

  • measured work-item history

    Features finished per week since the decision: 2, 4, 7, 4 and 6.

    features finished per week, with bugs finished

    • features finished
    • bugs finished
    • features finished, weeks in focus

    Hover, tap or focus a mark for its values.

  • reported team retrospective, simulated

    Support reports no new escalations on export. Engineers say it’s clearer what to pull next, and the engineer who had been working around the order says they’d rather argue about the list than ignore it.

the data behind scenario c (all weeks)

101 simulated work items. The chart leaves out the 9 finished before Aug 3, so its bottom band counts work finished since then, and the bands always add up to the items in scope. Counts are taken at the end of each day, UTC; the team’s working day is 09:00–17:30 UTC. Download the full history as JSON.

Scenario C: items in each state at the end of each Sunday, UTC
week ending readyin progressblockedin reviewdone admitted
Aug 9 03207 12
Aug 16 342014 23
Aug 23 832219 34
Aug 30 1142127 45
Sep 6 1612136 56
Sep 13 1650043 64
Sep 20 1630151 71
Sep 27 1630059 78
Oct 4 1530166 85
Oct 11 1450073 92
Scenario C: every work item's status entries, UTC
itemattributesstatus entries, UTC
C-001 feature · fall launch · finished before the chart starts ready Jul 20 09:02 → in progress Jul 20 09:25 → in review Jul 23 09:47 → done Jul 23 16:40
C-002 bug · minor · internal · finished before the chart starts ready Jul 20 09:06 → in progress Jul 20 09:28 → in review Jul 22 13:58 → done Jul 23 12:46
C-003 feature · fall launch · finished before the chart starts ready Jul 21 09:02 → in progress Jul 21 09:25 → in review Jul 24 12:40 → done Jul 27 11:03
C-004 feature · fall launch · finished before the chart starts ready Jul 22 09:02 → in progress Jul 22 09:25 → in review Jul 24 16:41 → done Jul 27 13:23
C-005 bug · export pipeline · major · customer-facing ready Jul 22 09:06 → in progress Jul 22 09:28 → blocked Jul 23 11:00 → in progress Sep 11 09:15 → in review Sep 15 09:10 → done Sep 15 12:07
C-006 feature · fall launch · finished before the chart starts ready Jul 23 09:02 → in progress Jul 23 09:25 → in review Jul 28 09:38 → done Jul 28 12:09
C-007 bug · normal · internal · finished before the chart starts ready Jul 24 09:02 → in progress Jul 24 09:25 → in review Jul 28 17:02 → done Jul 29 13:36
C-008 feature · fall launch · finished before the chart starts ready Jul 27 09:02 → in progress Jul 27 09:25 → in review Jul 29 16:23 → done Jul 30 15:17
C-009 bug · minor · internal · finished before the chart starts ready Jul 27 09:06 → in progress Jul 27 09:28 → in review Jul 28 17:29 → done Jul 29 12:13
C-010 feature · fall launch · finished before the chart starts ready Jul 28 09:02 → in progress Jul 28 09:25 → in review Jul 30 15:03 → done Jul 31 13:39
C-011 feature · fall launch ready Jul 29 09:02 → in progress Jul 29 09:25 → in review Aug 3 13:37 → done Aug 3 15:58
C-012 bug · export pipeline · major · customer-facing ready Jul 29 09:06 → in progress Jul 29 09:28 → blocked Jul 30 11:00 → in progress Sep 11 09:16 → in review Sep 15 11:08 → done Sep 15 14:34
C-013 feature · fall launch ready Jul 30 09:02 → in progress Jul 30 09:25 → in review Aug 3 10:02 → done Aug 3 16:22
C-014 bug · major · customer-facing ready Jul 31 09:02 → in progress Jul 31 09:25 → in review Aug 4 13:16 → done Aug 4 17:12
C-015 feature · fall launch ready Aug 3 09:02 → in progress Aug 3 09:25 → in review Aug 5 09:09 → done Aug 5 14:47
C-016 bug · normal · internal ready Aug 3 09:06 → in progress Aug 3 09:28 → in review Aug 6 09:13 → done Aug 6 15:12
C-017 feature · fall launch ready Aug 4 09:02 → in progress Aug 4 09:25 → in review Aug 6 09:11 → done Aug 6 15:10
C-018 feature · fall launch ready Aug 5 09:02 → in progress Aug 5 09:25 → in review Aug 10 12:42 → done Aug 11 12:11
C-019 bug · minor · internal ready Aug 5 09:06 → in progress Aug 5 09:28 → in review Aug 7 10:32 → done Aug 7 13:28
C-020 feature · fall launch ready Aug 6 09:02 → in progress Aug 6 09:25 → in review Aug 10 09:23 → done Aug 10 17:14
C-021 bug · minor · customer-facing ready Aug 7 09:02 → in progress Aug 7 09:25 → in review Aug 12 12:58 → done Aug 13 09:13
C-022 feature · fall launch ready Aug 10 09:02 → in progress Aug 10 09:25 → in review Aug 13 12:08 → done Aug 14 10:09
C-023 feature · fall launch ready Aug 10 09:06 → in progress Aug 10 09:28 → in review Aug 13 13:27 → done Aug 13 16:15
C-024 bug · major · customer-facing ready Aug 10 09:10 → in progress Aug 26 10:30 → in review Aug 31 12:25 → done Aug 31 17:03
C-025 feature · fall launch ready Aug 11 09:02 → in progress Aug 11 09:25 → in review Aug 13 09:03 → done Aug 13 15:14
C-026 feature · fall launch ready Aug 11 09:06 → in progress Aug 11 09:28 → in review Aug 14 10:24 → done Aug 14 17:09
C-027 feature · fall launch ready Aug 12 09:02 → in progress Aug 13 09:25 → in review Aug 17 09:47 → done Aug 17 15:32
C-028 bug · normal · internal ready Aug 12 09:06 → in progress Sep 7 09:34 → in review Sep 10 13:34 → done Sep 11 11:42
C-029 feature · fall launch ready Aug 13 09:02 → in progress Aug 14 09:25 → in review Aug 19 11:51 → done Aug 19 16:33
C-030 feature · fall launch ready Aug 14 09:02 → in progress Aug 14 09:28 → in review Aug 18 10:28 → done Aug 18 14:33
C-031 feature · fall launch ready Aug 14 09:06 → in progress Aug 14 09:31 → in review Aug 19 12:12 → done Aug 19 15:24
C-032 bug · minor · internal ready Aug 14 09:10
C-033 feature · fall launch ready Aug 17 09:02 → in progress Aug 17 09:25 → in review Aug 20 15:08 → done Aug 21 11:22
C-034 feature · fall launch ready Aug 17 09:06 → in progress Aug 18 09:25 → in review Aug 21 16:23 → done Aug 24 14:39
C-035 bug · normal · customer-facing ready Aug 17 09:10 → in progress Sep 8 09:25 → in review Sep 10 12:39 → done Sep 11 10:58
C-036 feature · fall launch ready Aug 18 09:02 → in progress Aug 19 09:25 → in review Aug 21 12:28 → done Aug 24 11:26
C-037 feature · fall launch ready Aug 18 09:06 → in progress Aug 20 09:25 → in review Aug 24 09:49 → done Aug 24 13:15
C-038 feature · fall launch ready Aug 19 09:02 → in progress Aug 20 09:28 → in review Aug 24 12:42 → done Aug 25 10:21
C-039 bug · major · customer-facing ready Aug 19 09:06 → in progress Sep 7 09:28 → in review Sep 10 12:40 → done Sep 11 09:18
C-040 feature · fall launch ready Aug 20 09:02 → in progress Aug 21 09:25 → in review Aug 25 14:59 → done Aug 26 09:27
C-041 feature · fall launch ready Aug 21 09:02 → in progress Aug 24 09:25 → in review Aug 27 12:51 → done Aug 27 15:42
C-042 feature · fall launch ready Aug 21 09:06 → in progress Aug 24 09:28 → in review Aug 26 13:39 → done Aug 27 13:05
C-043 bug · normal · customer-facing ready Aug 21 09:10 → in progress Sep 10 09:25 → in review Sep 14 12:38 → done Sep 15 09:39
C-044 feature · fall launch ready Aug 24 09:02 → in progress Aug 25 09:25 → in review Aug 28 15:49 → done Aug 31 11:15
C-045 feature · fall launch ready Aug 24 09:06 → in progress Aug 25 09:28 → in review Aug 27 16:35 → done Aug 28 15:21
C-046 bug · minor · internal ready Aug 24 09:10
C-047 feature · fall launch ready Aug 25 09:02 → in progress Aug 27 09:25 → in review Aug 31 09:03 → done Aug 31 11:17
C-048 feature · fall launch ready Aug 25 09:06 → in progress Aug 28 09:25 → in review Sep 1 16:00 → done Sep 2 09:49
C-049 feature · fall launch ready Aug 26 09:02 → in progress Aug 28 09:28 → in review Sep 1 11:14 → done Sep 2 10:07
C-050 bug · normal · internal ready Aug 26 09:06 → in progress Sep 10 09:28 → in review Sep 14 15:54 → done Sep 15 11:30
C-051 feature · fall launch ready Aug 27 09:02 → in progress Aug 31 09:25 → in review Sep 2 16:05 → done Sep 3 12:52
C-052 feature · fall launch ready Aug 28 09:02 → in progress Sep 1 09:25 → in review Sep 4 12:21 → done Sep 4 15:42
C-053 feature · fall launch ready Aug 28 09:06 → in progress Sep 1 09:28 → in review Sep 4 15:48 → done Sep 7 12:03
C-054 bug · minor · internal ready Aug 28 09:10
C-055 feature · fall launch ready Aug 31 09:02 → in progress Sep 2 09:25 → in review Sep 4 13:12 → done Sep 4 16:04
C-056 feature · fall launch ready Aug 31 09:06 → in progress Sep 2 09:28 → in review Sep 4 09:20 → done Sep 4 13:50
C-057 bug · minor · internal ready Aug 31 09:10
C-058 feature · fall launch ready Sep 1 09:02 → in progress Sep 3 09:25 → in review Sep 7 13:55 → done Sep 8 12:25
C-059 feature · fall launch ready Sep 1 09:06 → in progress Sep 11 09:25 → in review Sep 15 16:29 → done Sep 16 15:18
C-060 feature · fall launch ready Sep 2 09:02 → in progress Sep 15 09:25 → in review Sep 17 10:52 → done Sep 17 15:48
C-061 bug · minor · internal ready Sep 2 09:06
C-062 feature · fall launch ready Sep 3 09:02 → in progress Sep 15 09:28 → in review Sep 17 12:05 → done Sep 17 14:07
C-063 feature · fall launch ready Sep 4 09:02 → in progress Sep 16 09:25 → in review Sep 18 13:02 → done Sep 21 10:58
C-064 feature · fall launch ready Sep 4 09:06 → in progress Sep 16 09:28 → in review Sep 21 16:56 → done Sep 22 11:42
C-065 bug · major · customer-facing ready Sep 4 09:10 → in progress Sep 7 09:31 → in review Sep 9 11:30 → done Sep 9 15:52
C-066 tech debt · export pipeline ready Sep 7 09:02 → in progress Sep 7 09:25 → in review Sep 9 17:29 → done Sep 10 11:09
C-067 feature · fall launch ready Sep 7 09:06 → in progress Sep 16 09:31 → in review Sep 18 10:26 → done Sep 18 17:20
C-068 bug · normal · customer-facing ready Sep 7 09:10 → in progress Sep 18 09:25 → in review Sep 22 13:52 → done Sep 22 16:25
C-069 feature · fall launch ready Sep 8 09:02 → in progress Sep 18 09:28 → in review Sep 22 11:35 → done Sep 23 09:52
C-070 feature · fall launch ready Sep 9 09:02 → in progress Sep 21 09:25 → in review Sep 23 10:35 → done Sep 23 14:32
C-071 bug · minor · internal ready Sep 9 09:06
C-072 feature · fall launch ready Sep 10 09:02 → in progress Sep 21 09:28 → in review Sep 23 10:21 → done Sep 23 12:36
C-073 bug · minor · internal ready Sep 11 09:02
C-074 feature · fall launch ready Sep 14 09:02 → in progress Sep 22 09:25 → in review Sep 25 09:02 → done Sep 25 12:00
C-075 bug · normal · customer-facing ready Sep 14 09:06 → in progress Sep 23 09:25 → in review Sep 28 16:31 → done Sep 29 12:23
C-076 feature · fall launch ready Sep 15 09:02 → in progress Sep 23 09:28 → in review Sep 25 11:35 → done Sep 25 16:22
C-077 feature · fall launch ready Sep 16 09:02 → in progress Sep 24 09:25 → in review Sep 28 15:30 → done Sep 29 12:19
C-078 bug · minor · internal ready Sep 16 09:06
C-079 feature · fall launch ready Sep 17 09:02 → in progress Sep 24 09:28 → in review Sep 29 15:44 → done Sep 30 14:31
C-080 bug · minor · internal ready Sep 18 09:02
C-081 feature · fall launch ready Sep 21 09:02 → in progress Sep 28 09:28 → in review Sep 30 14:53 → done Oct 1 11:50
C-082 bug · minor · internal ready Sep 21 09:06
C-083 feature · fall launch ready Sep 22 09:02 → in progress Sep 29 09:25 → in review Oct 2 13:15 → done Oct 5 11:00
C-084 feature · fall launch ready Sep 23 09:02 → in progress Sep 29 09:28 → in review Oct 1 15:01 → done Oct 2 13:07
C-085 bug · normal · internal ready Sep 23 09:06 → in progress Sep 30 09:25 → in review Oct 2 09:16 → done Oct 2 16:01
C-086 feature · fall launch ready Sep 24 09:02 → in progress Oct 1 09:25 → in review Oct 5 14:25 → done Oct 6 11:07
C-087 bug · major · customer-facing ready Sep 25 09:02 → in progress Sep 28 09:25 → in review Oct 1 14:47 → done Oct 2 10:41
C-088 feature · fall launch ready Sep 28 09:02 → in progress Oct 2 09:25 → in review Oct 6 17:12 → done Oct 7 16:23
C-089 bug · minor · customer-facing ready Sep 28 09:06 → in progress Oct 2 09:28 → in review Oct 7 16:50 → done Oct 8 11:16
C-090 feature · fall launch ready Sep 29 09:02 → in progress Oct 5 09:25 → in review Oct 7 14:55 → done Oct 8 10:19
C-091 feature · fall launch ready Sep 30 09:02 → in progress Oct 5 09:28 → in review Oct 8 13:46 → done Oct 9 10:18
C-092 bug · minor · internal ready Sep 30 09:06
C-093 feature · fall launch ready Oct 1 09:02 → in progress Oct 6 09:25 → in review Oct 8 15:19 → done Oct 9 11:57
C-094 bug · normal · internal ready Oct 2 09:02 → in progress Oct 7 09:25
C-095 feature · fall launch ready Oct 5 09:02 → in progress Oct 8 09:25
C-096 bug · normal · internal ready Oct 5 09:06 → in progress Oct 8 09:28
C-097 feature · fall launch ready Oct 6 09:02 → in progress Oct 9 09:25
C-098 feature · fall launch ready Oct 7 09:02 → in progress Oct 9 09:28
C-099 bug · normal · customer-facing ready Oct 7 09:06
C-100 feature · fall launch ready Oct 8 09:02
C-101 bug · normal · internal ready Oct 9 09:02
Simulated scenarios: invented work items produced by a small, documented simulation, so the charts follow from the events rather than being drawn. Lines join the counts at the end of each day: exact at every day’s end, smoothed in between. The conversations and Product input are authored scenario context, marked “reported.” None of this describes a team I’ve managed.
// investigating friction

The measures behind the investigation

When a flow chart raises a question, these are the measures I reach for next. None of them is a verdict on its own.

Throughput: is work moving predictably?

I count throughput in the Kanban sense: work items completed in a period. Not commits, not story points, not people on the team. I don’t have a magic number that means a team is healthy, because team size, item size and the nature of the work all matter.

What catches my attention is instability. A spike isn’t automatically a win, just as a dip isn’t automatically a failure. Both are invitations to understand what happened.

throughput, items per week
synthetic example
  • finished (entered done)
  • entered ready

Week of Aug 17: 11 finished while 6 entered, against a typical 6. In review went from 5 items to 0: a queue emptying, not a team suddenly working twice as hard.

Weeks run Monday 00:00 to Monday 00:00 UTC.

weekly counts
Items finished and admitted per week, Monday to Sunday UTC
weekfinishedentered ready
Aug 3 – Aug 957
Aug 10 – Aug 1677
Aug 17 – Aug 23116
Aug 24 – Aug 3066
Aug 31 – Sep 666
Sep 7 – Sep 1347
Sep 14 – Sep 2066
Sep 21 – Sep 2766
Sep 28 – Oct 456
Oct 5 – Oct 1176
Synthetic example: scenario B above, counted Monday to Monday UTC, rework items included.

I manage for healthy, predictable flow, not bursts of activity.

Cycle time: how long does work take once it starts?

Cycle time runs from the first time someone picks an item up to the moment it’s done, weekends included. I look at the team-wide median first, because it tells me what’s typical without letting one stubborn item drag the center around. The breakdown by kind of work and the outliers come second, as places to investigate.

It also has blind spots. The median only counts finished work, so an item that’s started but stuck stays out of it until it finishes; I watch how long open items have been in progress, their work-item age, alongside it. And work that hasn’t started has no cycle time at all, which is why scenario C could look healthy on this measure while bugs aged in the queue.

cycle time, days
synthetic example
median
4.5 days
85th percentile
8 days
completed items
10

Each dot is one finished item. Hover, tap or focus a dot for its dates.

Started but not finished on Sep 23, so not in the median: K (feature), 8 days so far.

by work type
Median cycle time by work type, from the same 10 completed items
typeitemsmediannote
feature 3 4 days small sample (under 5)
bug 4 6 days small sample (under 5)
tech debt 2 4 days small sample (under 5)
spike 1 5 days small sample (under 5)

With 10 items split 4 ways, every subgroup is too small to compare. These are leads for a conversation, not findings.

all 12 items, as of Sep 23 09:00 UTC
All 12 sample items with arrival, first start and done instants and their status as of Sep 23 09:00 UTC
itemtypearrivedstarteddonestatuscycle days
A feature Sep 1 09:00 UTC Sep 2 09:00 UTC Sep 5 09:00 UTC done 3
B bug Sep 1 09:00 UTC Sep 2 09:00 UTC Sep 6 09:00 UTC done 4
C spike Sep 2 09:00 UTC Sep 3 09:00 UTC Sep 8 09:00 UTC done 5
D tech debt Sep 2 09:00 UTC Sep 5 09:00 UTC Sep 11 09:00 UTC done 6
E bug Sep 4 09:00 UTC Sep 6 09:00 UTC Sep 9 09:00 UTC done 3
F feature Sep 5 09:00 UTC Sep 7 09:00 UTC Sep 12 09:00 UTC done 5
G feature Sep 6 09:00 UTC Sep 9 09:00 UTC Sep 13 09:00 UTC done 4
H bug Sep 8 09:00 UTC Sep 11 09:00 UTC Sep 20 09:00 UTC done 9
I tech debt Sep 9 09:00 UTC Sep 13 09:00 UTC Sep 15 09:00 UTC done 2
J bug Sep 10 09:00 UTC Sep 14 09:00 UTC Sep 22 09:00 UTC done 8
K feature Sep 12 09:00 UTC Sep 15 09:00 UTC — in progress —
L bug Sep 14 09:00 UTC — — ready —
Synthetic example: 12 invented items, 10 finished. The 85th percentile uses the nearest-rank method. Items still in progress are listed separately, never counted as finished; the table lists all 12 with one as-of time.

Time in status: where does work spend its time?

If items linger in review, testing, blocked, or another stage, we may have a coordination problem rather than a coding-speed problem. Two items can both take eight days, one in complex development and the other mostly in review. The same number calls for different questions.

Time in a status doesn’t say whether the item was waiting or being worked on. Five days in review could be a queue, one long review, or rounds of changes; telling those apart takes more evidence, like review timestamps or a conversation with the people involved.

time in status, days from first start to done
synthetic example
P-1 mostly development 8 days
P-2 mostly in review 8 days
J blocked a day, then back to development 8 days
  • in progress
  • blocked
  • in review
  • in test

Hover, tap or focus a segment for its exact interval.

intervals
Each status visit from first start to done, in UTC
itemstatusenteredleftdays
P-1 in progress Sep 2 09:00 UTC Sep 8 09:00 UTC 6
P-1 in review Sep 8 09:00 UTC Sep 9 09:00 UTC 1
P-1 in test Sep 9 09:00 UTC Sep 10 09:00 UTC 1
P-2 in progress Sep 2 09:00 UTC Sep 4 09:00 UTC 2
P-2 in review Sep 4 09:00 UTC Sep 9 09:00 UTC 5
P-2 in test Sep 9 09:00 UTC Sep 10 09:00 UTC 1
J in progress Sep 14 09:00 UTC Sep 15 09:00 UTC 1
J blocked Sep 15 09:00 UTC Sep 16 09:00 UTC 1
J in progress Sep 16 09:00 UTC Sep 19 09:00 UTC 3
J in review Sep 19 09:00 UTC Sep 22 09:00 UTC 3
Synthetic example. Three items, eight days each, built from their status entries; time in ready before work starts isn’t shown. Blocked is recorded as its own state, not inferred from a long review.
// quality & tradeoffs

Quality, customers and the plan

A bug count on its own tells me very little. I care about severity, customer impact, how long bugs have been open, whether arrivals are outpacing fixes, and what each one costs downstream. Quality work belongs in the normal, continuous prioritization of the backlog, ordered by Product and Engineering together.

An old bug isn’t automatically a problem. A minor bug we deliberately deferred is a decision; a bug nobody has looked at in weeks is a stuck one. The chart can show me the age. Only a conversation tells me which it is.

bugs opened and resolved
synthetic example
Bugs opened and resolved per week, UTC
weekopenedresolved
Aug 31 – Sep 6 2 1
Sep 7 – Sep 13 2 1
Sep 14 – Sep 20 1 1
Sep 21 – Sep 23 (partial) 0 1
  • one bug opened (entered ready)
  • one bug resolved (entered done)

Hover, tap or focus a square for the bug behind it.

open bug backlog

Open bugs by severity at two instants
as ofopenminornormalmajorcritical
Sep 16, end of day32010
Sep 23 09:00 UTC11000
  • L · minor · ready · open 9 days, counted from when it entered the backlog

5 bugs in the sample; 1 bug open on Sep 23.

Synthetic example: bug tickets from the same 12 items. The sample doesn’t record whether a bug was found before or after release, or its customer impact, so this isn’t an escaped-defect chart.

How items are counted on this page: each one counts once, when it reaches Done, and nothing reopens. Rework is a new item linked to the work it fixes, so it arrives and finishes on its own, which is why scenario B’s throughput includes rework. These samples have no duplicate or cancelled tickets; in real data I’d close those apart from resolutions and leave them out of both the resolved count and throughput, and a reopened ticket would go back into the open backlog and count as finished once, at its final Done, which is also where its cycle time ends.

Estimated vs actual: what did we learn?

An estimate helps people plan; it isn’t a promise or a character judgment. I measure it at the level of dated project milestones, not individual tickets, and against the original forecast rather than the latest revision: re-forecasting can make the numbers look more accurate without delivery getting any more predictable. With real projects I keep the original date, each revision, and any material change in scope.

For some of the projects I’ve led, a variance of roughly two weeks in either direction has felt like a normal planning range. That’s a personal rule of thumb, not a universal standard; the right tolerance depends on the project’s size, uncertainty, and commitments. When the gap is larger or repeats, I want to know what the estimate missed, and to use that to plan better rather than to punish anyone for being wrong.

estimated vs actual, calendar days
synthetic example
milestone a
milestone b
milestone c
milestone d
milestone e
milestone f

±14 days: my context-dependent tolerance for some projects, not a universal expectation

5 of 6 milestones shipped within 14 calendar days of the forecast.

milestone dates
Forecast and shipped dates per milestone; positive is late
milestoneforecastshippedcalendar days
milestone a2026-02-272026-02-20−7
milestone b2026-03-272026-04-03+7
milestone c2026-05-082026-05-18+10
milestone d2026-06-192026-07-17+28
milestone e2026-08-142026-08-12−2
milestone f2026-09-252026-09-250
Synthetic example: six invented milestones, each one’s original forecast (set when it was planned) against its ship date. Positive is late. A highlighted dot is a question about what the estimate missed.
// what the dashboard doesn’t show

Healthy metrics aren’t a healthy team

The numbers don’t tell me how the engineers feel about the way work is moving. In scenario B, the data looked better than before while one person was quietly working evenings to hold it together. The dashboard showed unusual approval hours; the conversation explained the effort behind them.

That’s where one-on-ones matter. I want to hear whether people feel blocked, exhausted, confused, supported, or confident, and I compare those conversations with what the flow suggests. The absence of complaints doesn’t prove anyone is doing well.

It only works if people trust they can tell me a process is failing. When someone exposes a weakness, the response has to be curiosity and thanks, not blame, or the next weakness stays hidden. Coaching works the same way: a pattern in the data is a reason to talk with someone, never a substitute for it.

Metrics also inform business decisions without making them. Engineering should shape those decisions, and material risks, particularly in regulated industries, may need compliance, risk, legal, or regulatory review rather than an informal sign-off.

Healthy metrics should reflect a healthy system, not extraordinary effort required to keep an unhealthy system running.

// in short

I use metrics to understand the system, conversations to understand the people, and judgment to decide what needs to change.

// more on how i work philosophyHow I lead engineering organizationsHow I approach engineering teams, difficult decisions, technical judgment, and accountability.read how i leadguideWorking with meMy preferences, tendencies, blind spots, and the systems I've learned to build around them.read my user's guide