Leadership Garden
Last quarter I nearly sent my manager a slide I’m still slightly embarrassed by.
It had three numbers on it. Merged pull requests across my org were up a little more than double year over year. Median cycle time had dropped by about a third. Almost every engineer was using a coding agent every day. I had the speaker notes written, and I was going to call it our best half for delivery in years.
Then I tried to write the fourth bullet, the one about what customers got out of all this. I looked at that empty line for most of an evening. (I cut the slide in the end, which is the only reason I can write this post without wincing.)
If you run managers who run teams, there’s a decent chance you have a slide like that open somewhere right now. I don’t think the fourth bullet is blank because our teams are doing bad work. Something about the way evidence moves through an org this year makes the first three bullets easy to write and the fourth one close to impossible, and I want to work out what that something is.
Paulo André’s essay Quick to Build, Slow to Know is the best thing I’ve read on this, and I’d send it to anyone who owns a roadmap. His word for it is overshoot. The org learns in days that it can produce more and in months whether it should have, and in that gap sensible people keep expanding. I have one quibble. In his telling, overshoot is something “the system” does, which is true and lets everyone inside it off a little easy.
I sit in the middle of that system, between the teams who see the work and the executives who make commitments on it. From here, the system is mostly people like me deciding which line makes it onto the slide. He closes by asking what kind of leader this needs, and I’ll try to answer from my chair.
Speed reports itself
Every AI tool we bought this year came with a dashboard. Acceptance rates, sessions per developer, lines suggested, PRs touched. Our engineering metrics platform added an “AI impact” tab within a quarter. I didn’t ask for any of it, and all of it updated daily.
Nothing in our stack updates daily on whether the work mattered.
Think about what would actually tell you. Did customers keep using the feature after the first week? Did support tickets about that flow go down? The integration someone built in an afternoon, does it still work now that the partner changed their API, and is the new service as cheap to run as we said? That evidence doesn’t land on a dashboard. Some of it surfaces as a hunch in a retro six months later. Some of it never surfaces, because the people who would have noticed moved teams.
So the evidence is lopsided before anyone decides anything. I don’t think most leaders consciously prefer the fast kind. A vendor just delivers it to their inbox every Monday, while the slow kind is a project somebody has to staff, usually somebody who already has a full week.
The overshoot climbs the org chart
The route overshoot takes through an engineering org is the reporting chain. Picture how it moves in an org of forty or fifty engineers. An engineering manager sees her team’s throughput jump and mentions it in her monthly update. She’s careful about it. She notes that review load on her two seniors has gone up and that she isn’t sure the extra PRs are the ones that matter. Her director pulls together five of those updates. The caveats don’t survive the merge. What survives is “throughput roughly doubled across the group.” The head of engineering turns that into a planning assumption: if we ship twice as much, next half’s roadmap can absorb more. The roadmap goes to the business with dates on it. Sales starts mentioning features to customers.
Nobody in that chain did anything wrong. Go back and read the manager’s original update, and her line about the two seniors is still the most important sentence in it. It just didn’t fit the template the director was filling in, which had one cell per team for a headline number. By the time the chain reaches the people who make external commitments, a fast signal has turned into a promise, and the doubt that came with it is sitting in a monthly update nobody above the director will open.
(I’ve been the director in that story, and I’ve been the head of engineering. The caveats are always the first thing to go. I’ve cut them myself, because the slide had room for one line.)
Some companies go one step further and wire the fast signal straight into incentives. Managers at JPMorgan reportedly track how often developers use AI coding tools and factor it into performance. I can see how a well-meaning finance partner gets there: the license line is big, and usage is the one number the vendor will happily hand you. But once usage counts in someone’s review, you’ve told the whole org which signal keeps score, and people will play to it. I would too.
Scarcity was doing your prioritization
I’ve sat in a lot of planning meetings where the most useful sentence anyone said was “that’s three engineers for six weeks.”
It was a bad mechanism, and I don’t miss those meetings. Estimates got padded by engineers who had been burned before, then haggled down by PMs who had already promised a date to someone, and the number everyone signed off on was a fiction both sides recognized. It also punished ideas for being expensive when it should have punished them for being wrong.
But it did one thing well. It put a price on yes. Somebody had to say out loud that an idea was worth six weeks, in front of the people who would spend those six weeks.
That price has mostly disappeared. A PM asks whether we can try something, and the honest answer is “sure, it’s an afternoon.” The answer is accurate. It also answers the wrong question, because the afternoon was never the expensive part.
Most of what a yes costs shows up after the code exists, usually as somebody’s on-call shift or somebody’s help article, a year after the person who said yes has moved on to other things. AI made none of that cheaper. There’s just more of it.
I saw this first with internal tooling, not product features. Over one quarter, one of my areas grew a handful of small services, scripts, and dashboards, each built by someone solving a real problem in an afternoon. I found out about most of them when an on-call engineer asked in Slack who owned the one paging her at 2am. Nobody answered for a day.
These days the question I ask in planning is who owns it in twelve months, and what would make us delete it. How long it takes barely comes up anymore. No owner means we don’t build it. An owner without an answer to the second part still gets built, but with a date on the calendar when we decide its fate.
Just Ship Faster fails
Stay SaaSy would tell me I’m overthinking this. In The Best Prioritization Is No Prioritization they argue that scoring frameworks are theater and nobody can prove they prioritized well, so you’re better off spending that time getting faster and letting shipping teach you. It’s a good post, aimed at a real disease, and it reads like it was written by someone who has sat through one RICE session too many.
I agree with more of that than I’d like to. Most scoring frameworks I’ve watched in practice were a way to dress up a decision someone had already made. I’ve also watched a team spend a month ranking a backlog that one week of building would have settled.
But the argument assumes that the lesson from shipping arrives before your next decision. If you change a pricing page on Monday and the conversion data is clear by Friday, speed really can replace judgment. You can choose badly, because you’ll find out before it costs much.
Now stretch the loop. You ship an integration that enterprise customers will roll out over two quarters. Or an internal platform that other teams will adopt whenever their own roadmaps allow, which in practice means sometime next year. You won’t know whether it worked for six months, maybe longer. Meanwhile you’ve made twenty more decisions, most of them built on the assumption that the first one was right. If the first one was wrong, you find out about all twenty-one together.
The rule I’ve landed on: speed can stand in for judgment only when time-to-know is shorter than the time between your decisions. Past that point, going faster mostly means more bets placed blind, and build time, the number we used to fight over in planning, becomes the least interesting thing about a proposal.
Time-to-know is a number
I used to treat “will we ever find out if this worked” as a philosophical question. I now treat it as something you can estimate, roughly, before you start.
For any bet worth a conversation, I want these written down before work starts:
- What would tell us this worked? I want a behavior, something like “the forty accounts that asked for this export use it at least twice a month.” “Engagement” doesn’t count.
- When is the earliest we could see that?
- Who is going to look, and is it already on their calendar?
The gap between the ship date and the answer to the second question is time-to-know. For a copy change, days. For a platform migration, or anything enterprise customers adopt on their own schedule, I’d bet on two quarters, and I’ve rarely lost that bet.
When we started writing this down, a surprising number of proposals couldn’t answer the first question at all. Those were exactly the ones that most needed a no, and under the old regime most of them would have died at the six-week estimate. We also found work where time-to-know was far longer than anyone had assumed. Nobody wanted to stack three follow-up bets on a foundation we wouldn’t be able to judge until spring, so we stopped doing it.
You can also shorten it, and right now I’d put more leadership attention there than anywhere else. The cheap version is instrumenting before you ship, so that finance asking how the feature is doing isn’t the first time anyone looks. The underrated version is shipping to five customers you can phone before shipping to five hundred you can’t. Much of what we used to call velocity work, I’d now call time-to-know work, and given the choice I’d fund it over another round of seat licenses.
Write down what is enough before you start
One of the questions Paulo ends on is how long the “enough” signal takes to show up. My worry is that most teams never decided what “enough” would look like in the first place. They decide afterward, once they’re invested, and nobody invested in a feature has ever looked at mediocre adoption numbers and concluded it should go. So write the kill criteria before the work starts, with a date attached. “If fewer than fifteen of the accounts that requested this use it by the end of February, we remove it.” Put the review on the calendar on the day you approve the work.
The harder part is cultural. Removing a feature feels like admitting failure, so teams avoid it, and the product slowly fills with things nobody would build today. I’ve started counting deletions as delivered results in quarterly reviews. People were suspicious of that at first, and a couple of managers asked if it was a trick to make the numbers look better. It wasn’t. A removal that was planned from the start is the slow signal doing its job.
Will Larson makes the case that quick, good, and durable decisions are what lets a team cash in on AI’s pace. I’d push on “durable”, which most people hear as “permanent”. A decision with a review date on it lasts longer in practice, because reversing it later is a calendar item, not a political fight.
What I report upward now
The version I eventually sent had the same throughput numbers on it. It also had a second column: for each major thing we shipped that half, the date we’d know whether it worked and what we’d be looking at. Three rows said January. One said “we don’t yet know how to measure this,” which was the most honest line in the deck.
My manager asked about that row more than any other. Good. That’s the conversation I wanted to be having.
I’ve kept a few habits from that quarter, and I’d push them on anyone in a similar seat:
- Pair every fast metric with its slow counterpart and the date it will be known. A throughput number on its own gets turned into a promise by someone two levels up.
- When your managers send caveats up, keep at least one when you summarize. If the template only has room for one line, change the template.
- Argue against putting AI usage into performance reviews. If the people above you insist, at least make sure they understand they’re rewarding the fast signal on purpose.
A column full of “pending” is not a comfortable thing to present when every peer in the room is reporting green. The alternative is committing next half’s roadmap on the strength of a number that measures typing.
The leader this needs
Paulo’s closing question was what kind of leader this system needs. My answer is less grand than I’d like: someone who can wait for the slow signal without pretending it already arrived.
In practice that’s fairly unglamorous. You’re the person in planning who keeps asking who owns this in a year, long after people have stopped finding the question charming. Your managers’ doubts survive your summaries. And when your boss asks whether the big bet worked, you can say “we’ll know in January” and then stop talking, without filling the silence with a projection you don’t believe. (That last one is the hardest for me. I’m working on it.)
This isn’t a case for slower teams, and I’d be annoyed if anyone used it as one. I want the speed. I just don’t want an org that takes how fast it builds as proof of how fast it learns, and then makes promises on the difference.
We got dramatically faster at building this year, and I can’t point to a single week we saved in finding out whether any of it mattered. That gap is most of the job now.