A book in progress
Trail Markers
Choosing Where AI Fits in Engineering
Provisional full-book preview · November 2026
Illustration based on the author’s trail photograph
This is a compact first draft of a longer book, not an approved publication. Its case material comes from identified published sources; the seven files in book/interview/ are desk research, not interviews conducted with practitioners for this book. The original interviews required by the drafting plan have not happened. Treat the questions and exercises as proposals, not findings about your organization. Numerical claims describe their cited settings; they are not promises about yours.
A note to the reader
You have probably been asked to make an AI decision with too little time and too much certainty in the room. Somebody has a demo, a spending proposal, a report about time saved, or a chart of code generated. None of these is a bad place to begin. None tells you whether customers received something useful, the company recovered its investment, the team can sustain the new workload, or an engineer still has room to learn and say no.
This book is for the person who must make that decision, and for the people who will live with it. It offers no universal ranking of tools and no target for the percentage of work that should involve AI. It offers a route: name the outcome and the people affected; find the actual constraint; compare an AI intervention with a credible non-AI option; set boundaries proportionate to harm; measure what happened; then keep, change, expand, or stop. When you lack a comparison, say so. When a result disappoints, learn from it rather than hiding it.
The nine competencies of an engineering organization provide a map, not a compulsory itinerary. Effective, efficient, innovative, and engaging are the goals in the competency guide; this book also makes reliability explicit so that a faster trip cannot be mistaken for a safe one. The four perspectives on value—customer, company, team, individual—tell us who arrived, who paid, and who may have been left behind.
How to use this draft
Read Part I for the decision lens, Part II for the constraint you actually face, and Part III for a pilot and measurement method. Part IV turns the method into a next-quarter choice. Chapter endings are prompts for discussion, not scores for judging individual engineers. Citations point to primary studies or named organizational accounts when available. A field experiment about completed tasks, a vendor's financial model, an employee's account, and an incident report are different kinds of evidence. The book will keep saying which is which because the decisions deserve that honesty.
Editorial status: All 20 chapters are provisional short-form drafts. Original reporting, deep chapter expansion, cross-chapter editing, expert checking, and author approval remain open.
About the author
Baiyun (Ben) Gao is an engineering practitioner with more than 20 years of professional experience in software engineering, enterprise architecture, cloud platforms, AI, and secure SaaS solutions. TOGAF-certified, he has led solution engineering and delivery across government, aerospace and defense, cybersecurity, satellite communications, broadcasting, and enterprise technology sectors. He focuses on translating architecture into measurable business value through scalable platforms, modernized delivery practices, and customer-centered innovation.
Part I
Define value and find the real constraint
1. Faster Is Not the Destination
On June 24, 2026, video podcast episodes that normally appeared on Spotify within minutes took hours to reach listeners. Creators reported that episodes were missing. Some tried uploading again, adding more work to the queue. Their problem was plain: they had published something for an audience, and the audience could not play it. The incident report does not say how many creators were affected or how long any one episode waited. It does say that some creators heard from their audiences before they heard from Spotify.
The delay had more than one cause. Transcoding had too little spare capacity for a surge in new content. A scheduled batch job was using some of that capacity to reprocess older episodes. A quality improvement had increased the processing needed for each video, and a scheduling bug was leaving roughly ten percent of available throughput unused. Internal alerts first fired at 13:30 UTC. Engineers stopped the batch job at 16:35; the first creator report arrived at 17:31; formal incident response began at 17:34. A scheduler fix went out that evening, more capacity came online after midnight, and the queues cleared at 01:02 on June 25. Spotify itself points to the roughly four hours between the first alerts and formal response as a gap it needs to close.
There is a limit to what this account tells us. It records a customer-facing delay and the operational steps taken to clear it; it does not provide per-episode queue times, engineer work logs, a financial loss, or an experiment isolating each cause. Nor was this an AI-caused incident. Its place at the beginning of a book about AI is deliberate. It shows what the organization owes people when it says work is going faster: an outcome they can actually use, not a tally of activity upstream.
The wrong end of the stopwatch
Engineering leaders have good reasons to care about speed. A shorter path to a safe change can let a team repair harm sooner or learn from customers while a choice is still reversible. The error is to start the stopwatch at the easiest point to measure and stop it before the intended result arrives. An AI license is an input. A generated patch or merged pull request is an output. A shorter release cycle is a delivery result. A creator's episode reaching listeners as expected is a customer outcome. These measures can form a chain, but the links are not automatic.
Consider what happens if the first link gets cheaper. A developer can now submit a change in half the time. A reviewer still needs to understand its purpose; tests must exercise its failure modes; another team may need to approve a deployment; someone must notice whether the customer can use the result. If submissions arrive faster than those downstream steps can process them, the work waits somewhere else. The author feels faster while the customer may not.
That is not an argument against AI. In three field experiments across 4,867 developers, researchers found an average 26.08 percent increase in completed tasks in the pooled analysis among developers given access to a coding assistant. Random assignment makes this stronger evidence for that task result in those settings than an enthusiasm survey. The experiments did not establish a 26 percent gain in customer benefit, profit, or every individual's output. We should value the task finding without asking it to answer a different question.
Salesforce describes the next link in the chain. In its engineering account, code volume rose by about 30 percent, pull requests often grew beyond 20 files and 1,000 changed lines, and review latency rose. Senior reviewers were switching among large changes; the company worried that the time spent on the biggest reviews was no longer enough for meaningful scrutiny. Salesforce reports the problem and its response, not a controlled estimate of AI's effect on customer outcomes or additional reviewer hours. Even so, the decision is concrete: making authoring cheaper had not made human attention abundant. A second AI tool for review might help, but only if it helps reviewers find consequential errors without turning their approval into a formality.
There is a more careful way to shorten a wait. In a separate Spotify workflow—releasing the main app, not processing podcast videos—engineers used time-series data to locate release delays. They found blocking bugs and App Store approval among the largest waits; manually advancing a ready release could add up to twelve hours when a stage finished outside working hours. A condition-checking release conductor reduced the average release cycle by about eight hours. Regression sign-off, build checks, crash thresholds, and external App Store review remained part of the process. The release account measures delivery time, not eight hours of additional customer use. The lesson is not to automate every gate. It is to distinguish a manual transition from a check that protects people.
Find the recipient before choosing the tool
For the podcast publishing problem, the recipient was a creator whose episode should have become playable to an audience. A useful starting measure is upload-to-playable time, segmented by new versus reprocessed episodes and by the affected creator population. Queue age and available capacity help explain that measure; a timely notice tells creators what is happening. These are proposed measures for investigating a similar workflow, not metrics Spotify reported as the result of an AI pilot.
The options are not confined to "more AI" and "no action." Spotify reports adding headroom, improving monitoring, and giving new content priority over background work. In a different organization, an AI-assisted alert summary might help a responder connect early signals to a growing queue. That is a hypothesis to test against a simpler threshold, better capacity planning, or a clearer escalation path. The first trial should ask whether the right person notices and acts sooner, then whether the customer-visible delay actually shrinks. Faster alert prose would not count if episodes still arrive hours late.
Now widen the view. A customer gains when the result arrives reliably. The company may protect trust and revenue, but this incident report supplies no figure for either. A team gains if it can detect and recover without quietly transferring more work to another rotation. An individual gains if they have the context and authority to act without becoming the permanent human patch for an unexamined system. Those are four distinct questions, not four ways to rename PR throughput. We will examine them in the next chapter.
A trail marker is useful at a junction because it identifies the destination and the hazard without dictating every footstep. For a leader considering AI, the first marker is the person and result that matter. The second is the wait or failure that blocks them. The third is the evidence that would show whether the chosen intervention helped without worsening another part of the journey.
At the junction: Pick a recent customer result that arrived late or failed after launch. Ask a product partner and the receiving engineering or operations team to trace it backward from first successful use. Mark active work, waiting, necessary safeguards, and missing timestamps. Which change would you try first: an AI intervention, a non-AI workflow fix, or better observation? What result would make you change your mind?
2. Four Perspectives on Value
Imagine four people assessing the same proposed rollout. A director points to more merged changes. A finance partner asks whether expenses fell or margins rose. A reviewer worries about a growing queue. A junior engineer wants to know whether the new workflow still teaches them to judge their own work. These are illustrative positions, not quotations or findings from one organization. They reveal four different units of value.
Begin with the customer. Can a person accomplish the intended job with less delay, confusion, or risk? The person buying a product, the person using it, and the person whose data it processes may have conflicting interests. Count successful tasks and harms in the relevant segments, not just aggregate activity. A feature used by many people may still fail those who need accessibility or a reliable recovery path.
The company needs sustainable return. It may gain contribution margin, reduce an actual expense, avoid a cost it would otherwise have incurred, reduce expected losses, or create capacity for worthwhile work. Do not call all five "savings." IBM's published ROI method distinguishes freed capacity from booked cost reduction and includes review and operating costs. Its reported return is a vendor-published model for its own product, not cash that another firm can budget before running a test.
The team needs a system it can keep operating. Throughput purchased by shifting verification to another team's nights and weekends is not an uncomplicated improvement. The Superhuman/AWS account reports more merged PRs among the same cohort of engineers while median PR cycle time stayed around 20 hours. That describes additional output and an unchanged median per-change delay; it does not tell us the return to customers or the whole company's profit. Look at end-to-end waits, rework, incident load, and who absorbs them.
The individual is not merely a unit of team capacity. Useful work, autonomy, pay and fair recognition, safety, and practice that builds judgment matter even when nobody can assign them a dollar value. Anthropic's employee research reports broader reach and faster exploration alongside worries about debugging practice and mentoring. Those are perceptions from a particular workforce, not a measured decline in skill. They are still reasons to ask a human question before imposing a usage quota.
The four perspectives need not all show immediate gains. An investment in a safer release may temporarily slow a team while protecting customers. The honest decision states who bears that cost, why it is justified, and when the trade-off will be reviewed. Some conditions—privacy, safety, fair treatment—are boundaries, not commodities to exchange for more generated work.
At the junction: Write one sentence for each perspective about a proposed AI use. For each sentence, write an observable indicator and one way it could mislead you. If the company sentence says "hours saved × salary," ask whether anyone's payroll actually changed or whether capacity was put to useful work.
3. Diagnose the Engineering System
Before prescribing a tool, follow a change. Choose a customer-facing request, a production defect, or an incident with a start and an end. When did somebody know what was needed? When was the first test written, the review requested, approval granted, release completed, and customer result checked? Who was waiting at each boundary? A team may be busy at every instant and still deliver slowly because no one owns the wait between teams.
The engineering competency framework gives nine places to look: delivery, quality and reliability, flow and developer experience, architecture, product focus, learning, people, collaboration, and security. These are not nine independent dials. An unclear customer promise makes planning unstable; an opaque architecture makes review slow; missing ownership prolongs an incident. Raise the competency that constrains the outcome, not the one most amenable to an AI demonstration.
Suppose the longest wait is a change-approval queue. Ask whether the delay is a protective control, an external dependency, or avoidable manual handoff. Removing a control without understanding its failure mode may shorten a chart and worsen the customer's experience. If the constraint is missing product clarity, generating implementation faster simply makes wrong work arrive earlier. If the constraint is poor documentation, retrieval can make stale answers easier to find; first establish current ownership and source-of-truth rules.
Do not assume the first bottleneck stays put. In its Hyperforce account, Salesforce describes faster local development alongside review and approval pressure, then changes to its surrounding workflow. Its before/after cycle-time figures also reflect approval-process improvements, so they cannot isolate AI's contribution. Follow arrivals, work-in-progress, service capacity, and queue age through all gates rather than attributing a whole-system gain to a single new tool.
The guide names effective, efficient, innovative, engaging as organizational goals. This book names reliable separately as an explicit countermeasure to rushing. A maturity level is a description of operating capability, not a target number of autonomous agents. An organization that can decline an unsafe automation may be more mature than one that deploys it everywhere.
Try the map: Draw one outcome's timeline on a page. Put a name at each handoff, a reason beside each safeguard, and a question mark where timestamps or customer evidence are missing. Interview the people who receive the work as well as those who generate it. Then choose a constraint to test.
4. Where AI Fits—and Where It Does Not
A ten-minute task does not necessarily improve when an assistant joins it. The instruction takes time; verification takes time; sometimes direct work is simpler. On another task, an agent may read dozens of files, draft a migration, and leave a human with a valuable starting point. The decision depends less on the name of the tool than on the shape and risk of the work.
Ask five questions before granting it a role. Is the result checkable? A generated test can be run, but a convincing explanation of a poorly understood failure may be harder to validate. Is the context available and permitted? An agent cannot infer undocumented customer constraints, and access to private data is not a free convenience. Is the action reversible? Drafting a note is different from changing production data. Who will own the outcome? A reviewer needs enough time and expertise to disagree. What is the credible alternative? Better documentation, a smaller change, a fixed CI pipeline, or a direct human conversation may solve the problem at lower cost.
This yields four legitimate decisions. Use AI for bounded, testable work with a named owner and an inexpensive verification path. Improve the workflow first if waiting, poor context, or oversized changes are the real constraint. Defer when measurement or permissions are not ready. Decline when disclosure, safety, weak verification, or deliberate skill practice makes use inappropriate. None is a moral verdict about an engineer.
The desk-researched account of non-use collects reasons employees give for declining AI: high-context decisions, short tasks, and independent practice. It does not establish a formal safe-to-decline policy or prove that performance reviews reward those choices. Leaders need to make the policy real by rewarding good judgment, including a carefully reasoned "no," not tracking prompts as a proxy for effort.
Autonomy should rise only as evaluation and reversibility improve. A suggestion in a local editor, a bot opening a PR, and an agent with production write access are three different commitments. Require clear permissions, traces of actions, and human approval before consequential or irreversible changes. A human-in-the-loop label alone is not a safeguard if the human is handed more material than they can meaningfully review.
At the junction: Choose one recurring task and fill in: what could AI do, what could a non-AI change do, how would each be verified, and who would absorb any error? If none of the outcomes can be checked within a reasonable window, do not scale the experiment yet.
Part II
Apply the nine competencies to valuable work
5. Delivery & Execution: Are We Working on the Right Promise?
Consider a familiar but illustrative problem: the sprint board is green, every ticket has an owner, and several closed early. The promised customer result is still late. The missing step may be an approval that no team owns, a requirement that changed without being discussed, or a feature that shipped but cannot solve the customer's job. A status summary is not a substitute for finding out which.
Start with a promise stated in the customer's language: "A creator can publish an episode and know when listeners can play it." Translate it into observable steps and a date for checking them. Then ask what work is already in progress, what must be decided before implementation, and where a smaller batch could reach a real user sooner. This is product and engineering work together; AI cannot decide what matters simply by summarizing the backlog.
An assistant can compare a proposal with past decisions, surface likely dependencies, or help split an oversized change. The alternative may be a ten-minute conversation between product, engineering, and the team that approves releases. Either intervention is successful only if it improves the promised result, not if it fills a sprint with more carefully worded tickets. In the Spotify release-process account, manually triggered stage transitions sometimes added hours. Automating condition-checked transitions shortened the average release cycle, while regression and crash gates remained. That is measured delivery improvement; customer adoption was not measured in the account.
When work slips, resist the temptation to ask who failed to work hard enough. Ask whether the queue was predictable, whether a necessary safety gate was resourced, whether a late product decision invalidated earlier work, and whether the next promise can be narrowed. A plan that requires permanently heroic effort is not predictable delivery. Let the team renegotiate scope when the evidence changes.
At the junction: For one promised outcome, write its acceptance condition, its customer-visible date, and the next unavoidable handoff. Compare the last three similar pieces of work by elapsed time, not just coding time. If the highest delay is a decision, schedule the decision before buying another coding tool.
6. Engineering Quality & Reliability: Who Verifies and Recovers?
Imagine a reviewer receiving a cleanly written change that modifies access checks in three services. The tests pass. The risk lies in a permission boundary the author has not mentioned. The critical question is not whether a human clicked "approve"; it is whether someone could identify the unsafe case and had time to test it.
AI can draft tests, point out gaps, and summarize incident evidence. It can also produce code that looks familiar enough to invite shallow review. A team needs risk-proportionate checks: small changes with observable effects; tests that exercise the failure mode, not merely the happy path; a reviewer who understands the boundary; and a recovery path when a change escapes. The non-AI alternative might be clarifying ownership, removing flaky tests, or practicing rollback. Automation should make those controls stronger, not make them appear optional.
Be careful with attractive statistics about "AI correction." One open-source study of Claude Code–identified pull requests found that 214 of 475 merged agentic PRs had a later commit. That is a revision rate among merged PRs, not the proportion of all AI changes needing substantial correction. Revisions may be minor; rejected submissions and edits before opening a PR are not in that denominator. The study did not find a statistically significant difference in measured revision effort from its matched human PRs. A local audit needs a pre-agreed definition: behavior, security, or architecture altered to make a change acceptable, separated from formatting and added scope. Record who found the issue, who specified the fix, and who performed it.
Reliable operation continues after merge. Track customer-impacting incidents and SLOs, change failure, recovery time, and on-call burden, segmented by change risk. A stable incident count over a short pilot is not proof of improved safety; rare harms need a longer window and explicit tests of dangerous cases. Google's SRE on-call guidance makes rollback and escalation practiced skills, not merely text in a runbook. Some migrations or external effects cannot be undone by redeploying an old binary.
The reliability check: Sample recent changes, including rejected ones. Estimate review and correction labor without guessing at unseen work, examine one escaped defect, and ask whether your next proposed AI use makes detection, decision, and recovery easier for the person on call.
7. Flow & Developer Experience: Where Did the Queue Move?
Imagine an illustrative change finished at noon. It waits for review until the next morning, for an environment until Thursday, and for a cross-team approval until Monday. Reducing authoring time from four hours to two would barely move the customer's date. Adding more changes to that queue may move it in the wrong direction.
Salesforce describes rising code volume and review latency after AI-assisted development accelerated code creation. It names human reviewers and larger PRs as constraints. Its report does not publish a ledger of additional hours by reviewer or prove the downstream customer effect. That limitation is exactly why an engineering leader should follow arrivals and waiting at each stage, not merely celebrate local speed.
Two approaches can improve flow. One makes the authoring step faster: an assistant drafts repetitive code, migration scripts, or tests. The other changes the system: smaller batches, a reliable CI pipeline, clear review ownership, earlier detection of common problems, and low-risk approvals that do not wait unnecessarily. A third often matters most—choosing fewer changes with clearer value. The best approach is the one that improves end-to-end time without overloading the people who verify and operate the result.
Do not confuse ease of work with absence of effort. The Microsoft-led field experiments report more completed tasks in the pooled comparison among developers given a coding assistant. They did not measure an equivalent lift in customer outcomes. The SPACE framework likewise cautions against using activity alone as a measure of developer productivity; satisfaction, performance, collaboration, and flow matter in different ways. Neither source says that every engineer should use a tool on every task.
A useful flow map has two kinds of time: active work and waiting. Record PR arrival rate, review age, time to first meaningful review, CI failures, rework, release lag, and customer time to use. Segment by size and risk. Ask reviewers about cognitive load, not just their average response time. An AI review pass may save effort or add another layer; test which.
At the junction: Follow five similar changes from a named customer problem to verified use. Point to the largest queue and the team that owns it. Do not fund a speedup upstream without a credible plan for the work arriving downstream.
8. Architecture & Technical Strategy: Can We Change and Undo Safely?
A new engineer can find the source code for a service. Can they find who owns it? Can they run the tests with ordinary access? If a release harms users, can they say what rollback would reverse and what it would leave behind? An agent with broad repository access faces the same missing map, plus the ability to act on a mistaken guess at scale.
Rippling's code-ownership account describes a catalog connecting code paths to teams and on-call channels. Infrastructure engineers reportedly found an on-call rotation in about 30 seconds after adoption. This supports easier owner discovery in that setting, not a measured first-day success rate for new hires. A catalog can be accurate and still fail to say how to test a service or reverse a migration.
AI can help explain an unfamiliar code path, propose a bounded refactor, or draft an architecture decision for human scrutiny. It cannot give an old dependency the rollback semantics it never had. The non-AI work may be more valuable first: establish owners, reduce coupling, make test commands reproducible, document data migrations and external side effects, and rehearse recovery. Then an AI assistant has context worth retrieving and an engineer has a way to check its advice.
Decisions about architecture should name the trade-off. A shared platform might make many changes easier but raise the cost of a failure that affects everyone. A migration may reduce maintenance effort while temporarily raising operational risk. Write down the alternative considered, the verification plan, the cutover owner, and the stopping condition. Revisiting that decision when facts change is a sign of maturity, not evidence that the first decision was careless.
At the junction: Run a consensual tabletop exercise with someone new to a service. Give them a service name, not the veteran's phone number. Observe the first broken link, unavailable credential, stale owner, failed test, or unclear rollback step. Do not ask them to perform an unapproved production rollback. Fix the path before granting an agent the same permissions.
9. Product & Customer Focus: Did Anyone Benefit?
An experiment can run faster without finding an answer sooner. A team may build more variants, launch them quickly, and never decide which customer behavior would count as a success. At that point experimentation is a production line for uncertainty, not learning.
The Thoughtworks/Etsy case describes pilot teams reducing time to customer validation from about 50 days to five after changes to prototyping and product delivery. It is a non-AI account, and its authors caution against isolating one process tweak as the cause. The useful lesson is to measure time to a validated decision, not the number of mockups. A fast prototype that reveals a bad idea can be valuable if it prevents a costly launch.
AI may help cluster feedback, propose interview questions, and make a prototype cheap enough to test. Each step needs its own check. Feedback summaries can bury rare but serious harms; synthetic personas cannot consent to a feature or show how it works in real life; a mockup praised in a meeting is not evidence of retained customer use. Invite people from the affected segment to try the thing, observe the task, and record where it fails.
Business metrics belong in the picture, but their order matters. A customer may accomplish a task more easily while the company spends more to serve it; that is a trade-off to assess, not a reason to ignore the customer. A feature can also raise short-term revenue while making support or trust worse. Separate adoption, task completion, customer effort, complaints, retention, and margin. State which is the primary outcome for this decision and which are checks against making another perspective worse.
The customer test: Write a customer job in one sentence and a disconfirming test for the proposed feature. Ask who might be excluded from the test, what you'd learn if adoption is high but task success is low, and whether the same learning is possible with a less costly non-AI experiment.
10. Innovation & Learning: What Changed Because We Learned?
Picture an illustrative demo day: six teams build prototypes, and the closing presentation is full of energy. A month later, nobody can say which idea was tested with a customer or which should be stopped. The event produced artifacts and enthusiasm. It has not yet produced a decision.
Innovation means taking a worthwhile uncertainty seriously enough to expose it to evidence. Start with the smallest question that could change a choice: do customers understand the workflow, can a team maintain it, will a new service reduce recovery time, or is a suspected use case too risky? A useful experiment can end with "do not proceed." That answer protects time and trust that would otherwise be spent defending an attractive idea.
AI can reduce the cost of a prototype, generate candidate test cases, or help a team explore an unfamiliar design space. It can also produce enough plausible alternatives to defer judgment indefinitely. Non-AI discovery—watching someone attempt the task, sampling incident records, talking to the reviewer of a similar change—may reach the real uncertainty faster. Compare the two routes by time to credible decision, not number of ideas generated.
The Superhuman/AWS account reports more experiments per engineer in a self-serve growth team; it credits cross-functional work as well as AI. It does not publish an AI-attributable customer-value increase from those experiments. That distinction does not make the experimentation meaningless. It tells us which later question remains: which experiments changed a product decision, and what happened to customers afterward?
Learning also has to travel. If a pilot team discovers a safe evaluation method, make it easy for others to reuse. If it discovers a failure, publish the boundary as clearly as a success story. Protect time to reflect; otherwise every available hour goes to the next demo and no one updates the route.
At the junction: For the last three experiments, find the written hypothesis, the observation that could have changed minds, and the resulting keep/change/stop decision. If none exists, choose one unresolved question and design a smaller test before starting another showcase.
11. People & Culture: Who Gets to Grow and Say No?
A junior engineer hands over a working feature. The reviewer asks why a retry is safe after a timeout; the engineer can describe what the assistant wrote but not the behavior when the request is repeated. The problem is not that the engineer used AI. The problem is that the team has a correct-looking artifact without enough shared understanding to own it. This is an illustrative scenario, not a reported interview.
Anthropic's employee study reports that some staff gained reach into unfamiliar tasks and prototyped faster, while others worried about losing direct debugging practice or opportunities to ask colleagues questions. These are reports from a particular workplace, not longitudinal tests of skill. A junior–senior study examined AI-assisted debugging and how senior reviewers could use code and prompt histories to understand reasoning. It does not show a universal decline in junior capability. The evidence is enough to justify designing practice, not declaring a generation incapable.
Give people two kinds of support: a usable tool and a way to build judgment independent of it. Let a junior explain design alternatives and failure modes, pair on a small incident, and occasionally diagnose a bounded problem without AI. Let a senior spend review time on the "why" instead of only correcting the patch. Ask whether that time is actually in the plan or merely praised in speeches. Protect privacy when sharing prompt histories for coaching.
AI adoption should not become a loyalty test. Atlassian's 2026 survey commentary reports perceived pressure to demonstrate AI use; it does not reveal what any one employer's promotion committee actually rewarded. A manager can make the standard concrete by praising a well-verified use, a justified refusal on a sensitive task, and a reviewer who caught a dangerous assumption. Measure mentoring access and workload at team level; do not turn prompt counts into a ranking of people.
A question for the team: Ask someone early in their career to explain one change they own, including how it fails. Then ask a reviewer what feedback they gave about the reasoning. If the only feedback is "the code passed," invest in practice before using more generated output as a hiring or productivity argument.
12. Collaboration & Communication: Is the Context Usable?
Consider an illustrative ownership search. The result points to a team that reorganized last year; an assistant summarizes the page fluently. The answer is easier to obtain and still wrong. Fast retrieval cannot turn stale context into current responsibility.
Make decisions, owners, and operating instructions discoverable by the people who need them. Record where a customer requirement came from, which team accepts a handoff, and who can approve a rollback. A service catalog helps only if ownership is maintained when teams change. In Rippling's published account, ownership metadata improved on-call routing; it took a coordinated migration to cover the organization. That is a reminder that shared context has a maintenance cost, not proof that a knowledge base updates itself.
AI may help answer routine questions from approved documents, point to a decision record, or draft notes after a meeting. The alternative is to retire stale pages, require owners for critical documents, and settle decisions where people can find them. Sometimes a conversation is the only way to expose a disagreement that no summary could resolve. For any generated answer, provide the source and a route to correct it. In high-stakes situations, prefer a person or current runbook over a confident synthesis of uncertain context.
Onboarding is a good test of collaboration because a newcomer cannot rely on remembered shortcuts. Ask them, with consent, to identify the present owner, run the tests, and explain whom they would call during a bad release. The desk-researched onboarding investigation found examples of owner discovery but no published observation of one newcomer doing all three tasks unaided. That missing evidence is a reason to run a local exercise, not to declare the problem solved.
At the junction: Follow one repeated question from its first answer to its fourth asker. If the answer is hard to find, improve access; if it is wrong, improve ownership and upkeep; if people disagree, make the decision explicit. Only then ask whether AI retrieval reduces the remaining friction.
13. Security & Compliance: What Is the Cost of a Mistake?
An assistant that suggests a test is not the same risk as an agent that can read customer records and modify production. Both may be called "AI adoption" on a dashboard. The permissions, review, and evidence required should be different.
Start with the action and its worst plausible failure. What data can the system see? Can untrusted content redirect its instructions? Can it create a PR, merge it, deploy it, or send data out? Who can stop or reverse the action, and what would a rollback leave untouched? OWASP's LLM application risks include prompt injection, sensitive-information disclosure, and excessive agency; this is a threat taxonomy, not a measured incident rate at your company. NIST's generative AI profile offers a risk-management lens, not a universal ban.
A good boundary makes low-risk work easy and high-consequence work inspectable. Let a drafting assistant work with approved context; sandbox an agent's tools; require evaluation on representative adversarial cases and an authorized human before consequential writes. Keep action logs that let responders reconstruct what happened without treating every employee's prompts as a surveillance feed. Review the full path, including dependencies and the chance that an apparently harmless summary becomes a decision input.
Security also has an opportunity cost. A late, opaque approval gate can strand safe changes; an unrestricted fast lane can transfer risk to customers. Pair security findings and exposure with approval delay, rework, and customer trust. Automating vulnerability triage may let a security engineer focus on severe issues, but only if false positives, false negatives, and who verifies patches are counted.
At the junction: For one proposed agent, list every read and write permission, the source of untrusted input, the irreversible action, and the human who owns the result. Remove unnecessary authority before asking reviewers to accept more generated work.
Part III
Make and test consequential decisions
14. Choose a Bet Worth Testing
Suppose customer bug reports sit in a triage queue for days because each report needs a component owner and a reproducible case. An AI classifier could suggest an owner faster. That is a plausible bet, not yet a customer benefit. Misrouting one severe report could make the customer wait longer than before.
Write the bet as a chain: suggested classification → verified routing → faster appropriate fix → shorter customer resolution time. Name the customer group, the receiving team, and the decision owner. Compare the AI classifier with a non-AI alternative: simpler categories, clearer ownership rules, or better intake. A fast classifier only matters if downstream capacity can act on its suggestions.
Salesforce's BugWiser account reports a dramatic reduction in a historical classification workload and roughly 90% of classification predictions accepted unchanged in early deployment. Its comparison contrasts person-days of manual effort with elapsed time using a system; those are different units. It does not establish the customer-resolution improvement or net return from the same pilot. Use the account as evidence that this task might be worth testing, not as a forecast to insert into your budget.
Choose one primary claim. If the customer is the intended beneficiary, make verified resolution time primary. Check routing accuracy, severe-case misses, review labor, tool and operating cost, and the triage team's workload. Specify the observation window and the threshold that would trigger a pause before people become invested in keeping the tool. A small, representative pilot beats a large launch whose outcome cannot be interpreted.
Do not solve an absence of evidence by multiplying indicators. Ten green metrics can coexist with an unchanged customer result. Give each metric a role: outcome, mechanism, cost, or safety check. If no reliable baseline exists, start with instrumentation and a smaller question. Waiting for a defensible test is different from waiting for perfect certainty.
The pilot card: Complete this sentence with named owners: "We will test [intervention] against [alternative] for [population], expecting [primary outcome] by [time], while pausing if [guardrail] worsens." If the last clause cannot be written, the bet is not ready.
15. Set Boundaries Before Delegating
An agent can draft a patch, open a PR, and explain its intent. It can also be wrong about the edge case it was never shown. The next decision is not whether a human is "in the loop" on a slide. It is what the human can actually inspect, when they have authority to stop, and whether the result can be undone.
Set permissions by task. A documentation draft might use approved internal pages and create no external effect. A proposed code change should carry tests and provenance into a human review. A production write needs narrower authority, a demonstrated recovery path, and explicit authorization. Treat untrusted documents and issue text as data that may contain instructions to an agent; constrain tools and data exposure accordingly. The same control should apply to comparable human and AI changes where their risks are comparable.
Policies are usable only if people know when not to use the tool. Sensitive information, an uncheckable answer, a ten-minute direct task, and protected independent practice are all legitimate reasons to decline. A blanket prohibition may drive work underground; a usage quota may reward performative prompts. State who can approve an exception, how it is recorded, and how an engineer can challenge a generated change without being penalized for slowing a dashboard.
Evaluation is a rehearsal of the work the agent will actually do. Include routine tasks, failure cases, changing context, and the cases where a superficially correct result would cause harm. Record whether an evaluator caught the error, not merely whether a model produced a confident explanation. A rollback plan should include migrations and external side effects that redeploying old code cannot reverse.
At the junction: Draw three columns for a proposed workflow: can read, can propose, can change. Put a named human decision owner and verification test beside each consequential action. If the owner cannot examine the evidence in time, reduce the agent's authority before scaling it.
16. Measure the Result, Not the Usage
Imagine an illustrative finance meeting that begins with a number: engineers say they save six hours a week. Somebody multiplies those hours by a salary and proposes a return. Yet payroll is unchanged, reviewers may be working longer, and no one has identified the customer result that filled the new capacity. The arithmetic is tidy because the hardest questions were left out.
Build a ledger with different columns. Actual savings are expenses removed. Avoided costs depend on a credible counterfactual. Created capacity is useful only if it is applied to valuable work; it is not automatically cash. Enabled revenue needs an observed path from the change to customer use and contribution margin. Reduced risk needs a baseline, severity, and assumptions about losses that might have occurred. Count tool fees, compute, integration, training, review, rework, governance, and ongoing support in the same observation window. Do not put the same change in two benefit columns.
IBM's published method and RevTech account is unusually explicit about categories and costs. It reports an approximately tenfold modeled return for its own tool using matched pre/post windows and business-loss assumptions. That is informative, not independently verified cash savings or a randomized effect. Likewise, the Microsoft-led field experiments measured a pooled increase in completed developer tasks among people given a coding assistant. They did not measure a matching gain in company profit.
For your own pilot, compare similar work over the same period; randomize eligible work or phase rollout where feasible. Log staffing, task mix, platform changes, and changed approval rules. A before/after improvement during several concurrent reforms is an observation, not a single-cause estimate. Make uncertain assumptions visible and show a range rather than a precise-looking point. Short pilots may be able to test speed and review effort while remaining too small to establish a change in rare incidents.
Use delivery and developer-experience metrics as diagnostics. Shorter lead time with worse change failure or overloaded reviewers is a warning. More PRs without customer use are not a benefit line. Satisfaction is worth listening to because people matter, but it cannot stand in for a customer or financial claim.
Audit your claim: Take your current AI business case and label every line "measured," "self-reported," "modeled," or "unknown." Find the most influential assumption and the next observation that could falsify it. If the conclusion survives that challenge, you have a stronger decision than an adoption chart could offer.
17. Lead the Human Transition
The people most affected by an AI rollout often enter the meeting after the benefits have been promised: the reviewer who must check a larger change, the on-call engineer who owns its failure, the junior who needs practice, and the product partner who must explain the result to customers. A change process that calls their questions "resistance" discards information the organization needs.
Invite these people before selecting the metric. Product can say what customers are waiting for; finance can distinguish capacity from booked savings; security can define an unacceptable boundary; reviewers can expose a new queue; engineers can say where the tool helps or makes work harder. Make it safe to give an unfavorable answer. A survey about whether employees "like AI" is not a substitute for a discussion about work, autonomy, and responsibility.
Training should include how to challenge the tool, not only how to prompt it. A team can practice recognizing a plausible error, explaining why a test matters, and declining a task it cannot validate. Pair novices with experienced reviewers for reasoning, not just stylistic corrections. Give reviewers time to do this well; otherwise mentoring becomes an unfunded expectation and quality becomes somebody else's problem.
Some changes create genuine tension. A company might want more output with the same headcount; employees might fear that the new capacity will be used as an argument to eliminate their roles. Do not promise an outcome leadership cannot guarantee. State what will be measured, how decisions about staffing and development will be made, and which information will not be used for individual surveillance. Recognize careful non-use, error detection, and collaboration as contributions, not failures to adopt.
At the junction: Before a pilot begins, ask one affected reviewer or operator and one person early in their career what would make this change valuable—and what would make it unsafe or unfair. Put those answers beside the executive goal, and decide which concerns require a design change.
18. Build Only the Enabling Platform You Need
An internal platform can make every team faster, or become an expensive destination in its own right. Its value depends on whether people can safely reach the work that matters: current context, appropriate permissions, reliable tests, a review path, and a way to know what happened afterward.
Start with a recurring, bounded need. Perhaps ten teams need to search the same set of maintained decision records; perhaps reviewers need a consistent test report for agent-opened PRs. Provide the smallest shared capability that can be evaluated. Let teams show demand before centralizing every experiment. Avoid building a polished gateway that no one can use on the work that actually waits.
The Superhuman/AWS account describes governed connectors, self-service prototyping, and shared infrastructure accompanying wider AI use. The account also describes workflow changes and an unchanged median PR cycle time, so it cannot isolate the platform's contribution to customer value. The organization itself reports that validation became a constraint. Those are reasons to budget evaluation and reviewer capacity with the model and compute, not after code volume rises.
Keep context fresh and access narrow. A retrieval layer over stale ownership records speeds up wrong answers. A connector with broad write permissions turns a mistaken instruction into an operational event. Publish costs per validated outcome rather than only tokens per request. Include maintaining templates, evaluating model changes, handling incidents, and retiring unused features. A platform team is a service provider with users and a product to improve, not a license distribution office.
At the junction: Name one repeated bottleneck the platform could remove and one team willing to test that claim. Specify how you will know it helped and which shared capability can be deferred until that evidence exists.
Part IV
Scale what works; leave room to change course
19. When to Scale, Change, or Stop
Pilots are usually asked to produce a success story. The more valuable service may be to prevent a large mistake. If a triage assistant routes routine reports quickly but misses severe ones, stopping it is an outcome, not an embarrassment. If it helps only one task class, narrowing the workflow may be better than launching everywhere.
Return to the decision made before the pilot. Was the primary outcome customer resolution, reduced operating cost, a safer system, team capability, or individual growth? What was the baseline and comparison? What happened to the other three perspectives? An increase in local speed with a larger review queue is not an unambiguous gain. Nor is an unchanged incident count across a handful of changes proof that risk disappeared.
Choose among four verbs. Keep the bounded use when it meets the intended outcome and its guardrails. Change scope, permissions, process, or measurement when a fixable constraint limited it. Scale only when the result survives a harder comparison and the receiving teams can carry the workload. Stop when verification cost, harm, or weak demand defeats the case. State the decision owner, review date, and evidence that would reverse the decision.
Tools change, so a negative result from an old version does not settle the question forever; neither does a positive result from a different organization settle yours today. The DORA 2025 report describes AI as amplifying the surrounding system. That is a reason to revisit both the tool and the conditions that help it, not a reason to chase maximum autonomy as a maturity score.
At the junction: Ask a person who would bear the cost of scaling to make the strongest case against it. Write down what evidence would change their mind and what evidence would change yours. If neither can be specified, pause rather than treating momentum as proof.
20. Your Next Ninety Days
Imagine returning to the meeting where somebody first proposed the tool. This time the question on the table is not "How many people will use it?" It is "Which important result is waiting, what holds it back, and what is the smallest safe way to learn whether this will help?" That question does not settle the answer. It gives the people responsible a fairer way to find it.
In the first thirty days, choose one named customer or operational outcome. Trace the path backward through discovery, build, review, approval, release, and use. Ask the person at each boundary where work waits and who can see a failure. Record the baseline and the limits of what you know. Put an AI option beside a non-AI improvement, a defer option, and a reasoned decline.
In days thirty-one through sixty, run one bounded comparison if the permissions and measurements are ready. Name the primary outcome and guardrails before adoption, include the whole cost, and give a person authority to pause the experiment. If there is no credible comparison, start by improving the instrumentation or shortening the question; do not pretend a license rollout is a pilot.
In days sixty-one through ninety, sit with customer, finance, engineering, security, reviewers, and affected workers. Ask what became better, which change was observed rather than merely reported, and who absorbed work elsewhere. Record a keep/change/scale/stop decision, what would reverse it, and the next unanswered question. If the test found no benefit, you have still avoided a larger unexamined commitment.
There are trail markers at the important junctions: the destination, the context and permissions, the verification gate, the release, and the return to learn from what actually happened. They are not commands to take every step in the same way. The purpose is to help people and teams explore without losing their responsibilities to customers—or each other.
The next marker: Make the first decision small enough to revisit. Do not ask the team to prove its enthusiasm. Ask it to show, with the people affected, whether the route led anywhere worth going.
Field Guide
These are working templates, not universal standards. Adapt them with the people who will use and be affected by them.
A. Competency map
When an outcome stalls, look across the nine competencies: delivery and execution; quality and reliability; flow and developer experience; architecture and technical strategy; product and customer focus; innovation and learning; people and culture; collaboration and communication; security and compliance. Identify a concrete constraint and owner. Do not assign a maturity score to an individual or assume the highest score requires the most AI.
B. The four-way decision card
Problem and recipient: Who is waiting for what? Current path: Where does work wait or fail? Options: What could AI change? What non-AI improvement would compete with it? What would justify deferring or declining? Consequences: What can each option read, propose, or change? Who verifies and recovers? Decision: Which smallest test, by whom, and when will it be revisited?
C. Value ledger and pilot card
| Perspective | Primary or check? | Baseline and comparison | Cost, harm, or uncertainty |
|---|---|---|---|
| Customer | Task success, time to use, or safe resolution | Same population and observation window | Exclusion, trust, reliability, privacy |
| Company | Margin, booked saving, avoided cost, or funded capacity | Finance-owned assumptions and actual spend | Tool, compute, review, training, operation |
| Team | End-to-end flow and safe delivery | Comparable risk, change size, and staffing | Review queue, rework, on-call load |
| Individual | Agency, growth, sustainable work | Voluntary feedback and practice opportunities | Surveillance, workload transfer, lost mentoring |
Record one primary decision metric, observation window, target or meaningful change, pause condition, comparison design, and a named decision owner. Put self-reports, measured events, and modeled values in separate columns; do not double-count freed hours as savings and new output.
D. Usage-policy prompts
Specify approved tools and data; maximum permissions; prohibited disclosure or irreversible actions; review and audit needs; escalation and rollback authority; representative evaluation cases; and a justified way to decline AI without penalty. Ask whether a human reviewer can actually inspect the work entrusted to them. A signed policy does not replace a tested workflow.
E. Evidence label
Next to each consequential number, write its source, publication date, population, denominator, method, comparison, and limitation. Mark it measured, self-reported, modeled, inferred, or unknown. A randomized task study is not a customer-revenue study; an incident report is not a model-evaluation study; a vendor's ROI model is not independently booked cash.
F. Terms
Outcome: A change in the condition or ability of a person or organization, not the artifact that might produce it. Capacity: Time or ability made available, whose value depends on what it enables. Counter-metric: A measure chosen to detect a cost or harm hidden by an apparent gain. Verification: A check able to falsify a proposed result, not a confident explanation of it. Trail marker: An explicit decision or safety checkpoint that helps people steer without controlling every action. Provisional draft: A text awaiting original reporting, source review, revision, and author approval; this book remains one.