Browse the book & resources

Supporting material

Organization Engineering Competency

Working support material for a provisional manuscript; not a verified claim about any reader’s organization.

This guide defines the competencies an engineering organization needs to be effective, efficient, innovative, and engaging. Use it to assess organizational maturity, set improvement goals, and align teams on what "good" looks like.

> This framework assesses the organization and its teams, not individual engineers. For individual growth, use a separate career ladder.

---

Competency Areas

#AreaPurpose
1Delivery & ExecutionShip the right outcomes predictably
2Engineering Quality & ReliabilityBuild systems that work and recover fast
3Flow & Developer ExperienceRemove friction from idea to production
4Architecture & Technical StrategyMake sound, sustainable technical decisions
5Product & Customer FocusConnect engineering work to customer value
6Innovation & LearningExplore, experiment, and adopt what works
7People & CultureAttract, grow, and retain motivated engineers
8Collaboration & CommunicationWork well across teams and functions
9Security & ComplianceProtect customers, data, and the business

---

Maturity Levels

LevelNameDescription
1InitialAd hoc, dependent on individual heroics, inconsistent results
2DevelopingPractices exist in some teams; partly documented; uneven adoption
3DefinedPractices standardized across the organization and measured
4ManagedDecisions driven by data; continuous improvement is routine
5LeadingIndustry-leading; practices are refined, shared, and copied by others

Aim for Level 3 across all areas before pushing any single area to Level 4–5.

---

1. Delivery & Execution

Definition: The organization plans realistically, commits to outcomes, and delivers them predictably.

Key behaviors

  • Plans are tied to business outcomes, not only feature lists.
  • Work is sliced small and delivered incrementally.
  • Risks and dependencies are surfaced early.
  • Scope is negotiated openly when constraints change.

Indicators

  • % of quarterly outcomes achieved
  • Milestone predictability (delivered within ±1 sprint)
  • Rate of descoped or carried-over work
LevelLooks like
1Deadlines are routinely missed; plans change without explanation
3Consistent planning cadence; ≥80% of commitments met; risks tracked
5Outcome-based planning; forecasts are trusted by the business

---

2. Engineering Quality & Reliability

Definition: Systems are well tested, observable, and resilient; incidents are rare and resolved quickly.

Key behaviors

  • Automated testing is part of every change.
  • Services have defined SLOs and monitoring with alerts that matter.
  • Incidents are followed by blameless reviews with tracked actions.
  • Technical debt is visible and actively managed.

Indicators

  • Change failure rate
  • Failed-deployment recovery time
  • Sev1/Sev2 incident frequency
  • SLO attainment
LevelLooks like
1Manual testing; outages found by customers; firefighting culture
3CI with automated tests; SLOs on critical services; postmortems standard
5Proactive resilience (chaos testing, error budgets drive priorities)

---

3. Flow & Developer Experience

Definition: Engineers can move from idea to production quickly and safely, with minimal friction.

Key behaviors

  • Small, frequent deployments through automated pipelines.
  • Fast builds, tests, and code reviews.
  • Self-service environments and platforms.
  • Toil and unplanned work are measured and reduced.

Indicators

  • Lead time for changes and deployment frequency
  • CI duration and PR review turnaround
  • Unplanned work as % of capacity
  • Developer experience survey scores (ease of delivery, focus time)
LevelLooks like
1Infrequent, risky releases; long waits for environments or reviews
3Daily deploys on core services; CI under ~15 min; review SLAs exist
5On-demand deploys; internal platform treated as a product

---

4. Architecture & Technical Strategy

Definition: Technical decisions are deliberate, documented, and aligned with long-term business needs.

Key behaviors

  • Significant decisions are recorded (e.g., ADRs, RFCs).
  • Architecture supports independent team delivery.
  • A technology radar or standards guide technology choices.
  • Build-vs-buy and deprecation decisions are made explicitly.

Indicators

  • % of major decisions with documented rationale
  • Cross-team dependency count per delivery
  • Tech debt trend and legacy system footprint
LevelLooks like
1Decisions made informally; tightly coupled systems; tech sprawl
3RFC/ADR process; clear service ownership; agreed standards
5Architecture evolves intentionally; strategy anticipates business needs

---

5. Product & Customer Focus

Definition: Engineers understand the customer and measure success by impact, not output.

Key behaviors

  • Engineers take part in discovery and customer feedback.
  • Features ship with success metrics and instrumentation.
  • Teams review outcomes after launch and iterate.

Indicators

  • % of launches with defined and reviewed success metrics
  • Feature adoption and usage
  • Customer satisfaction (CSAT/NPS) trend for engineering-owned areas
LevelLooks like
1Engineering receives specs; no visibility into impact
3Shared outcome goals with product; launches are measured
5Teams own outcomes end to end and drive the roadmap with data

---

6. Innovation & Learning

Definition: The organization creates space to explore, runs disciplined experiments, and spreads what works.

Key behaviors

  • Capacity is protected for exploration and platform improvements.
  • Ideas are tested quickly with clear hypotheses.
  • Learnings are shared (demos, write-ups, guilds).
  • Proven improvements are adopted across teams.

Indicators

  • % of capacity allocated to exploration
  • Time from idea to validated learning
  • % of experiments reaching a keep/kill/scale decision
  • Cross-team adoption of internal improvements
LevelLooks like
1No time for exploration; ideas die without evaluation
3Regular experimentation; protected capacity; learning shared openly
5Innovation is a repeatable engine producing measurable business value

---

7. People & Culture

Definition: Engineers are engaged, growing, and supported in a psychologically safe, sustainable environment.

Key behaviors

  • Every engineer has a growth plan and regular feedback.
  • Managers are trained and evaluated on team health.
  • Workload and on-call are sustainable.
  • Hiring and promotion are fair, structured, and transparent.

Indicators

  • Engagement / eNPS score and survey participation
  • Regretted attrition
  • Internal promotion and mobility rate
  • On-call load and after-hours work trends
LevelLooks like
1High turnover; unclear growth paths; burnout is common
3Career ladder in use; regular 1:1s and feedback; engagement tracked
5Talent magnet; strong internal pipeline of leaders

---

8. Collaboration & Communication

Definition: Teams share context, coordinate well, and resolve conflict constructively.

Key behaviors

  • Clear team ownership and interfaces (who owns what).
  • Written communication is default for decisions and plans.
  • Cross-functional partners (product, design, ops) are involved early.
  • Knowledge is documented and discoverable.

Indicators

  • Cross-team blocker resolution time
  • Documentation coverage for critical systems
  • Partner satisfaction (survey of product/design/business stakeholders)
LevelLooks like
1Silos; tribal knowledge; frequent handoff failures
3Clear ownership map; documented decisions; regular cross-team syncs
5Seamless collaboration; information flows without friction

---

9. Security & Compliance

Definition: Security and compliance are built into everyday engineering, not added at the end.

Key behaviors

  • Security scanning is automated in the pipeline.
  • Threat modeling for significant changes.
  • Vulnerabilities are fixed within defined SLAs.
  • Access follows least-privilege principles.

Indicators

  • Mean time to remediate critical vulnerabilities
  • % of services with automated security scanning
  • Audit findings and repeat findings
LevelLooks like
1Security is reactive; audits are stressful scrambles
3Automated scanning; remediation SLAs met; regular training
5Security by default; continuous compliance

---

Mapping to Organizational Goals

GoalPrimary competencies
Effective1. Delivery & Execution, 2. Quality & Reliability, 5. Product & Customer Focus, 9. Security & Compliance
Efficient3. Flow & Developer Experience, 4. Architecture & Technical Strategy, 8. Collaboration & Communication
Innovative6. Innovation & Learning, 4. Architecture & Technical Strategy
Engaging7. People & Culture, 8. Collaboration & Communication

---

How to Use This Guide

  1. Assess: Each team or the leadership group rates current maturity (1–5) for each area, with evidence.
  2. Calibrate: Compare ratings across teams to agree on a shared baseline.
  3. Prioritize: Pick 2–3 areas to improve per quarter, focusing on the lowest levels first.
  4. Set key results: Turn the indicators into measurable targets with baselines and deadlines.
  5. Review: Reassess every 6 months and track the trend.

Guidelines

  • Measure teams and systems, not individuals; metrics tied to individual performance get gamed.
  • Pair system data (e.g., DORA metrics) with survey data (e.g., developer experience) for a full picture.
  • Use a counter-metric for each target so improving one area doesn't harm another.
  • Treat the levels as a guide for conversation, not a score to compete on.

---

Assessment Template

AreaCurrent level (1–5)EvidenceTarget levelOwnerNext review
Delivery & Execution
Quality & Reliability
Flow & Developer Experience
Architecture & Technical Strategy
Product & Customer Focus
Innovation & Learning
People & Culture
Collaboration & Communication
Security & Compliance