Supporting material
Organization Engineering Competency
Working support material for a provisional manuscript; not a verified claim about any reader’s organization.
This guide defines the competencies an engineering organization needs to be effective, efficient, innovative, and engaging. Use it to assess organizational maturity, set improvement goals, and align teams on what "good" looks like.
> This framework assesses the organization and its teams, not individual engineers. For individual growth, use a separate career ladder.
---
Competency Areas
| # | Area | Purpose |
|---|---|---|
| 1 | Delivery & Execution | Ship the right outcomes predictably |
| 2 | Engineering Quality & Reliability | Build systems that work and recover fast |
| 3 | Flow & Developer Experience | Remove friction from idea to production |
| 4 | Architecture & Technical Strategy | Make sound, sustainable technical decisions |
| 5 | Product & Customer Focus | Connect engineering work to customer value |
| 6 | Innovation & Learning | Explore, experiment, and adopt what works |
| 7 | People & Culture | Attract, grow, and retain motivated engineers |
| 8 | Collaboration & Communication | Work well across teams and functions |
| 9 | Security & Compliance | Protect customers, data, and the business |
---
Maturity Levels
| Level | Name | Description |
|---|---|---|
| 1 | Initial | Ad hoc, dependent on individual heroics, inconsistent results |
| 2 | Developing | Practices exist in some teams; partly documented; uneven adoption |
| 3 | Defined | Practices standardized across the organization and measured |
| 4 | Managed | Decisions driven by data; continuous improvement is routine |
| 5 | Leading | Industry-leading; practices are refined, shared, and copied by others |
Aim for Level 3 across all areas before pushing any single area to Level 4–5.
---
1. Delivery & Execution
Definition: The organization plans realistically, commits to outcomes, and delivers them predictably.
Key behaviors
- Plans are tied to business outcomes, not only feature lists.
- Work is sliced small and delivered incrementally.
- Risks and dependencies are surfaced early.
- Scope is negotiated openly when constraints change.
Indicators
- % of quarterly outcomes achieved
- Milestone predictability (delivered within ±1 sprint)
- Rate of descoped or carried-over work
| Level | Looks like |
|---|---|
| 1 | Deadlines are routinely missed; plans change without explanation |
| 3 | Consistent planning cadence; ≥80% of commitments met; risks tracked |
| 5 | Outcome-based planning; forecasts are trusted by the business |
---
2. Engineering Quality & Reliability
Definition: Systems are well tested, observable, and resilient; incidents are rare and resolved quickly.
Key behaviors
- Automated testing is part of every change.
- Services have defined SLOs and monitoring with alerts that matter.
- Incidents are followed by blameless reviews with tracked actions.
- Technical debt is visible and actively managed.
Indicators
- Change failure rate
- Failed-deployment recovery time
- Sev1/Sev2 incident frequency
- SLO attainment
| Level | Looks like |
|---|---|
| 1 | Manual testing; outages found by customers; firefighting culture |
| 3 | CI with automated tests; SLOs on critical services; postmortems standard |
| 5 | Proactive resilience (chaos testing, error budgets drive priorities) |
---
3. Flow & Developer Experience
Definition: Engineers can move from idea to production quickly and safely, with minimal friction.
Key behaviors
- Small, frequent deployments through automated pipelines.
- Fast builds, tests, and code reviews.
- Self-service environments and platforms.
- Toil and unplanned work are measured and reduced.
Indicators
- Lead time for changes and deployment frequency
- CI duration and PR review turnaround
- Unplanned work as % of capacity
- Developer experience survey scores (ease of delivery, focus time)
| Level | Looks like |
|---|---|
| 1 | Infrequent, risky releases; long waits for environments or reviews |
| 3 | Daily deploys on core services; CI under ~15 min; review SLAs exist |
| 5 | On-demand deploys; internal platform treated as a product |
---
4. Architecture & Technical Strategy
Definition: Technical decisions are deliberate, documented, and aligned with long-term business needs.
Key behaviors
- Significant decisions are recorded (e.g., ADRs, RFCs).
- Architecture supports independent team delivery.
- A technology radar or standards guide technology choices.
- Build-vs-buy and deprecation decisions are made explicitly.
Indicators
- % of major decisions with documented rationale
- Cross-team dependency count per delivery
- Tech debt trend and legacy system footprint
| Level | Looks like |
|---|---|
| 1 | Decisions made informally; tightly coupled systems; tech sprawl |
| 3 | RFC/ADR process; clear service ownership; agreed standards |
| 5 | Architecture evolves intentionally; strategy anticipates business needs |
---
5. Product & Customer Focus
Definition: Engineers understand the customer and measure success by impact, not output.
Key behaviors
- Engineers take part in discovery and customer feedback.
- Features ship with success metrics and instrumentation.
- Teams review outcomes after launch and iterate.
Indicators
- % of launches with defined and reviewed success metrics
- Feature adoption and usage
- Customer satisfaction (CSAT/NPS) trend for engineering-owned areas
| Level | Looks like |
|---|---|
| 1 | Engineering receives specs; no visibility into impact |
| 3 | Shared outcome goals with product; launches are measured |
| 5 | Teams own outcomes end to end and drive the roadmap with data |
---
6. Innovation & Learning
Definition: The organization creates space to explore, runs disciplined experiments, and spreads what works.
Key behaviors
- Capacity is protected for exploration and platform improvements.
- Ideas are tested quickly with clear hypotheses.
- Learnings are shared (demos, write-ups, guilds).
- Proven improvements are adopted across teams.
Indicators
- % of capacity allocated to exploration
- Time from idea to validated learning
- % of experiments reaching a keep/kill/scale decision
- Cross-team adoption of internal improvements
| Level | Looks like |
|---|---|
| 1 | No time for exploration; ideas die without evaluation |
| 3 | Regular experimentation; protected capacity; learning shared openly |
| 5 | Innovation is a repeatable engine producing measurable business value |
---
7. People & Culture
Definition: Engineers are engaged, growing, and supported in a psychologically safe, sustainable environment.
Key behaviors
- Every engineer has a growth plan and regular feedback.
- Managers are trained and evaluated on team health.
- Workload and on-call are sustainable.
- Hiring and promotion are fair, structured, and transparent.
Indicators
- Engagement / eNPS score and survey participation
- Regretted attrition
- Internal promotion and mobility rate
- On-call load and after-hours work trends
| Level | Looks like |
|---|---|
| 1 | High turnover; unclear growth paths; burnout is common |
| 3 | Career ladder in use; regular 1:1s and feedback; engagement tracked |
| 5 | Talent magnet; strong internal pipeline of leaders |
---
8. Collaboration & Communication
Definition: Teams share context, coordinate well, and resolve conflict constructively.
Key behaviors
- Clear team ownership and interfaces (who owns what).
- Written communication is default for decisions and plans.
- Cross-functional partners (product, design, ops) are involved early.
- Knowledge is documented and discoverable.
Indicators
- Cross-team blocker resolution time
- Documentation coverage for critical systems
- Partner satisfaction (survey of product/design/business stakeholders)
| Level | Looks like |
|---|---|
| 1 | Silos; tribal knowledge; frequent handoff failures |
| 3 | Clear ownership map; documented decisions; regular cross-team syncs |
| 5 | Seamless collaboration; information flows without friction |
---
9. Security & Compliance
Definition: Security and compliance are built into everyday engineering, not added at the end.
Key behaviors
- Security scanning is automated in the pipeline.
- Threat modeling for significant changes.
- Vulnerabilities are fixed within defined SLAs.
- Access follows least-privilege principles.
Indicators
- Mean time to remediate critical vulnerabilities
- % of services with automated security scanning
- Audit findings and repeat findings
| Level | Looks like |
|---|---|
| 1 | Security is reactive; audits are stressful scrambles |
| 3 | Automated scanning; remediation SLAs met; regular training |
| 5 | Security by default; continuous compliance |
---
Mapping to Organizational Goals
| Goal | Primary competencies |
|---|---|
| Effective | 1. Delivery & Execution, 2. Quality & Reliability, 5. Product & Customer Focus, 9. Security & Compliance |
| Efficient | 3. Flow & Developer Experience, 4. Architecture & Technical Strategy, 8. Collaboration & Communication |
| Innovative | 6. Innovation & Learning, 4. Architecture & Technical Strategy |
| Engaging | 7. People & Culture, 8. Collaboration & Communication |
---
How to Use This Guide
- Assess: Each team or the leadership group rates current maturity (1–5) for each area, with evidence.
- Calibrate: Compare ratings across teams to agree on a shared baseline.
- Prioritize: Pick 2–3 areas to improve per quarter, focusing on the lowest levels first.
- Set key results: Turn the indicators into measurable targets with baselines and deadlines.
- Review: Reassess every 6 months and track the trend.
Guidelines
- Measure teams and systems, not individuals; metrics tied to individual performance get gamed.
- Pair system data (e.g., DORA metrics) with survey data (e.g., developer experience) for a full picture.
- Use a counter-metric for each target so improving one area doesn't harm another.
- Treat the levels as a guide for conversation, not a score to compete on.
---
Assessment Template
| Area | Current level (1–5) | Evidence | Target level | Owner | Next review |
|---|---|---|---|---|---|
| Delivery & Execution | |||||
| Quality & Reliability | |||||
| Flow & Developer Experience | |||||
| Architecture & Technical Strategy | |||||
| Product & Customer Focus | |||||
| Innovation & Learning | |||||
| People & Culture | |||||
| Collaboration & Communication | |||||
| Security & Compliance |