DORA metrics show strong performance, but your system is still brittle. Learn how vanity metrics hide technical debt and how Keypup's MCP Server reveals the full story through system legibility analysis.
TL;DR: DORA metrics are powerful organizational indicators, but they can become dangerous vanity metrics that hide critical technical debt.
Your DORA dashboard looks great. Deployment frequency is up 40%. Lead time for changes has dropped to under 4 hours. Change failure rate is holding steady at 8%. By all organizational metrics, your engineering team is performing exceptionally.
But something feels wrong.
Developers are frustrated. The codebase is becoming harder to navigate. Minor changes require touching dozens of files. The same senior engineers are bottlenecked on every critical path. Pull requests that should be straightforward are taking days in review because "nobody else understands this part of the system."
This is the DORA paradox: strong aggregate performance metrics masking deteriorating system health.
"We optimized ourselves into a corner. Our DORA metrics looked amazingâwe were shipping fast. But we were also accumulating tech debt at an unsustainable rate. Shortcuts everywhere. The code worked, but nobody understood why. When our tech lead left, everything collapsed. Our 'high-performing' metrics were built on a house of cards."
This isn't an indictment of DORA metrics themselves. They remain valuable for organizational-level insights. But without system legibilityâunderstanding the internal structure, clarity, and maintainability of your engineering systemsâDORA metrics tell an incomplete and sometimes misleading story.
System legibility is the degree to which your engineering system can be understood, modified, and maintained by the team working on it. A legible system has:
High DORA metrics with low system legibility create a dangerous illusion of success. You're moving fast, but on increasingly fragile foundations.
Warning Signs Your DORA Metrics Are Masking Problems:
Reddit's engineering communities have been sounding the alarm. In r/ExperiencedDevs and r/devops, recurring threads warn about organizations that optimize for metrics at the expense of sustainable engineering practices.
"My previous company was obsessed with deployment frequency. We went from 20 deploys/month to 80. Management loved it. But here's what they didn't see: we stopped writing tests because they slowed us down. We stopped doing proper code review because it 'blocked velocity.' We shipped fast, but we shipped garbage. Six months later, our production incidents tripled."
DORA metrics are typically reported at the team or organization level. This averaging obscures dangerous concentration patterns:
"Our deployment frequency looked great until our Staff Engineer went on vacation. Suddenly nobody could deploy anything because he was the only one who understood the deployment pipeline. We were 'high performing' only because of one person. That's not resilience, that's risk."
Low lead time is celebrated, but it doesn't distinguish between:
Optimizing for speed without accounting for complexity creates perverse incentives.
A stable change failure rate can hide:
"Our CFR was steady at 5%. What the metric didn't show: we had a 'shadow QA process' where senior devs would frantically test in production right after deploys to catch issues before customers did. We weren't preventing failuresâwe were just catching them faster. Completely unsustainable."
Fast mean time to recovery sounds positive. But it can indicate:
The metric rewards speed of recovery, not the more important goal of eliminating the need for recovery.
This is where Keypup's MCP (Model Context Protocol) Server fundamentally changes the game. Instead of replacing DORA metrics, it adds the missing layer: deep-dive analysis into how work actually happens.
The Keypup MCP Server integrates directly into AI assistants like Claude, ChatGPT, Cursor, and Windsurf, bringing engineering analytics into your natural workflow. More importantly, it enables queries that reveal system legibilityâthe patterns that aggregate metrics miss.
The MCP Server provides AI assistants with structured access to your engineering data across GitHub, GitLab, Jira, Azure DevOps, and Bitbucket. Instead of viewing dashboards, you ask natural language questions, and the AI generates and executes the appropriate data queries.
But the real power is in the types of questions you can now askâquestions that surface technical debt, concentration risk, and structural problems.
Let's explore specific analyses that reveal what DORA metrics miss. Each example includes the MCP prompt, the underlying query structure, and what the results tell you about system health.
Your deployment frequency is high, but is it sustainable? Or is it dependent on a few key individuals?
MCP Prompt: Identify Author Concentration Risk
Show me the distribution of merged pull requests by author over the last 6 months. I want to see if our deployment capacity is concentrated in a small number of developers. Include the percentage each author contributes to the total.
What This Reveals
Key Insights đ
â ď¸ Bottlenecks and Problem Areas
đĄ Recommendations
đ Priority Actions
High deployment frequency is great, but how complex are those changes? Are you shipping thoughtfully designed features or accumulating incremental patches?
MCP Prompt: Analyze Change Complexity Over Time
Show me the trend of average pull request size (lines changed) and average number of files changed per PR over the last 12 months, broken down by month. I want to see if our PRs are getting larger and more complex over time.
What This Reveals
Key Insights đ
â ď¸ Bottlenecks and Problem Areas
đĄ Recommendations
đ Priority Actions
Low lead time looks good, but is it achieved through rubber-stamp reviews? Or are certain changes getting proper scrutiny while others aren't?
MCP Prompt: Analyze Review Patterns and Bottlenecks
Show me pull request cycle time broken down by stage: time to first review, time from first review to approval, and time from approval to merge. Group by repository or team. I want to see where reviews are actually happening vs where they're being skipped.
What This Reveals
Key Insights đ
â ď¸ Bottlenecks and Problem Areas
đĄ Recommendations
đ Priority Actions
Your change failure rate is stable, but are you preventing problems or just getting better at quick fixes?
MCP Prompt: Identify Fix vs Feature Ratio
Show me the ratio of fix-related commits (commits with "fix", "bug", "hotfix", "patch" in the message) to feature commits over the last 12 months by month. Also show me the trend of emergency deployments or reverts. I want to see if we're spending more time fixing things than building new capabilities.
What This Reveals
Key Insights đ
â ď¸ Bottlenecks and Problem Areas
đĄ Recommendations
đ Priority Actions
DORA metrics remain valuableâbut they must be paired with system legibility analysis to provide a complete picture of engineering health.
A Balanced Measurement Approach:
â Track DORA metrics for organizational benchmarking and trend analysis â Layer in legibility metrics to understand how performance is achieved â Use MCP-powered analysis to detect concentration risk, coupling, and shortcuts â Create feedback loops between metrics and team practices â Celebrate sustainable performance, not just fast numbers
The Keypup MCP Server is available in beta and integrates with Claude Desktop, ChatGPT with custom GPTs, Cursor IDE, and Windsurf. Installation takes less than 5 minutes.
Quick Start Guide:
The MCP Server provides access to the same data harmonization and calculation engine that powers the Keypup web platform, but integrates directly into your AI-assisted workflow.
High DORA metrics are desirable, but only if they reflect genuinely sustainable, healthy engineering practices. When performance numbers mask concentration risk, mounting technical debt, and deteriorating system legibility, they become dangerous vanity metrics.
The solution isn't to abandon DORA metricsâit's to augment them with deeper analysis that reveals how your team achieves those numbers.
Keypup's MCP Server integration provides that missing layer, enabling natural language queries that surface hidden patterns, structural problems, and early warning signs. It transforms engineering metrics from a compliance exercise into a genuine tool for understanding and improving system health.
The question isn't whether your DORA metrics look good. The question is: Can your team sustain that performance? Do they understand the systems they're building? Is success dependent on a few heroes, or distributed across the team?
System legibility analysis answers those questions.
Absolutely. DORA metrics remain the industry standard for measuring software delivery performance at the organizational level. The issue isn't the metrics themselvesâit's treating them as sufficient when they're actually necessary but incomplete. Pair DORA with system legibility analysis for a complete picture.
Both use the same data engine and calculations. The web platform provides interactive dashboards ideal for executive reporting and team meetings. The MCP Server integrates into AI assistants, enabling ad-hoc queries and deeper exploration. Many teams use both: the platform for monitoring, the MCP server for investigation.
Start there. If your metrics show low deployment frequency, high lead times, or elevated change failure rates, those are your immediate priorities. System legibility analysis becomes critical once you're "performing well" on paper but things still feel brokenâthat's when metrics are most likely to mislead.
Yes, but it's much harder. Metrics like "author concentration" or "PR complexity trends" reflect structural properties of your system that can't be easily manipulated without actually improving the underlying situation. That said, no metric is perfectâthe goal is a portfolio of indicators that together tell a coherent story.
Most teams benefit from monthly reviews as part of retrospectives. Run concentration risk analysis before major team changes (new hires, departures, reorgs). Run complexity and review analyses when velocity feels mismatched with team sentiment. Treat it like security scanningâregular checks with deep dives when something feels off.
Start with author concentration analysis for merged PRs. It's simple to calculate, immediately actionable, and often reveals surprising concentration patterns. If 2-3 people account for >60% of merged work, you've found your first system legibility problem.
Frame it as risk management. High DORA metrics with low legibility represent single-point-of-failure risk. If key people leave, can the team maintain performance? System legibility is about sustainable performance and reduced business risk, not just making developers happy (though it does that too).
Yes, though concentration patterns are expected and acceptable in very small teams (< 5 engineers). The value is in tracking trends: are you becoming more or less concentrated over time? As the team grows, is knowledge distributing or concentrating further? System legibility scales with your organization.
Join teams already using AI to make data-driven decisions faster than ever.
Discover why measuring individual developer performance with raw metrics leads to gaming behavior and erodes team trust. Learn how Keypup's MCP Server provides contextual benchmarking that accounts for mentoring, architecture work, and team contributionâtransforming metrics from surveillance tools into culture-building insights that recognize real value beyond lines of code.
Discover how to overcome the most painful project management challenges in software development. Learn how Keypup's MCP Server harmonizes data across Jira, Trello, and Git repositories to provide actionable insights, predictive analytics, and real-time visibilityâtransforming chaos into clarity and helping teams deliver on time, every time.
Learn how to achieve and maintain ISO27001 and SOC2 Type II certifications for your software development organization. Discover the specific SDLC requirements, audit processes, and how Keypup's MCP Server provides continuous compliance monitoring, automated evidence collection, and real-time audit trailsâsaving months of preparation time and reducing certification costs by up to 60%.