How can engineering teams effectively measure productivity beyond traditional metrics?

Engineering teams can measure productivity by focusing on deploy frequency and business value delivered, rather than traditional metrics like lines of code. This approach encourages smaller, more frequent releases, which can lead to greater efficiency and stakeholder satisfaction.

Moving Beyond Traditional Metrics

Traditional metrics like lines of code or story points often fail to capture the true productivity of engineering teams. Scott argues that the key is to measure productivity in terms of business value delivered per unit of time. He suggests that if teams are doing this right, they should be delivering more business value efficiently: "So if we're doing that the right way, then you would see that your team is delivering more business value per unit of time." ▶ 2:57

Brad and Scott discuss the limitations of using deploy frequency alone as a metric. While it's a straightforward measure, it needs to be balanced with quality indicators like incident rates and bug counts. The aim is not just to deploy often but to ensure that these deployments are valuable and maintain quality.

The Power of Smaller Deployments

One of the main advantages of focusing on deploy frequency is that it encourages smaller, more manageable changes. Scott emphasizes that smaller deployments can lead to significant velocity gains: "If you count the number of deploys, that will incentivize people to deploy more frequently, which means they'll have to do smaller things." ▶ 7:17 This approach aligns with the idea that smaller changes are easier to test, understand, and roll back if necessary.

Brad shares an anecdote about his experience with a big feature at Rover, where the team worked over several sprints. They managed to break down the work into smaller parts using feature flags, allowing them to demonstrate progress regularly. This not only kept the team motivated but also maintained stakeholder engagement throughout the project.

Balancing Speed and Quality

While increasing deploy frequency can lead to faster delivery, it's essential to maintain a balance with quality. Brad points out that focusing solely on speed can lead to "bullshit deploys" where the quality of the code suffers. The key is to ensure that the team is not just deploying quickly but also maintaining high standards of code quality and system stability.

Scott emphasizes the importance of predictability in engineering work. He suggests that teams should aim for a high level of estimation accuracy, which helps in delivering projects on time and building trust with stakeholders. "There's a lot of ways you can look at it, but just how good are you at predicting things are going to get done?" ▶ 6:13

Engaging Stakeholders with Demos

Regular demos are a powerful tool for engaging stakeholders and demonstrating progress. Brad recalls a successful example from Rover, where not only the engineering team but also marketing and operations participated in demos. This holistic approach ensured that all parts of the organization were aligned and contributed to the project's success: "Not only was the engineering team demoing what was getting built, there was a bunch of other stuff that had to happen." ▶ 27:27

Scott shares a similar experience where regular dress rehearsals for demos helped the team identify gaps and improve their presentations. This practice not only improved the quality of the demos but also boosted team morale and stakeholder confidence.

The Role of Predictability and Estimation

Predictability in delivering engineering projects is crucial for building trust with stakeholders. Scott highlights the importance of estimation accuracy, which allows teams to deliver projects on time. He notes, "predictable and then we can talk about ways to become faster and i 100" ▶ 31:20. This predictability helps in setting realistic expectations and ensures that stakeholders are kept informed about the progress.

Brad and Scott also discuss the potential pitfalls of focusing too much on deploy frequency. While it can drive smaller and more frequent releases, there's a risk of teams avoiding larger, more impactful projects due to the perceived complexity. Brad suggests that it's essential to balance this by encouraging teams to also tackle big projects, breaking them down into smaller, manageable parts.

Conclusion

Measuring engineering productivity requires a shift from traditional metrics to a focus on deploy frequency and business value. By encouraging smaller, more frequent deployments and maintaining quality, teams can improve efficiency and stakeholder satisfaction. Regular demos further enhance this approach by providing tangible evidence of progress and fostering a collaborative environment across the organization. Ultimately, the goal is to create a sense of momentum and progress that is communicated effectively to all stakeholders.