What are the potential pitfalls of using deploy frequency as a productivity metric?

Using deploy frequency as a productivity metric can lead to a focus on quantity over quality, potentially discouraging larger, impactful projects. It may create a culture that prioritizes frequent small changes over substantial innovation.

The Allure of Deploy Frequency

Deploy frequency is an attractive metric because it's straightforward to measure. As Scott pointed out, "the tooling is easy we've already got the measurements and measuring deploys is about the easiest possible thing to measure right" ▶ 43:21. This simplicity makes it appealing for organizations looking to quantify productivity. The idea is that more frequent deploys can indicate a team is delivering more business value per unit of time ▶ 2:57.

The Risk of Prioritizing Quantity Over Quality

However, this metric can lead teams to prioritize the number of deploys over the quality or impact of their work. Brad highlighted a key concern: "when we set a marker of we're going to start measuring something, people are going to respond to it" ▶ 11:07. This can result in teams focusing on small, incremental changes to boost their deploy count, potentially at the expense of larger, more innovative projects. The fear is that teams might avoid tackling big, bold changes because they instinctively know these won't immediately boost their deploy numbers.

The Cultural Impact on Innovation

The emphasis on frequent deploys can also shape company culture. Scott compared this approach to favoring singles over home runs, suggesting it might lead to a culture that misses out on big opportunities ▶ 13:25. The analogy of Babe Ruth versus Ichiro illustrates the potential downside: while singles are safe and consistent, they might not drive the kind of innovation and growth that swinging for the fences can achieve.

Balancing Small and Big Deploys

Brad and Scott discussed the importance of balancing frequent small deploys with the capacity to undertake significant projects. Brad shared an anecdote from his time at Rover, where the team worked on a substantial feature over several sprints. Despite the challenge, they were ultimately satisfied with the outcome, demonstrating that big projects can be rewarding if managed well. The key is to ensure that the culture supports both frequent small deploys and the capacity for larger, more complex initiatives.

Communicating the Value of Deploy Frequency

One of the challenges with using deploy frequency as a metric is explaining its value to non-technical stakeholders. Scott noted that if stakeholders see a steady increase in deploy frequency, they might perceive the team as more productive. However, it's crucial to communicate that this metric should be part of a broader framework that includes quality and impact ▶ 17:51.

The Role of Predictability

Another aspect discussed was the importance of predictability in engineering work. Scott emphasized that "there's a lot of ways you can look at it, but just how good are you at predicting things are going to get done?" ▶ 6:13. This predictability is crucial for planning and setting realistic expectations with stakeholders, which deploy frequency alone may not adequately address.

Encouraging a Balanced Approach

Deploy frequency can be a useful metric, but it should not be the sole measure of productivity. It's essential to balance it with other metrics that capture the quality and impact of work. As Brad and Scott discussed, the goal is to create a culture that values both frequent small changes and the ability to execute significant projects. This balanced approach can help organizations avoid the pitfalls of focusing too narrowly on deploy frequency while still benefiting from its insights into team productivity.

Conclusion: A Balanced Approach

Ultimately, while deploy frequency offers a clear and easy-to-measure indicator of activity, it should be used in conjunction with other metrics to provide a more comprehensive view of productivity. By fostering a culture that values both frequent small deploys and the capacity for significant projects, organizations can ensure they are not only moving fast but also making meaningful progress.