Sunday Industrial Experience Report, 15. November, 16:00 - 17:00
Challenges and Opportunities in Deploying Enterprise-Wide Program Analysis Tools
Beginning in 2006, a small team within Google began deploying a system to automatically run static analysis tools whenever some part of the codebase changes and notify developers of potential problems. While the tools were able to identify problems in any given version of the code, providing accurate, timely results for a large, rapidly changing code base turned out to be extremely challenging.
I'll present the architecture of our tool automation infrastructure and discuss requirements on how fast and accurate analysis tools need to be to provide value in a large-scale fast-paced software development environment. I'll then describe how the system evolved during our various attempts at making improvements, including:
- Integrating the results into the existing developer workflow,
- Incorporating developer feedback to filter and prioritize warnings,
- Dealing with the many valid reasons that developers have to not care about defects,
- Improving the scalability of the analysis, including some unsolved issues
The general theme is that the experience of the end user makes a big difference if you want people to continue to use your tools over a long period of time. I'll finish with a summary of how various aspects of tool performance impact the end user experience and highlight areas where improvements would be valuable.
John Penix is a Senior Software Engineer on Google's Engineering Productivity team where he works on large-scale test automation and program analysis platforms. He was the technical lead for an enterprise-wide deployment of static program analysis tools including integration of analysis results into the developer workflow. Prior to joining Google, John was a Computer Scientist at NASA's Ames Research Center where he helped develop techniques to enable model checkers to verify properties of flight software. John received a Ph.D. in Computer Engineering from the University of Cincinnati. He is a member of the Steering Committee for the IEEE/ACM International Conference on Automated Software Engineering.