FMCAD 2009
Formal Methods in Computer Aided Design
Austin, Texas, USA
November 15 - 18

Sunday Industrial Experience Report, 15. November, 16:00 - 17:00

Challenges and Opportunities in Deploying Enterprise-Wide Program Analysis Tools

John Penix, Google


Beginning in 2006, a small team within Google began deploying a system to automatically run static analysis tools whenever some part of the codebase changes and notify developers of potential problems. While the tools were able to identify problems in any given version of the code, providing accurate, timely results for a large, rapidly changing code base turned out to be extremely challenging.

I'll present the architecture of our tool automation infrastructure and discuss requirements on how fast and accurate analysis tools need to be to provide value in a large-scale fast-paced software development environment. I'll then describe how the system evolved during our various attempts at making improvements, including:

The general theme is that the experience of the end user makes a big difference if you want people to continue to use your tools over a long period of time. I'll finish with a summary of how various aspects of tool performance impact the end user experience and highlight areas where improvements would be valuable.


John Penix is a Senior Software Engineer on Google's Engineering Productivity team where he works on large-scale test automation and program analysis platforms. He was the technical lead for an enterprise-wide deployment of static program analysis tools including integration of analysis results into the developer workflow. Prior to joining Google, John was a Computer Scientist at NASA's Ames Research Center where he helped develop techniques to enable model checkers to verify properties of flight software. John received a Ph.D. in Computer Engineering from the University of Cincinnati. He is a member of the Steering Committee for the IEEE/ACM International Conference on Automated Software Engineering.