Back in late August/early September I spoke at the Rocky Mountain Ruby conference in Boulder, Colorado. My talk was about lessons learned dealing with large functional test suites over my career. For example, at a previous job I ran a team that got up to 9,000 Selenium tests scattered across roughly 850 test fixtures. I’ve had similar experiences working on other projects too.
The talk was videotaped and is now up on the Confreaks video hosting site: Surviving Growing from Zero to 15,000 Selenium Tests. Yes, the talk’s title is wrong. I goofed when submitting it.
The fundamentals I lay out in this talk span all test tools and frameworks, so it doesn’t matter if you're writing your tests in Selenium or Watir, or using QTP or Visual Studio. Early in your automation effort you’ve got to address basic problems such as long-running suites, brittle tests, and focusing on automating only the most valuable, critical aspects of your system.
Of course Test Studio magically resolves all these issues for you and you’ll never have to think twice about any of these problems. OF COURSE WE DON’T DO MAGIC! I’m proud to be part of a great team producing a terrific tool, but you absolutely have to think ahead on these same sorts of issues! (We do, however, do a great job of helping you navigate these areas safely!)
This talk was the genesis for my “Automation Isn’t Shiny Toys” talk I’m giving a number of times over the next few months. I’ll be writing up a number of blog posts around this both here at my Telerik blog as well as more generalized posts on my personal blog.
I’d also recommend you watch the Testing Panel discussion from the same conference. I sat on that panel with three other really smart folks and there’s a lot of tremendously useful information on it.
Jim Holmes has around 25 years IT experience. He is co-author of "Windows Developer Power Tools" and Chief Cat Herder of the CodeMash Conference. He's a blogger and evangelist for Telerik’s Test Studio, an awesome set of tools to help teams deliver better software. Find him as @aJimHolmes on Twitter.