1. One can't find all the bugs
Software is an organic process and so is testing (sorry, no nice math proof that says you've found all the bugs). When bugs come back from the field, if the test team has a conscience, there is always a bit of remorse. How could we have caught this? Who dropped the ball? Why did this happen? Of the three questions, only the third is a valid one and only in the context of answering a better question: How do we prevent this from happening again?
Most organizations realize that mistakes happen. They have to. It's how organizations improve and grow. Repeating the same mistake is the true sin. Gear your processes to preventing the same brain-dead mistakes and you'll find the quality bar keeps going up.
2. Testing and Quality Assurance are not the same
We bandy these terms interchangeably, but stay clear on this:
- Testing is verifying (or not) whether code does what is expected based on some design in some form.
- Quality Assurance is testing plus challenging the design.
Quality Assurance is much more political. If you are testing alone, you are objective and don't have a stake in what is being tested. It could be a program deliberately designed poorly and as long as it works as intended it passes testing. Yes, it is exactly as poorly designed as the documentation said. However, in most cases, the software development team is looking to keep the quality bar high and welcome to a more quality assurance viewpoint. Valid feedback is welcomed.
A pitfall that new testers in an organization can have (whether they are new to testing or just new to the organization) is they don't get the culture of the organization and challenge at the wrong times. A good tester will listen and question before leading some pitched political fight. You can get the most contentious topics addressed in the form of a question, For example: "Bob's design is missing the point..." is much more flammable than "I thought the point the design was trying to address is..."
3. Pick your battles
Quality Assurance is about compromise. Hang out with a group of tech support people and you'll hear complaint after complaint that will make you think that software is developed by indifferent, clueless and "techie" people who don't "get" what people are like in the real world.
As a good QA person, you want the user experience to be as good as it can be. However, there is another side to this coin; some users can be easily confused and striving for the lowest common denominator can actually be a detriment. Is what you are fighting for good for the product? The team? The customer? Will you lose credibility next time you go to the well?
Politics is a big deal in testing. Rarely is it black and white, so listen well and weigh your odds of success. The best path is to make a solution that everyone can see as a win. However that is not always the option. Remember that if you have to play hard ball, play to win.
4. Automation As A House Of Cards
Automation is great when it works. That's the good news. The bad news? It is fragile. It is hard to maintain. It is easily outdated. It has unclear goals. It tries to do too much. It takes too long to develop. It is a hard sell to management as to savings potential.
It suffers the same issues as software development. It is hard to know if a tool can be purchased or whether developing your own is best. Technologies come and go out of style. If you can solve these problems, your automation should work great. Simple, huh?
5. How much testing do I do?
When you can't make the business case that shipping will significantly negatively impact the company bottom line it is time to go. There are some cool tools and techniques out there that can help you determine where you are.
Investigate pair-wise testing, model-based testing, and exploratory testing. None of these is a silver-bullet but used in proper context they can get you far down the road. Word of caution, some of these things have been hailed as the "be all" of testing. They fall in and out of fashion and can be completely cool or uncool depending on the shop you are in. Use what works for you and forget what others say. There is no tool or technique that is perfect. Only experience will tell you which way you ought to go. Sometimes manual testing with a checklist is all you need. Just keep it simple stupid and you'll be fine.
6. The only build that matters is the one you ship
All the process and metrics aside (and hopefully a nice quiet, predictable test cycle), the real build that counts is the one you ship. It is the one that makes the money. It is the one that makes your reputation.
I have seen absolutely garbage code suddenly come together very late in the cycle and I have seen stable, working code released with that one change at the last second that caused just one glaring cosmetic error such as an obvious typo which the whole world sees. Nobody notices all the work before that, just the error. The take-away here is the last build is the VERY worst time to be complacent.
|