Zombie lands: Selenium recorders

    Sometimes even best intents and plans may end up being just that. I’m not going to emphasize how much of a bad idea is to rely on capture replay tools, codeless tests and code generators in your automation testing strategy. Especially in a product company with long lived platform and production environments. Enough said. But those companies are not the only ones that you may work as a QA engineer. Some of them provide software services, virtual teams or whatever marketing calls the outsourcing in order to be different. Not better. This has its own advantages, such as vast variety of technologies and teams to work with. Experience matters. Also, boring testing stuff is really a rare sight.

    I will try to stay on the business side for this post, even if this blog is technically oriented. So, what is the main purpose of having QAs on your project? Is it about testing developers code? Automating tests? Or infrastructure? Reporting? Someone to ask why we have bugs released and given to the client? None of this really matters when you realize that QAs don’t assure quality.

    Yes, you heard me saying it.

    QAs simply report your software’s health at the end of the day. That’s it. They don’t fix bugs or implement your new feature. Why I said it!? The quality should be build in, not bolted on. Like it or not, quality is responsibility of all of us. Yes Mr. PM – yours too. There is a slim chance that your team is truly cross-functional and every team member can finish any simple backlog task (regardless of its type). If this is your case – Congrats! You’ve made it to the Big league. What happens if you, as a QA, work in a company where the majority of projects are short-lived, no more than 3 months, with no real change requests or production environment to support? In the same time is quite possible that you are also responsible for a projects that are the exact opposite. This is normal – business wants to make good use of your skills. Not taking “small” projects seriously, is a poor decision. Just their budget is different. Automation done right is expensive luxury, few can afford. Being agile, even if your environment is not, should be a must. After all, our responsibilities for both types of projects are the same.

    As advocates of the end user, we should understand what the core business is all about. And wishfully make the best in our efforts to assure its delivered with acceptable quality. Having the golden hummer of automation, sometimes we desire to bend everything with it. Is it really profitable to have complex and powerful framework and setup, if no one else (besides you) is willing to learn, use and maintain it? Good chances are you will find such people in time, but the project’s deadline is there, now. It is important to deliver and in many cases (I’ve witnessed) quality is the first priority that is cut down. And this is real life story for many of us.

    What are the alternatives!? Hate to admit it, but my best course action will be a Selenium recorder. I do think there is a perfect storm, when this makes sense. I know how many of you will stop reading after this line, but bare with me. Consider this:

• only one Manual QA, no coding skills what so ever
• website with little or no 3rd party integrations
• fairly straight-forward business flow
• static web pages, as templates are prepared long before back-end coding kicks in
• reviews were made and layout was approved as final
• no QA environments, just single Staging one
• no meaningful CI setup
• no real time for full cross- browser and platform runs
• lack of mature font-end unit testing culture
• short testing frame (couple of weeks top, out of budget with months dedicated for development), this was already agreed with the client and he is happy

    If all of the above present, I would go with a Selenium recorder. Every automation test case should justify its price, at any given time. I prefer 70% of the functionality covered by capture-replay tests over 20% created via complex UI testing framework and setup. Code coverage is always a sensitive topic. But if you keep the test pyramid you should be fine. I know what some of you may say:

– If your framework was that good, a manual QA can create the tests on the BDD level. Simple as that.

    Fair enough, but if your framework is designed well, your tests shouldn’t rely on imperative BDD, right? Ok, I lost another five good QAs reading this post. What I am trying to convince you (and myself) is that sometimes mindless tests are not so evil. No one can deny that recorded tests are cheap. Easy to create and execute. Everyone can do it. No complex setup is required. Plug and play solutions have their place under the sun. I will use a good practice from the DevOps, which states that creating new environment should be easier than fixing the existing one. Do you see how this is applicable in our context? The recorded tests should not be fixed! This is not in their nature. They should be used only until fit. The moment you start adding configurations to them, you better start using your complex, layered test framework. It is better suited and designed for this anyway. Don’t fall into the trap of adding too much gun powder to your Selenium IDE setup. I know how tempting is to add sugary plugins, but they only hide the problem. If you need more complex tests, recorders are not for this project. Same goes for Selenium Builder. I don’t consider this as a retrograde way of thinking, just being pragmatic.

    As bottom line, I would like to go back to the roots of automation – it is designed to free time and assist your testing. Having recorded scripts will save you the boring and repetitive tests for the manual QA staff. And they will have a bit extra time for the more important exploratory testing.