& Continuous Learning

This is part of our Quality Assurance (QA) Community of Practice’s testing collection. The goals of this collection are to spark ideas for teams without dedicated testers, to have open discussions about testing, and to share information on testing topics that we’re passionate about.

Today we will be focusing on automated smoke tests (often called acceptance tests), which can live pretty much anywhere. Most delivery pipelines are super flexible and can grab any repo, install requirements, and fire off your automated tests. While automated smoke tests can live anywhere, often the decision as to where is a mix of culture and convenience. Let's take a look at a couple of common places they can exist and understand when and where they should be placed.

Case 1: Automation Code Should be Separate from the Application Code

Let’s say your group is in charge of testing 3 separate applications that sometimes interact with each other (as in the user can hop from one to the other and what they do in one app may affect their experience in another app).
A developer might want User Interface (UI) test scenarios that touch all three applications. Putting the test automation code from these three apps in a single, separate repo would allow you to more easily write UI tests that will crawl across these 1-3 apps. Your tests can use the supporting automation code (page objects) to do this. Separating out the tests in this way has the added benefit of producing no duplication of code. Apps 1, 2, and 3 each have lots of automation code in the form of page objects (or something similar) and all of them are available to the test creator.

This minimization of duplicated code is the biggest reason to keep your automated test code separate from the application code. It is a decision based on the need to accommodate a specific type of testing: broad end-to-end tests.

Now let's look at the flip side. Writing UI tests so broadly is not very good practice. Those tests will be:

1. too long
2. too brittle
3. so coupled they have common law status
4. inefficient

Because of their nature, these types of tests are rarely found in a delivery pipeline, because nobody will trust them to reliably pass. And here a major bias is unearthed - the most useful test automation runs in the pipeline. I’m not saying tests that run outside the pipeline have no value, but they do have less value. Tests that pass in a pipeline allow for trust in an application and the new code that is being deployed. If a developer makes major changes and they see the pipeline stay green, they know the new code meets current best practices. The caveat to this is that this is only true as long as there are some helpful tests being ran.

Case 2: Automation Code Should Live with the Application Code

If your team or organization believes that test automation is important, it is not that much of a leap to then think test automation code is part of the application. Unit tests are considered part of the application, so it is logical that your higher order automation could also attain this status. The main difference here is that your API or UI tests are usually testing a deployed application. If your group has agreed that you don’t want the application getting deployed further down the pipeline toward production until it has passed some kind of live testing (acceptance or smoke tests in this example) then it seems like we’re treating the automated tests as if they really are part of the application.

This is where your team or organization’s culture now becomes a main factor in deciding where to put your tests. Do the developers want to run the tests against localhost? You can accomplish that if need be. Putting the automation test code in the application repo sends the message, “This all needs to work before we can confidently push this application to production.” And now the whole team has a vested interest in having automated tests that are reliable and provide fast feedback. Pushing code to production can be scary but if you trust your automated smoke tests it just becomes a normal part of the day.

For more information in support of Case 2, check out this article on 6 reasons to co-locate your app and automation code.