Sometimes, you may come up with the idea to shorten time of your tests execution by preparing your APP to be in the testable state for the moment when the test starts. This totally makes sense, especially in mobile UI automation which is incredibly slow. Some people call this approach “Power over purity”. We will review pros and cons in this article.
By preparing the APP to be in the testable state I mean calling APP’s methods from the test. There can be different methods: opening screen under the test, performing fetching, putting data in APP’s database or login/logout. Someone will argue, that automation should use the APP in the same way as the user does. But firstly, you can’t automate everything. So, why don’t you give yourself more possibilities by calling APP’s methods? Secondly, you can combine calling APP’s methods and user interactions to find a golden medium. But about this later.
Using APP’s methods you will drastically decrease your tests execution time. Also, this will exclude the possibility of random failures while preparing the data. Last but not least, it is possible to automate cases which are impossible to automate using regular UI methods.
You have a risk to miss some issues, especially when calling APP’s methods without a plan. And you should implement the methods you will call in both: the APP and the automation itself.
After some discussions with colleagues I decided to propose a compromise. It sounds like: “We will use methods that put the APP in testable state, but also will have a small subset of tests which will cover transitions that we can miss while using the APP’s methods”
Even using the hybrid method you have a risk of overusing the method and getting less coverage instead. In order not to let it happen you will need to review every new automation procedure that uses the APP’s method. This will help you to analyze the cases you can miss and the cases to cover on top of that.
The way it looks in the real project
Mostly I use APP’s methods in two cases: The first is login and logout. This saves a lot of time, as you do login/logout in every test. The second one is calling APP’s routes, as this allows you to get in any part of the APP instantly, as routes directly get you in the section you need. But, I created separate test suites for login, logout and transitions through the APP, as these two shortcuts don’t follow the user flow and have a risk of concealing issues there.
Other cases where I use APP’s methods have nothing to do with the user flows. Those just extend the possibilities of automation. For example, you may need to perform background fetching without waiting for 15-30 minutes or reset some flags in database, that will allow you to repeat testing without re-installing the APP or creating another user. You will not write any additional tests for these cases, as they are just additional checks which you can’t achieve using normal user flows.
A very simple example of using APP’s methods you can find here. This mobile automation sample project is explained in this article. If you will dive deeper in the colour change step, you will find this code:
This backdoor will call
changeColor method from the APP and the colour of the label will change without introducing the number and clicking the UI button.
For sure this simple example will not save a lot of time, but it should give you a bit of understanding how you can use methods from your APP.
Note: If you are interested in Calabash backdoors, you can find the description here.
You will decide by yourself whether you prefer purity or better coverage. There are advantages and disadvantages in both cases, but the final decision will totally depend on your team decision. I can only suggest the hybrid method, trying to use advantages of both approaches.