Learning to test with JankariTech

People visit Pokhara because it is the starting point for many treks to the himalayas in Nepal. However, the Parewa Labs team was there for three days to conquer a different mountain.

Our Learn Python app was getting bigger and bigger, with users constantly requesting features: from support for other programming languages to dark mode. Our own roadmap contained quizzes and programming challenges.

However, we know from experience that as we build more features, there is a risk that old features can break. We don't want our users to be stuck with an app that doesn't work. We turned to JankariTech, an IT company located in Pokhara that specializes in helping companies build rock-solid products.

After all, we are teaching people how to code. If our own code isn't good, we are not being honest with our users.

JankariTech were experts at testing webapps. Even though they had never done automated testing on mobile apps before, they said they were ready for the challenge.

A few meetings and hotel bookings and flight tickets later, the trip was on.

Day 1: Brainstorming

We reached Pokhara at about 10am. The view from the airport was majestic as always. We were ready for an awesome 3 days of learning.

After reaching JankariTech's office and a round of Namaste's, we started a brainstorming session.

We asked and answered questions like?

• What can be achieved in 3 days?
• What are the low-hanging fruits here?
• What would be a nice demo at the end of the 3 days that shows our progress?

As with any brainstorming session, a lot of ideas were thrown around. Artur, the co-founder of JankariTech had looked into Flutter and automated testing with Flutter. As someone who swears by Behavior Driven Development (BDD), he had looked into the tools that would allow us to integrate BDD into our workflow and build a solid foundation for the next days to come.

He showed us a demo of some tests he had written for the "Hello World" app in flutter- a counter app with a "+" button that would increase the counter at the middle of the screen. He'd even added a "-" button and written some tests for both the "add" and "subtract" features.

Till now, we had been writing a lot of unit tests, and they had definitely reduced recurrence of bugs, but when he connected his mobile to the laptop, tapped away on his keyboard and with a programmer's proud "Enter" tap, showed us how we could see the tests running on his mobile visually, we were sold. This was what we wanted, being able to visually see our tests running on actual devices.

Artur and his team at JankariTech are experts at Test Automation and there was a lot we could learn, so they organized a short presentation for us where Artur talked about Martin Fowler's test pyramid.

Artur explained to us that we should write as many unit tests as possible because service tests and UI tests are slow, and they are costly. Adding unit tests in the code early in the development allows developers to save time later when they write service and UI tests.

We had a productive discussion around the merits of going the opposite way. Our app already had a lot of unit tests, but most of the bugs were appearing in interfaces between two screens. Adding UI tests or widget tests as they are called in the Flutter ecosystem would allow us to test the UI flows. One another advantage of UI tests is that a well written UI test ends up testing all the services and functions underneath.

Then Artur explained the anatomy of a gherkin feature- a language that allows businesspeople to write pseudocodish 'feature files' that clearly explain what each feature in the app should be able to do.

Feature: Serve Coffee
In order to run our cafe, Customers should be able to order coffee

Scenario: Buy coffee
Given there is 200gm of coffee beans left in the machine
And 1 coffee cup needs 15gm of coffee beans
When I press the coffee button
Then I should be served a coffee

Scenario: Machine out of coffee
Given there is 10 of coffee beans left in the machine
And 1 coffee cup needs 15gm of coffee beans
When I press the coffee button
Then Machine should show an error message `Out of Coffee`

We collaboratively wrote a feature file together with Artur testing and correcting our assumptions along the way. This was fun because everyone could be involved- our designer, management, project managers and the people who would eventually write the code.

Day 1 ended with a quiet dinner by the Lakeside. We were excited about what Day 2 was going to bring.

Day 2: Getting stuck and unstuck

On Day 2, our daily standup was intense. We had to choose which tests we would write during this trip and which we would leave for later.

Should we write shallow tests that covered a lot of features or go in-depth on just one feature and write unit tests, integration tests and UI tests that we could use as a base for other tests?

Both approaches had their benefits. Writing shallow tests for a whole workflow would make a killer demo when we returned back to Kathmandu, while having detailed tests for one feature would give us the framework for all our testing we do later.

The confidence from the previous day probably made us ambitious, and we decided to do a bit of both. We would completely test the login feature but also include tests that do basic tests for the lesson workflow.

Our little brainstorming session ended with a list of haves/wants and we derived the targets from those.

It was time to write the tests, but it turns out our optimism was misplaced.

Is the code testable?

We soon realized that testing the Login feature was going to be challenging. Right now, the only way to test the feature was to use an actual device that had the app running. We put in this kind of system for security but we knew it would slow us down.

We needed a way to create new users without the need for a mobile phone, run tests on our login system using these 'sample' users and delete them after the tests were done.

Once this process was set up, it was only a matter of writing feature files in gherkin syntax like the coffee dispenser example we saw above.

We also wanted to do this on a fresh system. In the long run, our plan is to test the code on various devices on every release. For that to happen, we had to have a seamless process for setting up the environment for our code and our tests and this was an opportunity to see if we were there yet.

Setting up the new system brought lots of challenges. There were version conflicts between different versions of Flutter and certain dependencies. Stackoverflow was no help. We gave up with a lesson that we needed a better setup progress and went back to writing tests on the existing systems that had the working conditions for the app and the tests.

The team spent a considerable amount of time trying to get the login tests in with as much detail at each level as possible. We wrote unit tests, integration tests and UI tests that captured our intentions and user workflows.

Artur also told us that in future, we could use the tests to make sure that any update to Flutter doesn't break our app. We could upgrade to the latest version of Flutter, run all the tests and rollback to the previous version if the upgrade to the newer version broke anything. Then we would fix the build to handle the breaking changes in the new Flutter version and release it.

This was definitely a step in the right direction but a major time-hogger. Would we reach the goals we had set for us at the beginning of the trip and on today's standup? Read on to find out

P.S. No matter how frustrating your day is, the beautiful Pokhara Lakeside and the mountain ranges that surround it are always there to calm you.

Day 3: Retrospective

We started Day 3 by being welcomed by the most beautiful view of Machhapuchhre. The weather was absolutely gorgeous today and we were looking forward to reaching our goal of having a set of unit, integration and UI tests that would set the foundation for thorough testing of our app once we got back to the Programiz office in Kathmandu.

The blocking login automation was done and we started writing feature files for other workflows: lesson swiping, bookmarking, rewarded ads on our compiler etc. Deepak from the JankariTech team was always with us to question if we were writing the scenarios correctly.

We had lunch that lasted a bit longer than we had expected. Blame it on the view of the majestic Annapurna range and the beautiful weather on the day. By this time we were sold on the idea of starting something in Pokhara. Expect another blogpost when that becomes a reality soon.

Having feedback from the JankariTech team on these features was amazing, after all they do this full time and know where common mistakes can arise. We were much more confident about the fact that we could go back home and finish what we started.

The last few hours were spent in a retrospective. We would have loved to have more tests, but the whole process of BDD was very new to us. It turns out integration and UI tests look very simple on the outside, but there were lots of gotchas that we had to overcome.

Also, many of the interactions we were simulating in our test suite were stuff we had never done before, so we were constantly reading the Flutter documentation to see how they could be implemented. We will be faster now that we have many of the interactions in the app in the login flow tests that we wrote.

The 3 days just flew by and it was time to say goodbye to JankariTech and Pokhara. No trip is complete without a group photo. The Programiz team would like to thank JankariTech for their hospitality and teachings and Truenary Solutions for all the hours they've spent developing our app.

Parting Words

When you work with elite teams, stuff just gets done. This was the biggest lesson of all for us. Under the mentorship of the JankariTech team, we were now able to

• Write feature files in gherkin syntax to better describe the workflows that we wanted in the app
• Perform unit, integration and UI tests on our application code
• Visually see the test code being run on any device of our choice
• Understand the importance of the test pyramid and how writing better unit and integration tests can help us write lesser UI tests and run our test suite faster

Most importantly, we now saw Pokhara as a genuine future hub of the IT industry in Nepal. Until next time Pokhara! Your majestic mountains will be missed.