Home | MindEngine.net

Testing Front-end in isolation

For quite a long time (since 2007) I am working on test automation. GUI testing was a huge part of that and it was always something I never really enjoyed. I did have a reason, so as many of my coleagues, due to a lot of facts. It is not only hard to get a green build in our CI pipeline but it is also nearly impossible to execute all of the tests locally before commiting the code. At the moment I am working at eBay/Marktplaats and we have a pretty large set of Selenium tests (around 2500). The website itself (http://marktplaats.nl) is built using Service Oriented Architecture and we are trying to move towards using microservices but that is still a long way. At the moment our website consists of 5 front-end applications and around 75 back-end services. At the same time there are a lot of different databases. In some cases services share the same database and in some case they have a dedicated one. Unfortunately, when it comes to a Selenium test, it is always implemented as an integration test, meaning that all services and databases are supposed to be up and running. Due to the fact of having around 40 enginers in 6 teams working on the same codebase it is hard to get a green build. People often push broken code and, since front-end depends on every service, a small bug somewhere deep in a specific service can break the whole flow.

At some point I decided to rethink the UI testing strategy and started to look for something else. I had to find an answer to a lot of questions. What do we really want to test with Selenium? Can we reduce the scope of Selenium tests, make them run 2-3 seconds on average per test and drastically reduce the amount of flakiness in our CI? Do we really want a layout test to break due to a bug in back-end service? Can we make Selenium tests just part of the front-end repo without maintaining separate tests and stubs repos? And last but not least: can we make executing tests locally easier for everyone in the team?

Quite often I am asked the same question: "how do I run Galen in "SauceLabs?". Finally I got some time to write a complete tutorial. SauceLabs is a great service allowing you to easily test on various platforms and browsers including Android, iOS, Window 8.1 etc. I was surprised how easy it was for me to just reconfigure my Galen tests and point them to the SauceLabs Selenium Grid. Well, maybe not that easy, as there some little tricks and that is why I am writting this tutorial.

Writing Galen tests with Javascript

In version 1.0 a major change was introduced to execution and reporting system in Galen Framework. From now on it is possible to write and manage all your tests in Javascript. So what would be the difference compared to the original basic runner? What has changed is that you don’t need *.test files anymore. Instead you define your tests in javascript files with *.test.js suffix. Javascript gives you freedom to construct your own way of definining tests. Lets take a look at the simplest test:

test("Home page test", function () {
    // this is the body of the test

    // Instantiating the WebDriver, loading the page and changing the size of the browser window
    var driver = createDriver("http://testapp.galenframework.com", "1024x768");

    // Checking layout on the page
    checkLayout(driver, "specs/homepage.spec", "desktop");

    // Quiting the browser
    driver.quit();
});

There is one more thing that is possible to test with Galen Framework except layout. Since version 0.10.1 Galen Framework introduced a convenient way to test internatiolization. According to Galen official documentation you can test text for specific items on page. But what to do if there are multiple languages on the website and you want to test all of them? Here is a solution.

Testing colors on your website with Galen Framework

In previous articles I have been writing about layout testing in Galen Framework but there is also one field left unexplored – color testing. As of version 0.7.2 Galen Framework has a so called "color scheme" spec which allows you to check the color distribution for a specific element on page. Why would you need this? For example lets say you have a special form and by requirements you have to make it grayed-out for unregistered users. How would you check that with automated tests? With Galen Framework now you can do it by taking a screenshot and then checking color distribution on the form. Of course it is still far away from ideal visual verification but at least it is better than just verifying the CSS classes on the elements. Also for the moment this works best only in Firefox as there are some issues with taking scheenshots in Chrome and IE.

If you have version older than 0.7.2 please download the latest version of Galen Framework here
Lets see how we could use this spec...

Configuring Galen Framework for Windows

I have received a complain that Galen Framework is not very suitable for Windows so I had to introduce a simple batch file galen.bat. Also with this article I decided to make a complete guide for configuring Galen Framework for Windows 7 and for Windows XP. So if you are using Windows operating system and you have troubles with configuring Galen Framework please read this article

Automated layout testing for website was always a question if it is at all possible. When Selenium rose in its popularity a lot of people tried to approach it for layout testing but more and more were stating that Selenium is not the right tool for it. Well, I still think it is the best tool ever for layout testing. And thats what I tried to prove with Galen Framework. In my mind the best way to test layout is by checking each individual element on page relatively to another elements. Selenium WebDriver offers functionality for getting location and dimension of element and it is doing this quite good in all browsers. Galen Framework allows you to express you expectations towards website layout and then uses Selenium to retrieve information about page elements. Lets see how to get the best coverage for layout testing.

GalenJs. Unleash the power of javascript in Galen Framework

In my previous posts I wrote about layout testing in Galen Framework. But what if we need to perform some actions on the browser to get to some specific page? For instance lets say we have to test a blog website. And to do it properly we should come up with a realiable automation so we need to post an article on the website so we can later open it and check the layout. To perform all these complex actions Galen Framework provides an action called run. It is used in order to invoke a javascript where you can take over the WebDriver and perform some actions on the page like clicking, typing text and basically anything that WebDriver allows.

For the demo purpose lets use this page
What we need to do is to log in using the form on the page. Lets start writting a basic test. First of all we need to download the latest galenjs.js file. You can get it from GitHub repository. Just put this file in your test project folder. Now lets create a login.js file.

// Include the GalenJs library
load("galenjs.js");

// Create a GalenJs instance from driver
var $ = GalenJs.create(driver);

// Create a login page object
var loginPage = $.page({
    email:      "#login",
    password:   "#password",
    submit:     ".login-form a.button"
});

// Perform login flow
loginPage.email.typeText("testuser");
loginPage.password.typeText("P@ssw0rd");
loginPage.submit.click();

That’s it. Now we can execute this script in our test suite. Lets create a file my.test

User profile test
    http://samples.galenframework.com/tutorial-selenium-interaction/tutorial.html 1024x768
        run login.js

Now you can run it with this command:

galen test my.test --htmlreport reports

So this is the basic stuff that you can do with GalenJs. As you see it is quite usefull since it allows to write a better code in javascript with minimal effort. And what is more important it allows you to get to any resource on your website so you can also test its layout with Galen Framework.

Cross-browser layout testing with Galen Framework

In the previous post TDD for Responsive Design I have introduced you the Galen Framework - the tool which is very useful when it comes to automating layout testing for responsive website. In this post I would like to show you how to use Galen Framework for testing cross-browser compatibility. We all know when it comes to a website development it is a hell to support it in different browsers. With every little change on the website there is always a chance of breaking the layout in some other browser. And the sooner you find it – the better. Lets see how to automate that using Galen Framework together with Selenium Grid.