Automating iOS Native App Testing using Frank

One of the new movements in software engineering is BDD or Behavior Driven Development. In essence, the software behavior – as described by the various stakeholders – drives the development. Out of this movement and coming from the Ruby camp came Cucumber – a Ruby based tool for BDD. Out of Cucumber came Frank – a BDD approach to iOS native app testing. Pete Hodgson has a blog that describes Frank. There’s also a video on the blog page with a demo of Frank.

In a nutshell, Frank consists of 2 main parts:

  • A HTTP server running on the app or iPhone simulator to take in commands as JSON payloads
  • A Ruby library to perform the various test actions eg. search for text
Frank uses Cucumber to provide the BDD framework to interact with the Ruby library to control the app/simulator. Frank also uses UISpec code to perform the actual controls to the simulator and the CocoaHTTPServer as the HTTP server.

As most testers and test automation folks are more interested in using Frank as a tool to remote control an iOS app on the simulator for test automation, this article will cover mainly how to make JSON calls to Frank to run tests. If your interest is in using the Cucumber interface on Frank please consult Step 7 onwards on the Frank tutorial – https://github.com/moredip/Frank/blob/master/tutorial/Tutorial.md. You will need to install Cucumber as well.

Installing Xcode

You will need to have Xcode to compile Frank into your app. Steps to download Xcode can be found in my previous article
Simulating iPhone Safari Browser. Of course, this means you need a Mac to run.

Installing Frank

As I’ve mentioned in the article on Robotium I strongly believe in getting simple example code to work to use as a baseline.

  1. Download the Frank tarball from https://github.com/moredip/Frank
  2. Unzip the package
  3. In the unzipped folder you will find an “examples” folder. In there, you will find the “Employee Admin” example.
  4. Under the “EmployeeAdmin” folder, look for the EmployeeAdmin.xcodeproj and open it in Xcode
  5. On Xcode, first select the target executable to be “EmployeeAdmin Frankified” (see screenshot below) and then do a “Build and Run”. This compiles the project and the iOS simulator will pop up. The screenshot (see below) shows the app compiled as iPad app.
  6. Kill the simulator.
  7. Install cucumber and frank-cucumber. Frank and Cucumber come packaged as Ruby gems. The Ruby gem tool comes pre-installed in MacOS X so just open a terminal and do the following: gem install cucumber and then gem install frank-cucumber
  8. Navigate to the EmployeeAdmin folder using terminal. You should see a “features” folder there.
  9. Run cucumber by typing in “cucumber” in the terminal.
  10. You should see Frank go through the paces with the EmployeeAdmin app.

Note: the Frank team is not always good at ensuring their examples work. I had issues getting the example to work on v0.6.1 as well.

Choosing target executable

Choosing target executable

EmployeeAdmin App on iOS simulator

EmployeeAdmin App on iOS simulator

One more thing to note is that you need to enable accessibility features. There’s a link to a screencast on the Frank site. See https://github.com/moredip/Frank/wiki/Getting-started-with-Frank.

Integrating Frank with your iOS app

The instructions to do this can be found in the Frank site and it’s pretty comprehensive: https://github.com/moredip/Frank/blob/master/tutorial/Tutorial.md – Step 2 to 6 are the steps I used to integrate Frank to the app.

Once done, when you hit http://localhost:37265/ you should see the Symbiote UI (see screenshot).

Note: the Symbiote UI works for Firefox (with Firebug turned on) and on Chrome. It does not work on Safari.

The Symbiote UI

The Symbiote UI

Use the tool to identify the various UI elements by selecting the “Accessible Elements” tab on Symbiote. When you click on the items there, it will flash on the app on the simulator.

Making JSON calls to Frank

One good way to explore how the JSON calls work is to use a browser based plugin that is able to make HTTP Post calls to a given URL. I’ve found 2 that fit the bill:

  1. Advanced REST Client application for Chrome
  2. Poster for Firefox
Advanced REST client application

Advanced REST client application

Poster plug-in on Firefox

Poster plug-in on Firefox

The Advanced REST client application can be found in the Chrome Web Store and Poster can be found by searching on “Add-Ons” on Firefox.
The parameters that are needed are:
URL http://localhost:37265/map
Headers Host: localhost:37265

User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.19) Gecko/20110707 Firefox/3.6.19

Accept: application/json, text/javascript, */*

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip,deflate

Content-type application/x-www-form-urlencoded
X-Requested-With XMLHttpRequest
Method Post

The content to be sent is a JSON string eg.
{"query":"button label text:'Say Hello!'","operation":{"method_name":"touch","arguments":[]}}

The response is also a JSON string.

Structure of the JSON request

The JSON request has 2 parts:

  1. a query
  2. an operation
You can get a sense of the type of queries/operations available by looking at the UIScript examples – see http://code.google.com/p/uispec/wiki/Documentation#UIScript
You could also reverse engineer the possible queries and operation by looking at the Ruby code in https://github.com/moredip/Frank/blob/master/gem/lib/frank-cucumber/frank_helper.rb. For example:

matches = frankly_map( query, 'isHidden' )

means to construct the JSON string with “query” as the query and the operation ‘isHidden’.

Structure of the JSON response

This is an example of a Frank JSON response:
{"results":["Hello World!!!"],"outcome":"SUCCESS"}

in response to the following JSON request:
{"query":"view text:'Hello World!!!'","operation":{"method_name":"text","arguments":[]}}

The query is to search for text “Hello World!!!” and perform an operation to pick up the “text” field of the list of UI objects returned. Hence if you look at the JSON response, the result is a JSON array. In this case the list is non-empty and hence the text was found.

The outcome field is to report if the operation passed or failed. Failed would be due to some internal error – not because element not found.

 

Advertisements

2 Comments

Filed under iOS, Native App, QA, Testing

2 responses to “Automating iOS Native App Testing using Frank

  1. kannan

    Good one. Thanks

  2. Amit

    I automated this and worked perfectly
    Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s