Mobile Business Application Performance

Posted by Phil Garton

[This blog post is the first attempt by our Enterprise Mobility team to consolidate some thoughts on mobile application performance  in general and to arrive at some kind of a matrix for acceptable performance levels & desired performance levels. This by no means is a complete reference document. We expect our research will continue as the parameters of the mobile market continues to change.]

1 Introduction

This document will discuss general guidelines for acceptable mobile application performance based on traditional web design teachings and user based perception of mobile app performance of advanced smart phone users. This information will be analyzed with actual mobile application performance of apps available in the market place.

1.1 Mobile Application Performance

How well a mobile application is received by the user community can be broken down into just a few key areas.

  1. Look and feel – is the app intuitive and easy to navigate.
  2. Does the app do what it says it does?
  3. How does the app perform?

We have guidelines from each of the major OS’s that give us direction on look and feel which aids us in defining acceptable navigation and user interface. And weather an app does what it says it does is pretty straight forward and easy to measure.  But how do we look at app performance, and how do we measure it, and what is acceptable?

The key aspects of mobile application performance can be broken down into these areas:

  1. How long does it take the app to launch?
    1. Measured in milliseconds or seconds
  2. How long does it take for the app to display information?
    1. Measured in milliseconds or seconds
  3. How does the app perform when switching between in app screens?
    1. Measured in milliseconds or seconds
    2. Feel or user perception is also important
  4. How does the app perform when navigating through the presented information?
    1. Measured in app memory usage over time
    2. Feel or user perception is also important

We will be looking at this from two prospective first that of the user and what they would deem acceptable based on trends and mobile application experience. Then secondly base on mobile app market place and measured performance.

This information will then be used to summarize an application performance guideline that is factual driven to meet user expectation levels.

2 User Expectations

2.1 Informed User Expectations

The smartphone user community has preconceived expectations on how mobile apps should perform based upon past user experiences as well as exposure through web based systems. The smartphone has been around in some form or another for the last 12 years, giving most users enough interaction with mobile applications to develop an expected level of performance. In an addition to this a large percentage of this user population uses or has used the web or web applications and the expectation has been set that a mobile app should perform as good as or better than a web page or web application.

The user community for mobile applications expectations are defined by the environments that they interact with on a regular basis, at work systems, web based systems and other mobile applications. From a user perspective mobile applications and web based applications are not that dissimilar and can be held to similar levels of expectations.

2.2 Web Design Guidelines

The web has been around for 23 years since first introduced. Throughout this time there have been many document studies and teachings of web design methods that increase traffic flow and return visits. These ideas and guidelines have evolved as the web has. The follow is a sample of guidelines that come to be adopted as general guidelines for building effective web pages.

Looking at web design guidelines we are able to allude to key items that a user would consider while visiting any given web page while searching the web. Performance, navigation, content, and appearance all aid in the acceptance to the usability and likely hood of a user returning to a web page. When we consider performance we can break it down into four guidelines that will impact a user’s perception of a web page performing to an acceptable level of performance.

  1. Web site should load in 3-5 seconds
  2. User should be able to reach desired information within 3 levels of the website (3 click rule)
  3. Navigation between pages should load in 1-2 seconds
  4. When interacting with page information indicator should be displayed results in 3-5 seconds

These four major guidelines of web design can teach us a lot about user’s expectations. These strategies were formed based on user research to improve return visits to increase traffic flow. Web designers and companies alike found out quickly that if a user could obtain site information quickly and without having to search deeply into a website, the user was 1 more likely to make a purchase and 2 was more likely to return to that site in the future.

2.3 Applying Web Ideology to Mobile Apps

With user feedback, site statics and analytics we are able conclude that the web design guidelines are affective and can be used to give us similar footing with the mobile space. When we apply this information to mobile applications as expectation levels and we consider our areas of interest in mobile app performance we can better understand the user’s expectations.

  1. How long does it take the app to launch?
    1. User Expectation – 3-5 seconds
    2. How long does it take for the app to display information?
      1. User Expectation – 3-5 seconds
      2. How does the app perform when switching between in app screens?
        1. User Expectation – 1-2 seconds
        2. How does the app perform when navigating through the presented information?
          1. User Expectation – 3-5 seconds if longer indicator should be displayed

3 Mobile App Performance

3.1 Market Place Performance

Now that we understand the root of user expectations and that we have applied them to create a user expectation guideline, we can assume that these user based expectations would be used for evaluating the apps that are available in the market place.

Polling the business mobile app market place for top rated mobile applications within the same application type we can determine an acceptable level of performance with that user community. We were able to determine six apps that were not only top rate but fit the same application. All of the applications were downloaded and tested using the sample data sets that are provided. The collected data is outlined below:

Application

Number of Data Rows

Application Launch Time (secs)

Time to Display Data (secs)

A

0

4

0

B

41

6

1

C

67

7

5

D

0

2

0

E

180+

2

2

F

21

3

1

The six applications were tested both for application launch time and time that it took to render the display of the data. Based on this sample set of data the average time to launch the application was 3.75 seconds and 1.28 seconds to display the data within the application.

To greater understand how we can apply this information to development of an application we must analyze the data and normalize it so we may understand as we increase the load on the application how it should act compared to the rest of the population. By using linear regression we are able to determine the time it should take to display data based on the amount of data that is being processed.

 

Data

The linear regression would indicate to us that according to the applications sampled it takes 0.0601 seconds to load each row of data displayed within the sample set.

Reviewing the data collected, we conclude that an app similar to the sample set studied should launch in 3 seconds and should return data displayed for 100 rows of data in 6.01 seconds.

4 Summary

By gathering information from web based application design as well as by sampling certain standard applications available in the market, we can determine a baseline consideration for mobile application performance, which is a combination of user expectation and comparative analysis. By reviewing both aspects of the user population, we make a determination on how a mobile app can have an acceptable level of performance.

By considering the four basic guidelines for web design, understanding how those expectations can be applied to our four areas for mobile application performance and then by tying that information together with actual application performance, we are able to generate a guideline for our acceptable performance. We then applied them to the four areas of application performance that we should consider.

  1. How long does it take the app to launch?
    1. Application should launch in 3 seconds
  2. How long does it take for the app to display information?
    1. Application should display information in 5 seconds
  3. How does the app perform when switching between in app screens?
    1. Application should switch screens in 1 second
    2. Feel or user perception is also important
  4. How does the app perform when navigating through the presented information?
    1. Application should present information in 3 seconds
    2. Memory usage should be kept as low as possible to better assist in reducing lag.
    3. Feel or user perception is also important

The above are guidelines only that can be used for baseline reference.

Development tools can be used as needed, to reduce or assist with these performance considerations.

In addition, we also would like to remind that factors like – transitions, screen indicators and the kind of SDK tools used, may potentially have an impact on user perception of performance.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s