Mobile Network Emulation and Performance Testing Using HP Network Virtualization

Performance testing an application which is being used from many client devices using different set of network qualities is a difficult task. Without incorporating different network conditions while performance testing an application will provide results which are not closer to real world user experience and hence can not be used in capacity planning of and tunning the application.

Network condition testing becomes more relevant when mobile devices comes into picture since the network technologies and  quality for these devices posses maximum variation.

Below I will explain the process of emulating these real world network qualities using HP Network Virtualization tool (Formerly known as Shunra) and how these emulated network conditions can be incorporated in HP Load Runner scenarios. I will try to follow the process by using a simple scenario containing a single script with single transaction.





Since mobile networks have such a huge variation of network speeds and network availabilities there are network emulation options available in HP Load runner which are explained as below.

Explanation of Various run time settings and options in HP Load Runner-

There are following three options available in run time settings for emulating different network conditions –

1) Use maximum bandwidth – This option is used while we do not want to test the network bottlenecks for our application.

2) Use Bandwidth – These are predefined bandwidths which are popularly used around the globe as standards for example bandwidths of ISDN (128Kbps) , Ethernet(10Mbps), Wireless 802.11g (54Mbps) etc.

3) Use Custom bandwidth (bps) – Here we can define custom bandwidths according to our network testing conditions in bytes per second. Like 1000 bps, 2000 bps etc.

Following snap shot shows where these settings are located in run time settings -

We can see the settings are not enough to emulate real time mobile network conditions which in most of the cases are not consistent in both their good and bad behaviors.
These conditions for example do not deal with following two consistent properties of mobile networks –

1) Packet loss during transmission.

2) Random network delays.

So HP has one product called HP Network Virtualization (Earlier known as Shunra) which can be helpful in emulating real world network conditions.
In the HP Load Runner Controller on the “Design” Tab if we click on “Show Virtual Location Settings” it will take us to “Enable Network Virtualization” page.

Here we have different options to incorporate real world network conditions in our test.

Explanation of different “Virtual Location Settings” in HP Network Virtualization tool –

• “Enable Network Virtualization” check box – This check box is by default checked. If we do not want to emulate any network condition we may uncheck this for same.

• “Per Group” – If “Enable Network Virtualization” is checked then we have two options for assigning network conditions to virtual users, one of these is assigning network conditions to Vusers who belong to a group.

If different Vuser groups are running a same Load Generator we can assign different network conditions for these if required.

For assigning different “Virtual Locations” to same script we need to add that script as many times and assign different “Group Names” for different “Virtual locations we want to emulate.

• “Per Load Generator” – If we have more than 1 load generator and we want to emulate requests coming through them as from different networks having different conditions we can select this settings. This setting will perform network virtualization per load generator and virtual users running from those load generators will pass through the emulated network conditions on them. If we select this option we will not be able to allocate different “Virtual Locations” to different Vuser groups.

There are pre defined network conditions available which can be configured if required or we can create new conditions on our own according to our test conditions.

For example – Suppose we want to create a new network condition from Mumbai to Paris we can configure as follows –

• In the” Virtual” Location column specify the name of network condition. In Our case we can name it “Mumbai to Paris”. In the “Description” column we have to give small description to network condition we are trying to create so that it can be used by others in their scenarios if they need the similar network condition to test in their tests.

In our case we have given it “Client in Mumbai. Server is in Paris. Individual bandwidth” description, which means our client location is Mumbai and Server is located in Paris and we are assigning the bandwidth to each Virtual user separately i.e. bandwidth of connection is nt shared between the Vusers but they have their own bandwidth which we are specifying assigned to them individually.

• Now we need to specify the network condition among these two point i.e. Mumbai and Paris . If we click on “Configure” button on the “Virtual Location Settings Page” we will be taken to “Virtual Location Editor” page where we have following three options are available to configure network conditions for virtual location “Mumbai to Paris” –

I) Custom – Here we specify the network conditions manually as per our network understanding and research. For Example if have the Latency, Packet Loss, Download bandwidth and Upload bandwidth data for the network which we want to test we can specify those conditions using “Custom” settings.

For Example if we have following data for the network from “Mumbai to Paris” we need to manually specify these conditions through “Custom” location definition method –

a) Latency – 500 ms

b) Packet Loss – 2%

c) Download bandwidth- 512 KBps

d) Upload bandwidth- 256 KBps

*Latency – Latency is an expression of time taken by a packet of data to get from one designated point to another in a network. Contributors to network latency are –

i) Propagation time – Time taken by data packet to travel from one point to another depends on the medium of transmission. Every medium has some delay attached to it when data is transferred through it. Data in vacuum can travel at speed of light i.e. 300000 Km/s but cannot travel faster than 200000 Km/s in an optical fiber.

ii) Processing time of network hubs like repeaters, routers, switches, bridges etc.

*Packet Loss – Not all the data packets which are sent through a transmission medium reaches the destination. There may be many which are lost during the transmission and may be required to send again. Causes of packet loss includes week signal strength at receiver end, noise in transmission medium, hardware failure, corrupted software, congested network etc.

*Bandwidth – This is the data limit (generally Kilo Bytes) that can be transferred per unit time (generally seconds) through a network connection. This limit is generally set by network service provider. There are generally different limits set for downloading data through a connection and uploading data through it.

For more information on these network properties go through following links –

http://whatis.techtarget.com/definition/latency http://searchenterprisewan.techtarget.com/definition/bandwidth http://searchnetworking.techtarget.com/definition/packet-loss

II) Global Library – HP provides HP NV Global library which contains real world network related data that we can use to test those network conditions. We need to provide the input parameters to NV Library as follows and it will fetch the network condition variables based on inputs. We need to have licence to access NV Global Library.

Suppose in our example we gave following inputs –

Client Location – Mumbai, India

Server Location – Paris, France

And Specify following Network conditions –

Technology – 3G

Carrier – Airtel

Time – Business Hours

Communication quality – Fair

Note – I we do not select any specific carrier or in other words if we select “Other” option for carrier HP NV will automatically average the network conditions for selected “Technology” and provide averaged readings of network.

Please refer to following snapshots -

If you see in the “summary” section of “Virtual Location Editor” after pressing “Next” and then again “Next” you will see network conditions imported from the HP NV files as follows-

III) Import from a File – Third option available for specifying network conditions is to import network profile from a captured network file created by using HP network capture tools like HP Network Capture or HP Network Capture Express. These tools are helpful for testing those network conditions which are not available on HP NV library and we are also clueless about the network condition parameters.

Extensions for such file are .ntx and .ntxx. These tools are available on Apple store and Android play store for free.

• Run time Settings for “Virtual Location” – Here also we have following settings which we need to specify before assigning this network profile to Vusers, those are as follows-

I) Bandwidth allocation – There are two options available for allocating bandwidth we allocated in the step above –

a) Shared bandwidth - This is the first option for sharing bandwidth among different Vusers. This option will enable all the Vusers of a group running on the same LG to share the bandwidth allocated in the step above. i.e. if there are 30 Vusers to be executed from a single LG and these Vusers are distributed among 3 groups A, B and C and we have specified a particular network bandwidth condition for Group A then all the Vusers in Group A on this LG will use the same network condition. But if the Vusers of Group A are distributed on two or more LGs then they will use the network bandwidth on those LGs separately.

b) Individual bandwidth – In this case the defined bandwidth is allocated to each Vuser individually.So if we have specified that the 10 Vusers of Group A will use 10 KBps bandwidth then 10 KBps of bandwidth is allocated to all 10 Vusers. Total bandwidth for 10 Vusers in that case will be 100 KBps for Group A Vusers.

This setting is particularly used when we want to test Mobile users or home users since in those cases the bandwidth of networks are completely allocated to server single user.

• Enable Packet Capture – This setting allows us to capture the data packets that are transferred across network while performance test runs. This feature is provided to find out any bottle neck in a network and should be used in Single user test. Since the information collected is huge this option should generally be not helpful while running a full blown load test and also that is not the intended and useful way to use this feature.

To enable this setting we have to enable it at two locations –

1) At Global location i.e. at “Virtual Locations Settings” page. Here we need to click “Common Settings” button which will take us to “Virtual Location Test Level Settings” where we need to turn on the “Enable Packet Capture” option. We also need to specify the maximum packet list size in MBs that should be maintained and also specify how information about recorded packets can be removed if it increases the predefined slot size provided.

2) At Local Virtual Location Now that the packet capture is enable at the scenario level we need to enable it at every virtual location profile we need to capture. For that we need to go to the required “Virtual Location” and hit the “Configure” button. After that click the “Edit button” adjacent to “Run Time Options”.

Now expand the “Enable Packet Capture” option and turn on the “Enable Packet Capture” option.

Example - Creating, Executing and analyzing results of a performance test scenario having network conditions specified using HP Load Runner Controller-

Now I will explain the whole process of creating a scenario in Load Runner Controller, executing it with different network conditions and analyzing results.

For simplicity I have captured just the request to open yahoo home page in a script :) :)

Scenario Properties –

Number of scripts – Two (Same Script repeated twice with different group names for two virtual locations)

Script Functionality – Open Yahoo Home Page

Transaction in Script – “Yahoo Home Page”

Number of Network Virtual Locations – 2

Virtual location 1 – Mumbai to Paris, Technology – WAN, Bandwidth – 56 KBps, Time – Business hours, Communication Quality – Good, Bandwidth – Shared among Vusers.

Virtual location 2 – New York to Paris, Technology – 4G, Carrier –AT&T, Time – Business hours, Communication Quality – Fair, Bandwidth – Individual to each Vuser.

Note 1– I will be adding Virtual locations as per group. Since I am using HP’s free version and do not have support for creating more Load Generators :). Also I can only use 2 Vusers in a virtualized test scenario according to HP free licence agreement.

Note 2 – Since I have 1 MBps network bandwidth at my home I won’t be able to provide more bandwidth collectively than that to any of the Vusers.

Note 3 – Since my script has just one transaction and that too just a URL hit, I have modified the network conditions like adding latency 1000 ms and packet loss 10% etc etc so that we can see some difference in the results. Otherwise the amount of data we are sending it will never make any difference. I am not mentioning those changes here since that will only confuse us more :):).

Step 1 – Configuring virtual locations using HP Network Virtualization tool as follows –

Step 2 – Creating the scenario and allocating Virtual users and setting Scenario schedule as follows.We rename the group names as “yahoo_page_navigation_mumbai_to_paris” and “yahoo_page_navigation_newyork_to_paris” to differentiate the transactions after the execution ends.

Step 3 – While the test is running we can see the Network related readings on the left side as shown in below snapshot.

Step 4 – After the execution, collation and analysis we need to now look into the test results based on locations. We can do that as follows –

Note – I am taking example of average response time based on location but this location based analysis can be done for other performance metrics too.

a)      Go to the results.

b)      In the results open “Average transaction response time” graph.

c)       Right click on the graphs and select “Set Filter/Group by” option.

d)      In the pop up page select “Location name” in the “Group by” section.

e)      Now you can see the “Average Response time” metric s for the single transaction we have in our script based on two different network conditions we have configured to emulate.

If you want this in a bit more expressive way we can go to “Display options” after right clicking on the graph and select “Graph type” as “Bar” and “Display option” as “3D” and we will have more visually soothing and easy to understand graph in our results.

We can easily see that the “Average response time” values of same transaction from two different network locations are different.

If we have not used any WAN emulations in this test may be our results would have been better, but that would not be a correct inference.

Hence network conditions are a must testing space and more so while testing mobile applications.