Performance Testing Using iTKO LISA

Introduction

iTKO LISA supports both the creation and staging of Web Services tests.  Lisa separates test cases from staging documents. Test cases define which components are tested. Staging documents define how those tests are run. The staging documents can be reused across the project and allowed us to define scenarios with different numbers of users and periods of time for average and peak loads. This feature of load testing makes it easier to find bottlenecks by determining each subsystem's performance under various load conditions.

1.    Creating a Staging Document

To create a staging document select File->New->Staging Document from the main menu as depicted in the screen-shot below. This opens the Staging document editor.

2.    Staging Document Editor

At the bottom of the Staging doc editor, there are five tabs Base , Reports , Metrics , Documentation , Source View .

·         Base:  This describes the basic parameters of a Test Case.

·         Reports: Where you select and add your Reports and select the types of Events you want to capture.

·         Metrics: Where you select your Metrics, and specify your sampling intervals.

·         Documentation: A text area where you can document your staging document.

·         Source View: The Staging document is saved as an XML file with a suffix of .stg.

By default the Staging document editor opens in the Base tab. The Base tab of the Staging Document divided into 3 panels Test Pacing, Load Pattern Selection and Distribution Selection.

2.1 The Test Pacing Panel:

It allows global adjustments to think times, information to pace tests such that a given number of tests complete in a specified time period.

Enter the following parameters:

·         Run Name: This is the name given to this staging document.

·         Think time: The Think Time in percentage. Here, a global change can be applied for all the test steps that are included in the Test Case. Here a step's think time range of 4-6 seconds would be reduced to 2-3 seconds if the staging document Think Time percentage was set to 50%.

·         Enable LISA Pathfinder: You can choose to enable/disable the LISA Pathfinder utility by selecting True/False.

·         Num Test Executions: The number of test executions you want to complete in a given time (Per Given Time).

·         Per Given Time: The time period in which you want the tests to run. You can specify h for hours, m for minutes, s for seconds. For example, we can specify that we want LISA to adjust the time such that 1500 Tests complete in 5 minutes.

2.2          Load Pattern Selection Panel:

It allows you to set the duration of the test, the number of virtual users and the load pattern for those virtual users (if you have more than one virtual user).

a.    Run N Times:  This pattern is applicable when you are running only one or just a few virtual users but you want to be able to specify how many times the test will run. You are not concerned with any loading pattern. This pattern will start all virtual users at the same time.

To configure this pattern enter the following parameters:

i.      Instances: The number of virtual users

ii.    Cycles: Choose between running continuously until the time in the Max Run Time setting has been reached. In the latter case, specify the Number of Times to run.

iii.   Max Run time: Choose between No Max or Maximum Run time. In the latter case specify the Maximum Run Time.

a.    Immediately Ramp: This pattern is applicable when you are running just a few virtual users but you want to be able to specify how long to run the test. You are not concerned with any loading pattern. This pattern will start all virtual users at the same time.

To configure this pattern enter the following parameters:

i.      Instances: The number of virtual users.

ii.    Max Run time: Choose between No Max or Maximum Run time. In the latter case specify the Maximum Run Time.

The example above shows there are 10 virtual users running the test concurrently for 60 min.

b.    Stair Steps: This pattern introduces virtual users to the system in well defined steps rather than all at once. You specify the total number of steady state users, the ramp up and ramp down times, and the number of steps for the ramp.

                 

To configure this pattern enter the following parameters:

i.      Steady State Instances: The maximum number of virtual users to run at steady state.

ii.    Number of Steps: The number of steps to use to reach max number of virtual users. The number of virtual users to introduce at each step is steady State Instances divided by Number of Steps.

iii.   Ramp Up Time: The time period over which virtual users are added to get to the Steady State. The time interval between steps is Ramp Up Time divided by Number of Steps.

iv.   Steady State Time: The time period of the meaningful test run.

v.    Ramp Down Time: The time period over which virtual users are removed. Since the tests will run to completion after a "stop" request, the ramp down times are approximate.

The example above shows there are 10 instances and 2 steps, so 5 virtual users will be added each step. The ramp up time is 5 secs and the test will run in steady state for 1 minutes. Two virtual users will be removed approximately every 5 seconds.

c.    Weighted Average Pattern: This pattern will add and remove virtual users based on a statistical calculation. This will approximate a bell curve distribution, with the majority of the virtual users being added within 2 standard deviations of the mid-point of the load, or unload ramp time.

To configure this pattern enter the following parameters:

i.      Steady State Instances: The maximum number of virtual users to run at steady state.

ii.    Ramp Up Time: The time period over which virtual users are added to get to the Steady State.

iii.   Steady State Time: The time period of the meaningful test run.

iv.   Ramp Down Time: The time period over which virtual users are removed. Since the tests will run to completion after a "stop" request, the ramp down times are approximate.

The example above shows the pattern for 10 virtual users ramping up over 1 minute, running in a steady state for 3 minutes, before ramping down over approximately 1 minutes.

d.    Manual Load Pattern: This pattern gives you the most control over the loading and unloading of virtual users. It is similar to the Stair Step pattern except that it allows you to specify both the number of virtual users to add, and the time interval for each step in the pattern, individually.

To configure this pattern you define each step as a row in a table (see figure above). In each row you define the time interval and the number of virtual users to add or remove (Instances Change) for that step. LISA will calculate and display the elapsed time (running time), and the total number of virtual users (Running Instances) for each step.

You can use standard Icons to Add, Remove or Change the current order of the steps from the available toolbar at the bottom of the table.

2.3 Distribution Selection Panel:

It allows you to distribute virtual users over your running Simulators. You can Add, Delete and move the simulators within the list by clicking on the icons in the toolbar. For LISA Server, with several Simulator Servers active, this panel allows you to specify how you want to distribute your virtual users over these Simulators.

There are three Distribution choices. By default the choice of distribution is "Percent Distribution".

1. Percent Distribution

2. Round Robin Distribution

3. Balanced Based on Instance Capacity

       

1.    Execution of Test Case using Staging document.

Click on ‘Start Test’ button to execute created test case using the staging. This opens the following screen.

Here you need to perform below steps.

·         Select Coordinator Server for execution.

·         Select Test Case to be executed under ‘Test Case Doc’.

·         Select created Staging document under ‘Staging Doc’.

·         Click on OK. 

Here Click on OK button. Select the Events tab in the bottom of the screen, following screen would be displayed. We can select the steps which we want for analysis purpose from Event Set window.

• Click on the Play button to start the execution.
• Once the execution is over following screen will be displayed.
• iTKO LISA delivers the results in an easy-to-use format. We can also check the performance status by selecting the ‘Perf Status’ tab from the bottom of the screen.