VuGen

Why Performance Tests?

• The failure of a mission-critical web application can be costly (B$)
• Assure performance and functionality under real-world conditions
• Locate and resolve potential problems before it hits on the users.
• Performance is the key to success of any web based application
• We will get to know the maximum capacity the system can handle for an application
• We can decide whether we should go for Hardware upgrades or Performance tuning

What are Performance Testing Types?

• Load Testing
• Stress Testing
• Spike Testing
• Endurance Testing
• Scalability Testing
• Hot Spot Testing
• Deadlock Testing

Load Testing:
Many concurrent users running the same application to see whether a system handles the load without compromising functionality or performance.

Test Objectives
• How many concurrent users can the system handle without increase in the expected response time?
• Can the system serve x concurrent users without any errors?

Important Terminologies
• System
• Response Time
• Think Time
• Work Load
• Transaction Mix
• Throughput

The System:
System is a broad sense of Operating Systems, Clients, Web Servers, Application Servers, Database Servers.



Response Time:
The time taken by a server to respond a client request.
Response time includes Network Time + Web Server Time + Application Server Time + Database Server Time.

Load Testing Web based Applications with LOADRUNNER


Think Time:
Time taken by user between two actions.


Workload:




Transaction Mix:
Varying execution frequency of different Transactions.


Throughput:
The amount of work that can be performed by a system or component in a given period of time

 Transactions per Second
 Web Interactions per Second
 Pages per Second
 Bytes per Second

Steps Involved in Load Testing

• Planning Load Tests
• Creating Vusers
• Creating Scenarios
• Executing Scenarios
• Analysis of the system under Load

PLANNING:

Objectives:

• Information that are required for load testing a system
• Where to obtain this information from?
• Organizing the system information collected
• Using the information effectively to carry out Load Tests.

What is a Business Transaction?
A business transaction is a set of user actions performed on an application

Examples:
• Reserving flight tickets
• Buying a computer from a online store
• Logging into your web-based email service

Important Information for any Load Test

• The architecture of the system
• A brief idea about the application.
• The list of business transactions
• The time at which the system has peak load
• The acceptable response time for the transactions
• Number of concurrent users the system is expected to handle during peak load

How to Obtain?

• IT documentation
• Application users
• IT Managers
• System Administrators
• Server and online statistics
• Developers


Getting Organized
• Select the Transactions to automate
• Decide on the Transaction Mix
• Obtain User details for the transactions
• The tables these transactions query
• The web server / application server components invoked by the transactions

Select the transactions:
• Give priority to business critical transactions
• Give priority to transactions that have high database activity
• Give priority to transactions that have high web server or application server activity
• Know the peak load time

Decide on Transaction Mix:
• Impact of the failure of a particular transaction on business
• The transactions that are fired during peak load
• the no. of times they are fired during peak load

User details:

• For each of the chosen transactions find
• Who are the users who fire the transaction?
• How many concurrent users are there?
• How often they fire the transactions?

For each of the chosen transactions find

• Tables queried
o The tables the transaction queries
o The tables the transaction updates

• Server components invoked
o The servlet / jsp that are invoked by the transaction.

Creating Vusers:

VuGen is a LoadRunner tool for creating Vusers or Vuser Scripts.

Using VUGEN

• Introduction to Virtual User Technology
• Record scripts
• General options
• Recording options
• Runtime options
• Replay recorded scripts

Virtual User Technology:

• Vuser simulates a real user
• Requires less resources – machines and people
• Greater control over test execution
• Can synchronize actions performed by users
• Collect and analyze results in a better way
• Vusers can communicate directly with a server by executing calls to the server API-without relying on client software


VuGen:



• Capture a business transaction by recording user actions performed
• Monitors the communication between the application and the server
• Generates the required function calls
• Inserts the generated function calls into a Vuser script

Recording a Script:



• Open VuGen
• Select Multi-Protocol Script option
• Select the Protocol using which you want to create a Script
• Click OK




Vuser Script Sections:

Each Vuser script contains at least three sections: vuser_init, one or more Actions, and vuser_end. Before and during recording, you can select the section of the script into which VuGen will insert the recorded functions.

The following table shows what to record into each section, and when each section is executed.

Script Section Used when recording... Is executed when...
vuser_init a login to a server the Vuser is initialized (loaded)
Actions client activity the Vuser is in "Running" status
vuser_end a logoff procedure the Vuser finishes or is stopped


VuGen Recording Options:
VuGen Recording Options are categorized into 3 sections:

• General Options
• Network Options
• Internet Protocol Options

General Options:•

Script: Select the language in which you want to record the script ( C by default)
• Protocol: Select the protocol

Network Options:

• Port Mapping: Select from which port you want to record the script

Internet Protocol Options:

• Recording Options: Select the mode of the recording (HTML or URL)
• Advanced Options:
• Correlation Options: Select Correlation Options


Recording Modes:

This allows us to specify the information to be recorded and which functions to be used when generating a script.

VuGen records the scripts in any of the following mode

 HTML Mode
 URL Mode

HTML Mode:
The HTML mode is based on user actions, and the scripts contain functions that correspond directly to the action taken

Functions Generated in HTML mode:

web_url()
web_link()
web_image()
web_submit_form()

The HTML based script option, which is the default recording mode, instructs VuGen to record HTML actions in the context of the current Web page.

The actions are recorded as URL steps (web_url), link steps (web_link), image steps (web_image), and form submission steps (web_submit_form).

URL Mode:

The URL mode is based on HTTP requests sent to the server as a result of user actions.

Functions generated in URL mode are:

web_url()
web_concurrent_start(NULL);
web_concurrent_end(NULL);
web_submit_data()
web_custom_request()


Deciding on Recording Mode:

HTML Mode
1.Easy to understand
2.Scalable

URL Mode
1.Not easy to understand unlike as the HTML scripts.
2.More scalable and effective for creating a load test

Deciding on Modes

 For browser applications, use HTML mode.
 For non-browser applications, use URL mode.

HTML Mode:

Advantages:

• Reduces need to capture dynamic values

– action tag values and hidden data are NOT hard coded
– they are retrieved from memory during playback
– if they are dynamic, the Vuser still runs

• Script is only as big as the business process–one step per page

Disadvantages:

• Scripts are less scalable
• Memory (cache) is searched during playback
– requires more memory
– requires more CPU power

URL Mode:

Advantages:• Flexibility

– support for Java Applets and ActiveX objects on the page
– ability to replay on Unix

• Scalability
– scripts are more scalable than HTML scripts because they require fewer resources


Disadvantages:

• Scripts require more correlation (nothing is retrieved from cache)
• Context-sensitive checks won’t work (parser is disabled)*
• Scripts are large (all images and frames are recorded as separate steps)

Setting Advanced Recording Options

Use the Advanced recording options tab to record custom headers, apply a content type filter, specify non-resources, and set the think-time threshold.



Reset Context for Each Action:

You can instructs VuGen to reset all HTTP contexts between Actions, to their states before the initial recording. This setting allows the Vuser to more accurately emulate a new user beginning a browsing session. It resets all cookies, closes all keep-alive connections, clears the cache, and resets the user-names and passwords. (Enabled by default)

Recording Custom Headers:

Web Vusers automatically send several standard HTTP headers with every HTTP request submitted to the server. Click Headers to instruct VuGen to record additional HTTP headers.

You can work in three modes:
Do not Record Headers,
Record Headers in list, or
Record Headers not in list.

Filtering Content Type
VuGen allows you to filter the content type for your recorded script. You specify the type of the content you wish to record or exclude from your script.
You can work in three modes: Do not Filter Content Types, Exclude content types in list, or Exclude content types not in list.

Handling Non-HTML Resources:

Many Web pages contain non-HTML elements, such as applets, XML, or javascript. These elements usually contain or retrieve their own resources.

For example, a javascript js file, called from the recorded web page, may load several images.
An applet may load an external text file. Using the Advanced options, you can control how VuGen records non-HTML elements.






Recording Scripts:



• Click on Record Button ( )
• Record dialog box pops up.



• Click Stop button to end recording
• Save the script.



Note:
• Record user transaction is the appropriate sections
• Do not name a script init, run, or end, as these names are used by VuGen.


Directory Structure:



 While recording, VuGen creates a series of configuration, data, and source code files which contain Vuser run-time and setup information.
 The results of each iteration of a script are stored separately.
 The Iteration0 directory is for system use and should be ignored by the tester.


Enhancing the Vuser Script:

• Insert Transactions into the Script
• Parameterizing the Script
• Correlating the Script
• Inserting Rendezvous points
• Inserting Informational and Runtime functions into the script

Load Runner Transactions:

• A LoadRunner transaction measures the performance of any user action that is made from the browser.
• Insert transaction by selecting the Start Transaction button in the script (lr_start_transaction)
• Click the End Transaction buttonafter selecting the end point of the transaction (lr_end_transaction)
• Select Transaction Status (optional) – LR_AUTO, LR_PASS, LR_FAIL

Note: LoadRunner Transactions can also be inserted during recording

Parameterization:

Parameter
• A parameter is a placeholder which re-places a recorded value in a Vuser script. At run time, a value from an external source is substituted for the parameter.

Parameterize Input data:

• During a run, VuGen replaces each parameter with a value from a data file or another external source, and inputs it to the Web application

Why Parameterization?

Parameterization solves certain problems that may occur during playback

• Date constraints make recorded date invalid
• Unique constraints prevent reusing recorded data
• Data caching
o Response times are inaccurate
o Real user activity is not emulated
o Servers are not exercised

Date Constraint:

Problem:
• Business process works only for the current date
• Vuser works only on the date it was recorded

A date that is hard-coded in the Vuser during recording may no longer be valid when that date has passed. Parameterizing the date allows Vuser execution to succeed by replacing the hard-coded date.

Solution:
• Parameterize the input date
• Use a Date Type parameter to make the Vuser work for any current date


Unique Constraint:

Problem:
• Some fields require a unique ID as in the case of a Primary Key or fields in database with Unique constraint

How to measure the performance of the process which creates a new product?
A field that uses unique data is constrained because the business process will not allow reuse of the data after it has been added to the system and processed.

Solution:
• Parameterize the unique id fields
• After running, delete the test ID numbers from the system so the Vuser can be run again

Data is cached:

Problem:
• Vusers all use the same data
• Data caching makes load test response times shorter than when real users load the system

Data gets relocated in the server’s cache when it is reused for each Vuser execution. As a result, server activity is not emulated during playback. While Vuser execution is successful, it will not produce test results meaningful for load testing because the server is not exercised. Using different data better emulates server activity and exercises the system. That improves the chances for meaningful load test results.

Solution
• Parameterize input data to force the system to get data from a database

How to create data pool?

 Extract master data from database
 Write queries on tables being accessed by the business transactions.
 Spool data to a file
 Write AWK scripts to generate a proper data pool from the spooled file

 Create valid transactional data
 Predict according to type of data
 Obtain from a functional expert

The Process:

• Determine which fields to parameterize
• Replace recorded values with parameters
• Decide which parameter type to use
• Choose the data access method and the number of iterations
• Run the Vuser and analyze the results to verify correct execution

Creating Parameters:

You create a parameter by specifying its name and type. There is no limit to the number of parameters you can create in a Vuser script.

In Web Vuser scripts, you can create parameters in the text-based script view using the procedure below, or you can create parameters in the icon-based tree view.

Parameterization – The Process:

To create a parameter:
1. Right-click on the string to parameterize. The right-click pop-up menu opens.
2. From the pop-up menu, select Replace with a Parameter. The Select or Create Parameter dialog box opens.
3. Type a name for the parameter in the Parameter name box, or select an existing parameter name from the list.





4. Select a parameter type from the Parameter type list.
5. Click OK to close the Select or Create Parameter dialog box.VuGen replaces the selected string in your script with the name of the parameter, surrounded by angle brackets.



6. To replace additional occurrences of the string with the same parameter, right-click on the parameter. From the pop-up menu, select Replace More Occurrences.

The Search and Replace dialog box opens. The Find What box displays the value you want to replace. The Replace With box displays the parameter name in angle brackets.

Select the appropriate check boxes for matching whole words or case. Click Replace or Replace All.


Correlations:

• In applications output of the previous transaction is used as an input for the next transaction.

For example:
• System generated session ids
• URLs that change each time you access the Web page
• Fields (sometimes hidden) recorded during a form submission

 The data is only good for the current session




Correlate the data:

 Capture output value from one step
 Use captured value as input to another step

Correlated data is data which is sent to the client from the server, and later sent back to the server by the client



Dynamic data is correlated automatically by the client application (running in the browser) while recording a script

The cause of this Vuser execution problem is that the dynamic value returned from the server was recorded and, therefore, was part of the Vuser script. When the script is played back, the Vuser attempts to reuse the recorded value, while the server is expecting another value, assigned for the current playback session.



 Reusing recorded dynamic data will cause errors when you replay the script
 Recorded dynamic data must be recorrelated using parameters

Types of Correlation:

• Manual
• Automatic

Manual Correlation:

• Determine the value to capture
• Find the right and left text boundaries of the value to capture
• Find which occurrence of the text boundaries should be used
• Add a web_reg_save_param function to the script, above the step which requests the page with the value to capture
• Add the parameter name, left boundary, right boundary, and occurrence to the function
• Parameterize the dynamic value in the script every time it occurs
• Verify correct execution

Automatic Correlation:




There are three types of automatic recording solutions
 Auto-Detect Correlation
 Rule-Based Correlation
 Correlating All Statements

Rule Name When to Use
Auto-detect
Correlation Detect and correlate dynamic data for supported application servers
Rule-Based When working with a non-supported application server whose context is known.
Correlate All Blindly correlate all dynamic data.


Auto-Detect Correlation
Automatically correlate the recorded statements, for application servers whose contexts

Run-Time Settings:

 Control the behavior of the Vuser when running the script
 Configure Web Vusers to emulate real users accessing the Web site

VuGen Run-Time Settings are categorized as follows:

• General
• Network
• Browser
• Internet Protocol

General Settings:

• Run-Logic : For setting iteration count (Number of iterations)
• Pacing: (As soon as the prev iteration ends, after prev end with fixed delay, at fixed delay)
• Log: Standard/Extended
• Think time: ignore/replay (as recorded, multiply by, use the percentage min and max, limit)
• Additional Attributes:
• Miscellaneous: Error Handling (Continue on error, Fail open transaction on lr_error_message, generate snapshot on error)), Multi threading (Run Vuser as a process/Thread), Automatic Transactions(Define each step/action as a Transaction)

Network Settings:

• Speed Simulations: For setting the bandwidth (Use max bandwidth, Use bandwidth, Use custom bandwidth)

Browser:
• Browser Emulation:
For browser emulation (User agent, Simulate browser cache, Download non-html resources, Simulate new user on each iteration, clear cache on each iteration)

Internet Protocol:

• Proxy: Setting proxy (No proxy, from browser, use custom proxy, Use proxy server)
• Preferences: (Checks( Image/text), Generate web performance graphs, Advanced)
• Download Filters: (Include only addresses in the list, Exclude addresses in the list)
• Content Check: (Enable content check during the replay)

Important Run-Time Settings:

 Browser Emulation
 Think Time
 Logs

Browser Emulation:



 Specify Browser type
 Simulate cache
 Emulate network capacity
 Emulate modem speed


Think Time Settings:



 Specified in seconds
 Ignore recorded think time
 Replay the recorded think time
 Randomize think time
 Multiply the recorded think time by specific values
 Limit the think time to a maximum value

Log Settings:



 Can disable logging
 Log a specific set of function calls and messages – Standard Log
 Log parameter substitutions
 Log response data from server
 All the messages and function calls raised

Performance Testing Process

Phase 1 – Requirements Study

The objective of this phase is to understand the performance test requirements, Hardware &Software components and Usage Model.

It is important to understand as accurately and as objectively as possible the nature of load that must be generated. Following are the important performance test requirement that needs to be captured during this phase.

• Response Time
• Transactions Per Second
• Hits Per Second
• Workload
• Number of con current users
• Volume of data
• Data growth rate
• Resource usage
• Hardware and Software configurations

Phase 2 – Test Plan

The following configuration information will be identified as part of performance testing environment requirement identification.

Hardware Platform

• Server Machines
• Processors
• Memory
• Disk Storage
• Load Machines configuration
• Network configuration

Software Configuration

• Operating System
• Server Software
• Client Machine Software
• Applications


Phase 3 – Test Design

During the test design period the following activities will be carried out:
• Scenario design
• Detailed test execution plan
• Dedicated test environment setup
• Script Recording/ Programming
• Script Customization (Delay, Checkpoints, Synchronizations points)
• Data Generation
• Parameterization/ Data pooling


Phase 4 –Scripting

• Browse through the application and record the transactions with the tool
• Parameterization, Error Checks and Validations
• Run the script for single user for checking the validity of scripts


Phase 5 – Test Execution
The test execution will follow the various types of test as identified in the test plan. All the scenarios identified will be executed. Virtual user loads are simulated based on the usage pattern and load levels applied as stated in the performance test strategy.

The following artifacts will be produced during test execution period:
• Test logs
• Test Result


Phase 6 – Test Analysis

• Analyzing the run results and preparation of preliminary report.


Phase 7 – Preparation of Reports

The test logs and results generated are analyzed based on Performance under various load, Transaction/second, database throughput, Network throughput, Think time, Network delay, Resource usage, Transaction Distribution and Data handling.

Manual and automated results analysis methods can be used for performance results analysis.

The following performance test reports/ graphs can be generated as part of performance testing:-

• Transaction Response time
• Transactions per Second
• Transaction Summary graph
• Transaction performance Summary graph
• Transaction Response graph – Under load graph
• Virtual user Summary graph
• Error Statistics graph
• Hits per second graph
• Throughput graph
• Down load per second graph
• Based on the Performance report analysis, suggestions on improvement or tuning will be provided to the design team:
• Performance improvements to application software, middleware, database organization.
• Changes to server system parameters.
• Upgrades to client or server hardware, network capacity or routing.