Load Runner

Explain the Load testing process?

Step 1: Planning the test.

Here, we develop a clearly defined test plan to ensure the test scenarios we develop will accomplish load-testing objectives. 

Step 2: Creating Vusers. Here, we create Vuser scripts that contain tasks performed by each Vuser, tasks performed by Vusers as a whole, and tasks measured as transactions. 

Step 3: Creating the scenario. A scenario describes the events that occur during a testing session. It includes a list of machines, scripts, and Vusers that run during the scenario. We create scenarios using LoadRunner Controller. We can create manual scenarios as well as goal-oriented scenarios. In manual scenarios, we define the number of Vusers, the load generator machines, and percentage of Vusers to be assigned to each script. For web tests, we may create a goal-oriented scenario where we define the goal that our test has to achieve. LoadRunner automatically builds a scenario for us. 

Step 4: Running the scenario.
We emulate load on the server by instructing multiple Vusers to perform tasks simultaneously. Before the testing, we set the scenario configuration and scheduling. We can run the entire scenario, Vuser groups, or individual Vusers.

 Step 5: Monitoring the scenario.
We monitor scenario execution using the LoadRunner online runtime, transaction, system resource, Web resource, Web server resource, Web application server resource, database server resource, network delay, streaming media resource, firewall server resource, ERP server resource, and Java performance monitors. 

Step 6: Analyzing test results. During scenario execution, LoadRunner records the performance of the application under different loads. We use LoadRunnerâ€â"¢s graphs and reports to analyze the applicationâ€â"¢s performance.

When do you do load and performance Testing? –

We perform load testing once we are done with interface (GUI) testing. Modern system architectures are large and complex. Whereas single user testing primarily on functionality and user interface of a system component, application testing focuses on performance and reliability of an entire system. For example, a typical application-testing scenario might depict 1000 users logging in simultaneously to a system. This gives rise to issues such as what is the response time of the system, does it crash, will it go with different software applications and platforms, can it hold so many hundreds and thousands of users, etc. This is when we set do load and performance testing.

What are the components of LoadRunner? –

The components of LoadRunner are The Virtual User Generator, Controller, and the Agent process, LoadRunner Analysis and Monitoring, LoadRunner Books Online.

What Component of LoadRunner would you use to record a Script? –

The Virtual User Generator (VuGen) component is used to record a script. It enables you to develop Vuser scripts for a variety of application types and communication protocols.

What Component of LoadRunner would you use to play Back the script in multi user mode? –

The
Controller component is used to playback the script in multi-user mode. This is done during a scenario run where a vuser script is executed by a number of vusers in a group.

 

What is a rendezvous point? –

You insert rendezvous points
into Vuser scripts to emulate heavy user load on the server. Rendezvous points
instruct Vusers to wait during test execution for multiple Vusers to arrive at a certain point, in order that they may simultaneously perform a task. For example, to emulate peak load on the bank server, you can insert a rendezvous point instructing 100 Vusers to deposit cash into their accounts at the same time.

What is a scenario? –

A scenario defines the events that occur during each testing session. For example, a scenario defines and controls the number of users to emulate, the actions to be performed, and the machines on which the virtual users run their emulations.

Explain the recording mode for web Vuser script? –

We use VuGen to develop a Vuser script by recording a user performing typical business processes on a client application. VuGen creates the script by recording the activity between the client and the server. For example, in web based applications, VuGen monitors the client end of the database and traces all the requests sent to, and received from, the database server. We use VuGen to: Monitor the communication between the application and the server; Generate the required function calls; and Insert the generated function calls into a Vuser script.

Why do you create parameters? –

Parameters are like script variables. They are used to vary input to the server and to emulate real users. Different sets of data are sent to the server each time the script is run. Better simulate the usage model for more accurate testing from the Controller; one script can emulate many different users on the system.

What is correlation? Explain the difference between automatic correlation and manual correlation? –

Correlation is used to obtain data which are unique for each run of the script and which are generated by nested queries. Correlation provides the value to avoid errors arising out of duplicate values and also optimizing the code (to avoid nested queries). Automatic correlation is where we set some rules for correlation. It can be application server specific. Here values are replaced by data which are created by these rules. In manual correlation, the value we want to correlate is scanned and create correlation is used to correlate.

How do you find out where correlation is required? Give few examples from your projects? –

Two ways:
First we can scan for correlations, and see the list of values which can be
correlated. From this we can pick a value to be correlated. Secondly, we can record two scripts and compare them. We can look up the difference file to see for the values which needed to be correlated.  In my project, there was a unique id developed for each customer, it was nothing but Insurance Number, it was generated automatically and it was sequential and this value was unique. I had to correlate this value, in order to avoid errors while running my script. I did using scan for correlation.

Where do you set automatic correlation options? –

Automatic correlation from web point of view can be set in recording options and correlation tab. Here we can enable correlation for the entire script and choose either issue online messages or offline actions, where we can define rules for that correlation. Automatic correlation for database can be done using show output window and scan for correlation and picking the correlate query tab and choose which query value we want to correlate. If we know the specific value to be correlated, we just do create correlation for the value and specify how the value to be created.

What is a function to capture dynamic values in the web Vuser script? –

Web_reg_save_param function saves dynamic data information to a parameter.

When do you disable log in Virtual User Generator, When do you choose standard and extended logs? –

Once we debug our script and verify that it is functional, we can enable logging for errors only. When we add a script to a scenario, logging is automatically disabled.
Standard Log Option:
When you select
Standard log, it creates a standard log of functions and messages sent during script execution to use for debugging. Disable this option for large load testing scenarios. When you copy a script to a scenario, logging is automatically disabled
Extended Log Option: Select
extended log to create an extended log, including warnings and other messages. Disable this option for large load testing scenarios. When you copy a script to a scenario, logging is automatically disabled. We can specify which additional information should be added to the extended log using the Extended log options.

How do you debug a LoadRunner script? –

VuGen contains two options to help debug Vuser scripts-the Run Step by Step command and breakpoints. The Debug settings in the Options dialog box allow us to determine the extent of the trace to be performed during scenario execution. The debug information is written to the Output window. We can manually set the message class within your script using the lr_set_debug_message function. This is useful if we want to receive debug information about a small section of the script only.

How do you write user defined functions in LR? Give me few functions you wrote in your previous project? –

Before we create the User Defined functions we need to create the external
library (DLL) with the function. We add this library to VuGen bin directory. Once the library is added then we assign user defined function as a parameter. The function should have the following format: __declspec (dllexport) char* <function name>(char*, char*)Examples of user defined functions are as follows:GetVersion, GetCurrentTime, GetPltform are some of the user defined functions used in my earlier project.

What are the changes you can make in run-time settings?

The Run Time Settings that we make are:

a) Pacing - It has iteration count.

b) Log - Under this we have Disable Logging Standard Log and

c) Extended Think Time - In think time we have two options like Ignore think time and Replay think time.

d) General - Under general tab we can set the vusers as process or as multithreading and whether each step as a transaction.

Where do you set Iteration for Vuser testing? –

We set Iterations in the Run Time Settings of the VuGen. The navigation for this is Run time settings, Pacing tab, set number of iterations.

How do you perform functional testing under load? –

Functionality under load can be tested by running several Vusers concurrently. By increasing the amount of Vusers, we can determine how much load the server can sustain.

What is Ramp up? How do you set this? –

This option is used to gradually increase the amount of Vusers/load on the server. An initial value is set and a value to wait between intervals can be
specified. To set Ramp Up, go to 'Scenario Scheduling Options'

What is the advantage of running the Vuser as thread? –

VuGen provides the facility to use multithreading. This enables more Vusers to be run per
generator. If the Vuser is run as a process, the same driver program is loaded into memory for each Vuser, thus taking up a large amount of memory. This limits the number of Vusers that can be run on a single
generator. If the Vuser is run as a thread, only one instance of the driver program is loaded into memory for the given number of
Vusers (say 100). Each thread shares the memory of the parent driver program, thus enabling more Vusers to be run per generator.

If you want to stop the execution of your script on error, how do you do that? –

The lr_abort function aborts the execution of a Vuser script. It instructs the Vuser to stop executing the Actions section, execute the vuser_end section and end the execution. This function is useful when you need to manually abort a script execution as a result of a specific error condition. When you end a script using this function, the Vuser is assigned the status "Stopped". For this to take effect, we have to first uncheck the â€Å"Continue on error” option in Run-Time Settings.  

What is the relation between Response Time and Throughput? –

The Throughput graph shows the amount of data in bytes that the Vusers received from the server in a second. When we compare this with the transaction response time, we will notice that as throughput decreased, the response time also decreased. Similarly, the peak throughput and highest response time would occur approximately at the same time.

Explain the Configuration of your systems? –

The configuration of our systems refers to that of the client machines on which we run the Vusers. The configuration of any client machine includes its hardware settings, memory, operating system, software applications, development tools, etc. This system component configuration should match with the overall system configuration that would include the network infrastructure, the web server, the database server, and any other components that go with this larger system so as to achieve the load testing objectives.

How do you identify the performance bottlenecks?

Performance Bottlenecks can be detected by using monitors. These monitors might be application server monitors, web server monitors, database server monitors and network monitors. They help in finding out the troubled area in our scenario which causes increased response time. The measurements made are usually performance response time, throughput, hits/sec, network delay graphs, etc.

Web server, database and Network are all fine where could be the problem?

The problem could be in the system itself or in the application server or in the code written for the application.

How did you find web server related issues? –

Using Web resource monitors we can find the performance of web servers. Using these monitors we can analyze throughput on the web server, number of hits per second that
occurred during scenario, the number of http responses per second, the number of downloaded pages per second.

How did you find database related issues? –

By running â€Å"Database” monitor and help of â€Å"Data Resource Graph” we can find database related issues. E.g. You can specify the resource you want to measure on before running the controller and than you can see database related issues

What is the difference between Overlay graph and Correlate graph? –

Overlay Graph: It overlay the content of two graphs that shares a common x-axis. Left Y-axis on the merged graph showâ€â"¢s the current graphâ€â"¢s value & Right Y-axis show the value of Y-axis of the graph that was merged. 

Correlate Graph: Plot the Y-axis of two graphs against each other. The active graphâ€â"¢s Y-axis becomes X-axis of merged graph. Y-axis of the graph that was merged becomes merged graphâ€â"¢s Y-axis.

How did you plan the Load? What are the Criteria? –

Load test is planned to decide the number of users, what kind of machines we are going to use and from where they are run. It is based on 2 important documents, Task Distribution Diagram and Transaction profile. Task Distribution Diagram gives us the information on number of users for a particular transaction and the time of the load. The peak usage and off-usage are decided from this Diagram. Transaction profile gives us the information about the transactions name and their priority levels with regard to the scenario we are deciding.

What does vuser_init action contain? –

Vuser_init action contains procedures to login to a server.

What does vuser_end action contain? –

Vuser_end section contains log off procedures.  

What is think time? How do you change the threshold? -  

Think time is the time that a real user waits between actions. Example: When a user receives data from a server, the user may wait several seconds to review the data before responding. This delay is known as the
think time. Changing the Threshold: Threshold level is the level below which the recorded think time will be ignored. The default value is five (5) seconds. We can change the think time threshold in the Recording options of the Vugen.

What is the difference between standard log and extended log? –

The standard log sends a subset of functions and messages sent during script execution to a log. The subset depends on the Vuser type Extended log sends a detailed script execution messages to the output log. This is mainly used during debugging when we want information about: Parameter substitution. Data returned by the server. Advanced trace.

Explain the following functions: -

lr_debug_message - The lr_debug_message function sends a debug message to the output log when the specified message class is set.

lr_output_message - The lr_output_message function sends notifications to the Controller Output window and the Vuser log file.

lr_error_message - The lr_error_message function sends an error message to the LoadRunner Output window.

lrd_stmt - The lrd_stmt function associates a character string (usually a SQL statement) with a cursor. This function sets a SQL statement to be processed.

lrd_fetch - The lrd_fetch function fetches the next row from the result set.

Throughput -  If the throughput scales upward as time progresses and the number of Vusers increase, this indicates that the bandwidth is sufficient. If the graph
were to remain relatively flat as the number of Vusers increased, it would
be reasonable to conclude that the bandwidth is constraining the volume of
data delivered. 

Types of Goals in Goal-Oriented Scenario -  

Load Runner provides you with five different types of goals in a goal oriented scenario:

• The number of concurrent Vusers
  
• The number of hits per second
  
• The number of transactions per second
  
• The number of pages per minute
  
• The transaction response time that you want your scenario
  

Analysis Scenario (Bottlenecks):

In Running Vuser graph correlated with the response time graph you can see that as the number of Vusers increases, the average response time of the check itinerary transaction very gradually increases. In other words, the average response time steadily increases as the load
increases. At 56 Vusers, there is a sudden, sharp increase in the average response
time. We say that the test broke the server. That is the mean time before failure (MTBF). The response time clearly began to degrade when there were more than 56 Vusers running simultaneously.

What is correlation? Explain the difference between automatic correlation and manual correlation? –

Correlation is used to obtain data which are unique for each run of the script and which are generated by nested queries. Correlation provides the value to avoid errors arising out of duplicate values and also optimizing the code (to avoid nested queries). Automatic correlation is where we set some rules for correlation. It can be application server specific.
Here values are replaced by data which are created by these rules. In manual correlation, the value we want to correlate is scanned and create correlation is used to correlate.

Where do you set automatic correlation options? –

Automatic correlation from web point of view, can be set in recording options and correlation tab. Here we can enable correlation for the entire script and choose either issue online messages or offline actions, where we can define rules for that correlation.  Automatic correlation for database, can be done using show output window and scan for correlation and picking the correlate query tab and choose which query value we want to correlate. If we know the specific value to be correlated, we just do create correlation for the value and specify how the value to be created.

What is a function to capture dynamic values in the web vuser script? –

Web_reg_save_param function saves dynamic data information to a parameter.

How many types of Vusers are available?

There are several types of Vusers
1. GUI
2. Database
3.RTE(Terminal emulator)
4. SAP
5. DCOME
6. Peoplesoft
7. Java
8. BAAN

VuGen contains the 3 sections.
1. Vuser-init
2. Action
3. Vuser-End

What is runtime setting?

Runtime setting includes loop.log and timing information.

 What is standalone mode?

It is the mode to verify that the script runs correctly or not.

What type of function generate and insert by the VuGen to the script when you record a script?

1. LR function (Vuser function)
2. Protocol function.

How VuGen create a Vuser script?

By recording the activity between client and server

How you edit the script?

While editing the script we have to insert the transaction point and rendezvous point.

What is the load runner start transaction and its syntax?

It will start the transaction on the script.
Syntax: Lr-start-transaction("transaction name")

What is the load runner end transaction and its syntax?

It will end transaction.
Syntax: Lr-end-transaction("transaction name", LR-AUTO)

Where you insert the rendezvous point?

Rendezvous point inserted into script to calculate the peak load of the server.
Syntax: lr-rendezvous("rendezvous point")

What are the elements in the load runner controller?

Title bar ( Name of the scenario presently working). Menu bar ( selecting the various command ). Tool bar. status bar.

What are the 5 icons appear in the bottom of the controller windows?

1. Host windows (list of machines)
2. Script windows (list of all the vuser script)
3. Rendezvous windows
4. Transaction windows(display all the transactions )
5. output window(Display error and notification message)

What is .lrs?

Loadrunner save the information in a scenario files.

What is scenario wizard?

Through scenario wizard we can create a new scenario.

What is filtering and sorting?

We can filter the information display only those items that meet the selected criteria(filter box) exam you can filter Vuser only those who are in ready state.
Sorting we can sort all the Vuser in the vuser list. In order to their vuser ID(1,2,3,4,5,6,7,8,9).

What is the information creating for each host?    

1. The status of the host.
2. The platform type of the host(window/unix)
3 Details of the scenario.

How to create a host list for a scenario?

1. Install remote command launcher on every machine.
2. Add the name of the host to the host lists.
3. Set attributes for each host.
4. Select which hosts will take part in the scenario?

What the host attributes determine?

1. The maximum number of vuser that host can run.
2. The initialization quota.
3. The location of the winrunner configuration file.
4. The location of the file during runtime.

How you set maximum number of vuser that a host can run?

We can modify the maximum number of Vuser according to the (available resource, the needs of yours scenario, Loadrunner license arguments).

What do you mean by initialization of quota?

Capabilities of the host that at a time how many vuser are initialize.

 

What protocol does LR supports?

Industry slandered protocols .For EX: HTTP and ODBC are explicitly supported by LR. Further more any protocol that communicates over a windows socket can be supported.

What do i need to know to do load testing in addition to knowing how to use the LR tool?

In addition to knowing the tool:
-management aspects of load testing, planning being paramount
-requirements gathering, profile/mix,SLA,acceptance criteria
-an general understanding of the protocol yo are working with, developers can be unhealpfull
-a basic understanding of c programming
-know that you will be working wit demising time scales and you are really at the end of the life cycle
-as a result of the above you may have to work unsociable covers including week ends
- managers and other "Powers that be"-"BOX tickers" will not understand your plight
You need to be able to communicate effectively at all levels wit different departments from business to Dev to system testing
-voice your problems as soon as possible- planning
-fail to plan - plan to fail

What can i monitor with LR?

Monitor system bottle necks during a test run can capture and display the performance data from every server or component.

How many users can i emulate with LR on a pc?

Unlimited, no dead end.
Depends on system response. That to in turn depend on various factors like entire system configuration etc.
If system bottleneck observes in very beginning or minimum number of Vusers no further addition of Vusers will be considered unless observed bottle neck is resolved.

What r the Vuser comp[components in the LR?

Application components used are client, data base or additionally business application server
Web server works on and through LAN, WAN or WWW. connection.
Application server components are client, business server and data base server with out use of www but through protocols like ftp.

LR function - How to get current system time?

This function is developed to use in mercury load runner performance tool. The main use of this function to return the current system time at any given point of time while LR script is running. This function can be used to report transaction times, script start time and end time.
long get_secs_since_midnight(void)
{
char * curr_hr;/*pointer to a parameter with current clock hr */
char * curr_min;/*pointer to a parameter with current clock min */
char * curr_sec;/*pointer to a parameter with current clock sec */
long current_time; /*current number of seconds since midnight */
hr_secs;/*current hour converted to secs */
min_secs;/*current minuets converted to secs */
secs_secs;/*current number of seconds */
curr_hr=Lr_eval_string("{current_hr}>");
curr_min=Lr_eval_string("{current_min}");
curr_sec=Lr_eval_string("{current_sec}");

hr_secs=(atoi(curr_hr)) *60 * 60;
min_secs=(atoi(curr_min)) * 60;
secs_secs=atoi(curr_sec);

current_time=hr_secs+min_secs+secs_secs;

return (current_time);
}

What are the reasons why parameterization is necessary when load testing the web server and the data base server?

Parameterization is generally done to test with multiple set of data or records.

What is LR?

LR accurately measure and analysis the system performance and functionality .

When LR is used?

When multiple users were work concurrently

What is the advantage of using LR?

1. LR automatically records the performance of the client server during test.
2. LR ckecks where performance delauys occure netwoerk client delays.
3. LR monitor the network and server resource to help the improve performance.

What do you mean by remote command launcher?

RCL enables the controller to start the application on the host machine.

What is loadrunner agent?

Agent is interface between host machine and controller.

When the load runner controller opens the WinRunner file, then what is the location of the winrunner configuration file?

Wrun.ini

What is scenario default?

 

Instruct the Vuser to use the winrunner configuration file.

What is local configuration file?

Instruct the Vuser to use hosts winrunner configuration file.\ What do you mean by path?
Use winrunner configuration file i.e., in a specific location on the network.

During run-time where the hosts saves the files?

In temporally in the local drive of each host.

What is script list?

It contain all Vuser script that Vuser can run.

What are the information contained by script windows for each script in the list?

1. Name of the Vuser script.
2. Type of the Vuser script.
3. location(Path).
4. command line option.

how to modify the script?

Using vuser script information dialog box.

What is the purpose of running the scenario?

To check the response time of client or server system under load.

Why we insert the rendezvous point while running the scenario?

If a multiple Vuser to perform a task at exactly the same time.

When a scenario run a exactly, what happens?

1. The controller checks the scenario configuration information.
2. Then next it invoke the application that you select to run with the scenario.
3. Then transform each script to its related hosts, when the vuser are ready they start execution.

How to run a scenario?

Open an existing scenario
Configure the scenario
set the result directory
Run the scenario

When you initialize the Vuser what happens?

The Vuser status change from DOWN to PENDING to INITIALIZING to READY.
If Vuser files to initialize, the Vuser status changes to ERROR.

What is pause command?

It changes the status of the Vuser from RUNNING to PAUSE.

What is running virtual user graph?

It displays the number of Vusers that execute vuser script during each second of the scenario run. Only running and rendezvous state are include(loading, ready and pause are not displayed).

What is report viewer?

Each report viewer contains the report header and report viewer tool bar.

What is report header and what are the information contains?

It display general scenario information and it contains the information like title, scenario, result start time, end time and duration.

What is rendezvous graph?
It indicate when Vuser were released from rendezvous point and how many Vuser are released from each point. It help the transaction performance time.

What is transaction per second graph(PASS)?

It display the number of completed, successful transactions performed during each second of scenario run.

What is percentile graph?

The percentage of transaction that were performed within a given time range.

What is transaction performance graph?

Displays the average

 

Website testing

Q. How can World Wide Web sites be tested?

Web sites are essentially client/server applications - with web servers and 'browser' clients. Consideration should be given to the interactions between html pages, TCP/IP communications, Internet connections, firewalls, applications that run in web pages (such as applets, java script, plug-in applications), and applications that run on the server side (such as cgi scripts, database interfaces, logging applications, dynamic page generators, asp, etc.). Additionally, there are a wide variety of servers and browsers, various versions of each, small but sometimes significant differences between them, variations in connection speeds, rapidly changing technologies, and multiple standards and protocols. The end result is that testing for web sites can become a major ongoing effort. Other considerations might include:

• What are the expected loads on the server (e.g., number of hits per unit time?), and what kind of performance is required under such loads (such as web server response time, database query response times). What kinds of tools will be needed for performance testing (such as web load testing tools, other tools already in house that can be adapted, web robot downloading tools, etc.)?
• Who is the target audience? What kind of browsers will they be using? What kinds of connection speeds will them by using? Are they intra- organization (thus with likely high connection speeds and similar browsers) or Internet-wide (thus with a wide variety of connection speeds and browser types)?
• What kind of performance is expected on the client side (e.g., how fast should page appear, how fast should animations, applets, etc. load and run)?
• Will down time for server and content maintenance/upgrades be allowed? How much?
• What kinds of security (firewalls, encryptions, passwords, etc.) will be required and what is it expected to do? How can it be tested?
• How reliable are the site's Internet connections required to be? And how does that affect backup system or redundant connection requirements and testing?
• What processes will be required to manage updates to the web site's content, and what are the requirements for maintaining, tracking, and controlling page content, graphics, links, etc.?
• Which HTML specification will be adhered to? How strictly? What variations will be allowed for targeted browsers?
• Will there be any standards or requirements for page appearance and/or graphics throughout a site or parts of a site??
• How will internal and external links be validated and updated? how often?
• Can testing be done on the production system, or will a separate test system be required? How are browser caching, variations in browser option settings, dial-up connection variations, and real-world internet 'traffic congestion' problems to be accounted for in testing?
• How extensive or customized are the server logging and reporting requirements; are they considered an integral part of the system and do they require testing?
• How are cgi programs, applets, java scripts, ActiveX components, etc. to be maintained, tracked, controlled, and tested?
Some sources of site security information include the Usenet newsgroup 'comp.security.announce' and links concerning web site security in the 'Other Resources' section.
Some usability guidelines to consider - these are subjective and may or may not apply to a given situation (Note: more information on usability testing issues can be found in articles about web site usability in the 'Other Resources' section):
• Pages should be 3-5 screens max unless content is tightly focused on a single topic. If larger, provide internal links within the page.
• The page layouts and design elements should be consistent throughout a site, so that it's clear to the user that they're still within a site.
• Pages should be as browser-independent as possible, or pages should be provided or generated based on the browser-type.
• All pages should have links external to the page; there should be no dead-end pages.
• The page owner, revision date, and a link to a contact person or organization should be included on each page.
Many new web site test tools have appeared in the recent years and more than 280 of them are listed in the 'Web Test Tools' section.

Q. From the testability point of view what is the difference between client/server testing and web testing

Client Server testing is a three tier architecture and when testing has to be done on this we need to consider all types of testing like the stress testing , data - volume testing, load testing and performance testing.

When u are doing a normal web testing then you will be testing navigation testing, frame testing, broken links or missing URL's and static text testing.

Q. What type of testing is carried out to find memory-leakages? Give a sample example.

Through Volume testing it is possible. i.e., An application tries to retrieve large amount of data that require large temporary buffer area. If the data exceed the buffer area the situation of memory leakage will occur and query will fail without returning any result as sorting did not finish as buffer exceeds the limit.
We need to know the memory size before the test execution and after test execution by using memory related API functions or MFC functions.

Q. How to Test the Cookies and Memory leakages?

For cookie testing follow the below url
http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=2935

 

For memory leakage testing follow the below url:
http://www.liutilities.com/products/wintaskspro/whitepapers/paper1/  

Q. In n tier Architecture what are the factors should be considered for testing?

In 3 tier architecture there are 3 layers in the architecture. They are 1) Application (Presentation) layer 2)Business Logic Layer and 3)Data layer

In n tier architecture, Data layer is divided into 2 layers i.e. Data access and Database.

In n tier architecture, Data access layer and Database layer may or may not reside on the same location. Keeping that into consideration we have to prepare Test strategy and Test Approach
 

Q. During the password field testing. What should be the focus?

During password field testing, the below options should be given focus:

1. Password should be in encrypted form
2. The field cannot be copied either by right clicking of the mouse or by Ctrl+c option

Q. How to do browser testing (creates a standard script and run it for the different browser combinations.)

The GUI architecture and events messaging differs from browser to browser. Like IE uses Win32::OLE messaging and Firefox uses some GTK based messaging. So it is generally difficult to create one standard script that runs on all browsers. But tools like Win Runner, QTP use complex procedures inside them to handle different browsers. Manual testing can always be performed if the application supports different browsers like IE, Firefox, Opera, Netscape etc.

Q. What bugs are mainly come in web testing what severity and priority we are giving?

In web testing, mainly the bugs come from navigation area. These could be missing links, broken links, invalid links etc. Also there are bugs in downloading data/image/audio/video files from the website to the local machine and in uploading data/image/audio/video from local machine to the web server. Other than these a lot of bugs also come from the contents/look and feel/cosmetic issues.

 Q. Write test cases for a web URL.

1. Type URL in the address bar (for e.g. click www.yahoo.com) and click 'go' button.
2. Check to see whether the page is navigated to the yahoo home page.
3. If navigated to yahoo home page test case is passed else failed
4.also check to see when we enter the URL in the address bar and press the enter button in the keyboard it navigates to the yahoo home page.
5. When we click the refresh button in the yahoo home page the same page should be displayed. 

Q. what happens in a web application when you enter all the data and click on submit button
suddenly the connection goes off? Will the data be present if you return to the page?

If the data reaches the web server by the time of disconnection, the system will persist the data in the database .If the connection fails before reaching the server, the data won't be persisted and data will be lost.

Q. What are the important scenarios for testing emails? how do you test emails? which tool is best for testing email?

We can categorize the different part on which tester may perform the testing:

1. For incoming mail with attachment
2. For outgoing mail with attachment
3. Mail failure
4. Other operations like Delete, Edit etc.

1. For incoming mail with attachment:

Check the proper incoming address or id.
Check not only the One address (To) but also for the cc and Bcc Addresses.
Check the maximum and minimum limit for number of addresses.
Check if address has some error like @ or dot(.) is missing.
Check if address has more than 1 @ sign.
Check if dot(.) placing and the number of times it is present in the email address
Check the address should only have @, ., _ and - special symbols that are standard signs.
Check that mail does not have unnecessary contents with it self.
Check if there is/are any attachments then they should open properly.
Check the attachment size not exceeds the standard size.
Check if there is more than 1 attachment then the calculated size must be under the Standard size.
Check if the email content has some images or some different Flash picture must show properly.
Check the different extensions files attached with the mails shows properly.
Check if the user read the mail, then it should be marked as 'read'.

2. For outgoing mail with attachment:

All the scenarios that we mentioned above for incoming mails are valid for outgoing mails also. Hence all of them have to be verified.

3) Mail failure: Check mail failure if mail is send to incorrect address and also the failure notice should indicate the reason for failure.

4) Other properties like Delete, Edit etc should be verified.

Q. What is Web Services Testing? Have you done this?

Web Services testing is nothing, but testing the application. In this testing we just see whether the concept (functionality) of web services is working or not. 

Q. What things should be considered in usability testing of web application?

Usability testing is done for "user friendliness". In this we check how comfortable the customer is in using the application. Suppose for an example while logging in he forgot his password, in usability testing u have to check whether there is an "forgot password option" and if we click this it is asking for secret question or not and many things u can test like there should be minimize and maximize button for a window....and so on.

Q. How will you calculate the response time at server end?

This can be done by using Microsoft Visual Studio 2005 and 2008. In which you can find new ways to test a particular webpage such as load testing, website testing and so on. There is one software called Fidller which is used to calculate the traffic rate for a website which is currently used by a number of users.

Q. What test cases you will execute for compatibility testing with different browsers?

There are lot of issues which may arise while testing in different browsers.

Following are some points while comparing IE with Firefox, in IE following issues may occur which need compatibility testing but do not occur in Firefox

1. Java Script errors.
2. DIV Tag issue.
3. GetElementBy Issue.
4. Parent and Client Window size, pixel related issues may come.
5. After JS in IE sometimes it may not take us to further navigations.

Q. What are the main components of Performance Test Report?

The main components of performance test report are, Processor Time, User load per second, Memory use, Threshold States, Server Response and so on. When you carry out performance testing you will  get the result in the form of graph, then you can see the response when user load increased and decreased per second. You can also check the processor performance and can also check threshold value when load get increased suddenly. These are some of the main components of performance test report.

 Q. What errors can occur when the page loads?

The page may take too long to load...or the page may fail to load
by performing performance testing we can verify how the system behaves when subjected to or beyond specification and requirements load limits.
Perform configuration testing to determine how the system deals with hardware, software, operating systems, network conditions etc.

Q. How is Perl used for testing?

PERL is basically used for scripting. So it will be helpful when you need to automate your test cases. Also for result collection. It actually depends upon use.

Q. Give some test cases for testing a search engine website say Google.

The test cases for a search engine would be very vast. It totally depends upon the scope of testing. Some of the test cases are as mentioned below:

1) Check for simple strings like "European Premier League" or "Grammy Awards 2005".
2) Test the functionality of multiple page display by clicking on page number.
3) Verify whether a combination string works like "European Premier League_Christiano Ronaldo"
4) Perform test for opening the links in new windows. 

Q. What can be the the security checks on the web site, other than login/password screens?

Other than login/password....you can do other security testing checks like SQL injection methods, Cookie encryption testing, testing of authorization & authentication etc

Q. Suppose their is a website and after clicking OK in login window there is a window opening with the message "the page can not be displayed". Is it a bug?

This can be bug but we can not be sure before checking the below factors:
- Is the internet connection working fine?
- Is the browser on which the error comes supported for the software?
- Is the URL correct?
- Is the popup blocker off?
If the answer to all the above questions is yes then this is a bug

Q. How will you determine if the architecture of any web site is of two tier, three tier or multi tier?

The architecture of different tier can be determined by checking the client, server and database. If there is a client and a database then this is two tier architecture. If the web application has an application server and database then it is a three tier architecture.

Q. What are the main bugs found in browser compatibility testing?

Following are the main bugs found in browser compatibility testing
- Particular pages are not opening in every browser like (opera, Firefox, Netscape)
- Cookies are not available in particular browser
- Exact link is not opening
- Link is broken or not exist

Q. What do you understand by the terms 'Response Time', 'Pages Per Second', 'Transactions Per Second'?

Response time is the time taken by the server to give response to a particular action or request.
Pages per second gives the number of pages downloaded per second.
Transaction Response time is the time taken to perform a transaction in the scenario.

 Q. What are the models used for Testing Web Application?

All the SDLC models can be used for testing web applications. A web application is a combination of one or more modules. Depending upon the web application we use different models. Ex:- V-model, Spiral model and waterfall model.

Q. How to Calculate Session Time Out in Web Testing?

We can calculate the session time as below:

Scenario1:

Login into the application and put the application in the idle state for the time equal to or a little more than the prescribed session time out time. As the application is open, click on any of the links. It should give error that session has expired.

Scenario2:

Login into the application and put the application in the idle state for the time a little less than the prescribed session time out time. As the application is open, click on any of the links. The link should open the appropriate page.

Q. What are the different ways in which cookie testing can be done for a website?

Following are the different ways to perform cookie testing:

1. Disabling Cookies- This is probably the easiest area of cookie testing. What happens to the Web site if all cookies are disabled? Start by closing all instances of your browser and deleting all cookies from your PC set by the site under test. The cookie file is kept open by the browser while it's running, so you must close the browser to delete the cookies. Closing the browser also removes any per-session cookies in memory. Disable all cookies and attempt to use the site's major features and functions. Most of the time, you will find that these sites won't work when cookies are disabled. This isn't a bug, but rather a fact of life: disabling cookies on a site that requires cookies (of course!) disables the site's functionality.

2. Selectively Rejecting Cookies- What happens to the site if some cookies are accepted and others are rejected? Start by deleting all cookies from your PC set by the site under test and set your browser's cookie option to prompt you whenever a Web site attempts to set a cookie. Exercise the site's major functions. You will be prompted for each and every cookie the site attempts to set. Accept some and reject others. (Analyze site cookie usage in advance and draw up a test plan detailing what cookies to reject/accept for each function.) How does the site hold up under this selective cookie rejection? As above, does the Web server detect that certain cookies are being rejected and respond with an appropriate message? Or does the site malfunction, crash, corrupt data, or misbehave in other ways?

3. Corrupting Cookies- Along the way, as cookies are created and modified, try things like
a. Altering the data in the persistent cookies. Since the per-session cookies are stored only in memory, they aren't readily accessible for editing.
b. Selectively deleting cookies. Allow the cookie to be written (or modified), perform several more actions on the site, then delete that cookie. Continue using the site. What happens? Is it easy to recover? Any data loss or corrupted? 

4. Cookie Encryption - While investigating cookie usage on the site you're testing, pay particular attention to the meaning of the cookie data. Sensitive information like usernames and passwords should NOT be stored in plain text for all the world to read; this data should be encrypted before it is sent to your computer.

Q. What is the difference in testing a 'https' site and a 'http'?

HTTP is Hyper Text Transport Protocol and is transmitted over the wire via PORT 80(TCP). You normally use HTTP when you are browsing the web, it is not secure, so someone can eavesdrop on the conversation between your computer and the web server.HTTPS (Hypertext Transfer Protocol over Secure Socket Layer or HTTP over SSL) is a Web protocol developed by Netscape and built into its browser that encrypts and decrypts user page requests as well as the pages that are returned by the Web server. HTTPS is really just the use of Netscape's Secure Socket Layer (SSL) as a sub layer under its regular HTTP application layering. (HTTPS uses port 443 instead of HTTP port 80 in its interactions with the lower layer, TCP/IP.) SSL uses a 40-bit key size for the RC4 stream encryption algorithm, new-age browsers use 128-bit key size which is more secure than the former, it is considered an adequate degree of encryption for commercial exchange. HTTPS is normally used in login pages, shopping/commercial sites.

Q. What is scalability testing with respect to a website?

The purpose of scalability testing is to determine whether your application scales for the workload growth. Suppose your company expects a six-fold load increase on your server in the next two months. You may need to increase the server performance and to shorten the request processing time to better serve visitors. If your application is scalable, you can shorten this time by upgrading the server hardware, for example, you can increase the CPU frequency and add more RAM (also, you can increase the request performance by changing the server software, for example, by replacing the text-file data storages with SQL Server databases. To find a better solution, first you can test hardware changes, then software changes and after that compare the results of the tests).

If the scalability tests report that the application is not scalable, this means there is a bottleneck somewhere within the application.

Scalability testing can be performed as a series of load tests with different hardware or software configurations keeping other settings of testing environment unchanged. When performing scalability testing, you can vary such variables as the CPU frequency, number and type of servers, amount of available RAM, and so on 

Q. Which server statistics are essentially monitored during a Performance test of a Web Application?

Web server statistics, database server statistics, networks statistics are monitored during the performance of the web application.

Q. How do you test the server response time? Do you use any tool? How to do it manually?

You can check the server response time using Load Testing (Non functionality testing). For this Load runner tool can be used. You can check server response time manually for limited users. For doing this for large number of users heavy resources are required which can be easily done using Load runner tool. But this is difficult to do manually.

 Q. How to write test scenarios for a web based address book?

Following are a few scenarios:

1. Clicking the Addresses icon should open the address book. Existing contact information should be displayed
2. By clicking the Add Contact, should be able to add a contact.
3. Contact information should be editable by clicking the contact button and selecting edit
4. Contact information should be able to delete by clicking the contact and selecting delete
5. Should receive the message to add a contact after sending mail to a new contact
 

Q. How can u test the security of a web site both manually and by using a tool .If by a tool then which one and how?

Following are some test cases for testing security of websites manually:
1. User should not be able to login after entering incorrect username/password.
2. User information like id etc.. Should not be displayed along with the site address i.e. the browser.
3. After clicking logout user should not be able to access the application using back button.

For tools, Firewall can be used for this purpose.

 

Q/A1

What is Software Quality Assurance?

Software QA covers the complete software development process - monitoring and improving the process, making sure that all standards and procedures are followed, and guarantying that issues are found and dealt with. SQA is oriented to defect 'prevention'.

Which is better - black, gray or white box testing?

It's impossible to declare one of the testing approaches to be better then another. It depends of Quality Assurance Engineer skill set, the type of the project, what is trying to be achieved during testing.

What is gray box testing?

In recent years the term gray box testing has appear into common usage. Gray box testing is a software testing procedure that uses an amalgamation of black box testing and white box testing techniques.

With gray box testing approach, Quality Assurance Engineer does have the knowledge of some of the internal structure of the application under test. In gray box testing, Quality Assurance Engineer creates some test cases for the internal mechanism of the application under test. For the rest of the test case, Quality Assurance Engineer uses a black box approach in applying inputs to the application under test and validates the outputs.

What is white box testing?

Sometimes White box testing called in different Quality Assurance organizations as glass box testing or clear box testing, uses an internal perspective of the application under test to design test cases based on the knowledge of internal structure. In order to work as white box tester, the tester has to work with the application code and therefore is needed to possess knowledge of coding and logic.

What is the difference between QA and testing?

The main difference between QA and testing is that software quality assurance is oriented to defect 'prevention', while software testing is oriented to defect 'detection'. In other words testing measures the quality of a developed software application, QA measures the quality of processes used to create a quality software application.

What is Ad Hoc Testing?

Ad hoc software testing is a type of testing executed without documentation and planning. Ad hoc tests are intended to be run only once, unless a defect is discovered. Ad hoc testing is a part of exploratory testing.

What is Acceptance Testing?

Acceptance Testing is black box testing performed by customer to determine whether to accept a software product. Normally performed prior to software application delivery to validate the software application meets a set of agreed acceptance criteria.

What is test strategy?

The test strategy is defined set of methods and objectives that direct test design and execution. The test strategy describes the overall testing approach for the testing of application under test including stages of testing, completion criteria, and general testing techniques. The test strategy forms the basis for test plans

What is test harness?

In software quality assurance, a test harness is a software application configured to verify an application under test or test environment.

What is traceability matrix?

In software quality assurance, a traceability matrix can be used to show relationships between software requirements and test cases.

What is test suite?

In software quality assurance, a test suite is a collection of test cases used to validate the software program to show it has some defined set of behaviours. Usually a test suite holds prerequisite steps, clear goals and instructions for each collection of test cases in addition to information on the system and environment configuration to be used during testing and validation.

What tests shouldn't we automate?

Most types of testing benefit from automation, but some testing type's needs a real human attention and intelligence. It is possible, but difficult to automate GUI even with agile compatible tools like Selenium. Usability testing, exploratory testing and test that will never fail should not be considered as targets for test automation.

What tests should we automate?

Some testing type's needs human attention and intelligence, but most types of testing benefit from automation. In the same time QA Engineer should automate only that which needs automating. No one can automate 100 percent of testing work, but in certain areas like performance testing, load testing, stress testing regression testing, your team may have chance of reaching near to 100 percent of test automation. Other areas of easy automation would be API testing, test data set up and creation

Do you recommend using test automation in agile environment?

Proper test automation should be a core agile practice. Successful agile projects depend on test automation. Thriving agile teams expect to have working software all the time, which allows them to build and deploy production ready software application as often as customer needed. Agile teams cannot accomplish this goal without constant and proper testing. The following list contains main reasons for test automation in agile process:

Manual testing is long process
Manual testing is not repeatable process
Manual testing is error prone
Automation frees software engineer time
Automation regression tests offer a safety net
Automated tests provide feedback as often as needed
Automated tests become living documentation

What is stress testing?

Stress testing is used evaluate the application's behaviour when it is pushed beyond the normal or peak load conditions. The main goal of stress testing is to discover application issues that appear only under high load conditions. These can contain such issues as synchronization problems, race conditions, and memory leaks. Graceful performance degradation under high load leading to non-catastrophic failure is the desired result. Load test engineer could use the same scripts and tools as were used for performance testing, but using a very high level of simulated load.

What is performance testing?

Performance testing is used to determine the response time/latency, throughput, resource utilization (CPU, RAM, network I/O, disk I/O) and workload of a software application. The main goal of performance testing is to identify how well your application performs in relation to your performance objectives. The intent of the performance testing is not to break the application under test. The intent is to observe and document performance under expected usage conditions. There are performance test available to help simulate load, for example Apache JMeter, WebLOAD, LoadRunner and so on. Using test automation tools Load Test Engineer can simulate load in terms of users, connections, data, and in any other ways.

Why does test automation project fail?

The following problems are often encountered during test automation projects and may result in failing test automation projects:

• Management doesn't treat test automation as software development. Anyone can test and automation is easy – just record and playback.
  
• QA team select wrong set of test cases for test automation. In the same time, management aims for 100% test automation of all test cases.
  
• QA Engineers spend time between manual and automation testing, instead of concentrating on one task.
  
• No one realized that automation test cases are difficult to maintain and manage.
  
• The development, maintenance, and management of automated test scripts often need additional time and resources than manual test execution by inexperienced tester.
  

How do you know when to stop testing?

There are several ways for Quality Assurance Engineer to find out when testing stops:

• Release, testing or customer deadline has been met
  
• All test cases have been executed and certain predetermined percentage of the passed
  
• Predetermined code coverage percentage is met
  
• The number of found bugs or bug severity falls to a certain point
  
• No money left to continue testing
  

Could you name a few testing activities?

QA Engineer role usually includes a variety of testing activities:

• Create test plans and test cases
  
• Execute tests
  
• Log bugs
  
• Communicate with various team members like testers, developer and managers
  
• Make crucial decisions on whether something is a bug or a design constraint
  
• Schedule projects
  
• Allocate human and technical resources
  
• Make crucial decisions about the software applications
  
• Automate testing
  

What makes a good QA Engineer?

A good QA Engineer should able to perform the following tasks successfully in any environment:

Verification: A good QA Engineer can officially state that it is possible to accomplish certain tasks.
Detection: A good QA Engineer seeks issues that exist, either in the process or the product.
Prevention: A good QA Engineer recognizes potential issues before they become visible.
Reflection: A good QA Engineer looks back at how problems and bugs ended up in the product and analyzes this data to find out how to make the process better in the future.

Define test automation requirements for developing web application?

Imagine that you were asked to evaluate web application from test automation friendliest point of view. These criteria could be used to name web application automation friendly in order to test an application with SilktTest, QTP, Selenium or any other test automation tool.

• All web pages should have names
  
• Similar objects should have consistent names
  
• Unique names should be used for various objects
  
• All the images have ALT attribute test assigned
  
• Dynamic content should have a proper name or html id
  
• All tables displaying data should have names
  

How do you keep your testing skills updated?

• Read testing websites, magazines and books to understand latest trends in software testing industry.
  
• Read about new testing tools available in market.
  
• Practice by trying out different testing tools
  
• Attend professional testing conferences
  

Does your team use continuous integration?

If tester doesn't understand what interviewer means by continuous integration, the tester probably didn't work in a good software environment. How can QA Engineer get steady code build for testing if there is no bulletproof method of building and deploying code to testing and production environment? If there is no continuous integration process in place, QA Engineers most likely would spend time finding and reporting "show-stopper" and unit level bugs. The interviewee should be prepared to answer what source control (also known as version control, source control or (source) code management (SCM) systems they used. There are plenty of them around and most popular are SV, Perforce and VSS. The interviewee also needs to know about continuous integration software like CruiseControl, Bamboo or Hudson.

What's in your Testers Tool Box?

The test interview is not only a test of interviewee specific knowledge, but an opportunity to knowledge exchange. As an interviewer I have to spend at least half an hour interviewing some potential Quality Assurance Engineer and I want to use these minutes wisely. For example, I like to interview testers about various tools they use during preparation and actual testing. Here are some wonderful tools I use in my day-to-day testing routine:

Firebug - extension for Mozilla Firefox browser allows the debugging, editing, and monitoring of any website's CSS, HTML, DOM, and JavaScript;

Selenium - a free software testing framework for web applications;

Windows Virtual PC - is a virtualization suite for Microsoft Windows operating systems, and an emulation suite for Mac OS X on PowerPC-based systems. Virtual PC allows you create separate virtual machines on your Windows desktop;

Cygwin - a Unix-like environment and command-line interface for Microsoft Windows;

OpenSTA - GUI-based web server benchmarking utility that can perform scripted HTTP and HTTPS heavy load tests with performance measurements;

WinSCP - an open source SFTP and FTP client for Microsoft Windows;

HttpWatch – an HTTP Viewer and HTTP Sniffer for IE and Firefox;

 

What do you hate about testing?

Tester interview questions usually focused on positive results, like most obvious test interview question is what do you like about testing, but should it be the case. I believe asking reverse interview question would open the real mind of candidate for Test Engineer position and would perfectly describe the software development organization where Test Engineer works now.

The most hated term among all testers is "UI Automation" and the misunderstanding from management around "UI automation", thinking it is the silver bullet to all software development problems. As result the company spends money on unproductive test automation software.

Next most hated issue would be the developers. Some developers know how to test, when to test and what to test, another just throw the code over the fence with issue so bad that a basic sanity test could have caught as blocking issue.
The testers do not like managers, because every time the customer raises a defect in shipped product the management would question the testing team why this defect was missed during the testing cycle and who missed the issue instead of doing the root cause analysis for the defect. Some managers continuously call Test Engineer as Quality Assurance Engineer while Quality Assurance is a process not a title and request the software application to be QA's when the meant tested.
Of course test engineers hate themselves. There are testers who get comfortable with what they already know and stop pushing themselves to learn more, other testers doing the same manual tasks again and again, while basic automation should be applied and used.

Why do you want to leave your current job?

"I've been working with my wonderful company to advance the state of testing. My management has reached a point where they are satisfied with the state of quality assurance team, while I am still striving to improve in the art of quality assurance. I feel that I can no longer add value at my present company and it is time for me to start a new life"

I'm not sure I want to leave my company, but in the same time your job posting interested me and I really would like to talk about the opportunity your company has available.