Problem:
Are there any situations in which WebApp testing should be totally disregarded?
Answer:
No, there are no disregarded situations in the web app testing process. Because,
Web application testing strategy exercises the each quality dimension by initially examining the ‘units’ of content, functionality or navigation.
The web engineering test plan is developed and identifies the work products and mechanisms for the evaluation of test results.
Total this web app testing process is done through seven different types of testing.
Content testing:-
Focus on various categories of content.
Interface testing:-
Exercises the interaction mechanisms that enable a user to communicate with the web app and validates aesthetic aspects of the inter face.
Navigation testing:
Exercise the usage scenario against the navigation design.
Component testing:-
Exercise the content and functional units
Configuration testing:-
Attempts to uncover errors / compatibility problems.
Security testing:
The intent is to find security holes.
Performance testing:-
This is designed to assess web app response time and reliability as demands an server – side.
Problem:
In your own words, discuss the objectives of testing in a WebApp context.
Answer:
4633-20-2P SA: 9420
SR: 6376
The objectives of testing in a WebApp context are:
1) Review WebApp requirements.
2) Estimate the priorities and set the goal to each user that will be adequately tested.
3) Define WebApp testing strategy by describing the types of tests that will be conducted.
4) Develop test plan
• Define test schedule and assign testing responsibilities
• Specify automated testing tools
• Define acceptance criteria for each class of test
• Specify defect tracking mechanisms
• Define problem reporting mechanisms
5) Perform unit tests
• Review content for syntax and semantics errors
• Review content for proper permissions
• Test operation of interface mechanisms
• Test each component for proper function
6) Perform integration tests
• Test interface semantics against use-cases
• Conduct navigation tests
7) Perform configuration tests
• Assess client-side compatibility
• Assess server-side configurations
8) Conduct performance tests.
9) Conduct security tests.
Problem:
Compatibility is an important quality dimension. What must be tested to ensure that compatibility exists for a WebApp?
Answer:
592- 20-3P SA CODE: 4475
SR CODE: 4467
In Web Applications testing, Compatibility is the one of the quality dimension. Here, compatibility is tested by executing the Web App in a variety of different host configurations on both the client and server sides.
The main intent of this is, to find the errors that are specific to a unique host configuration because different browsers some times produce slightly different results, regardless of the degree of HTML standardization with in the Web App. For avoid this some plug-ins may or may not require.
In some cases, small compatibility issues present no significant problems, but in others,
Serious errors can be encountered.
Steps for compatibility.
Step1:
Define a set of commonly encountered client-side computing configurations and their variants
Step2:
Maintain this information (computing platform, typical display devices, operating system, available browsers, connection speeds) in a tree structure
Step3:
Derive compatibility validation test suite from existing interface tests, navigation tests, performance tests, and security tests
Step4:
Goal is to uncover execution problems that can be traced to configuration differences
Problem:
Which errors tend to be more serious—client-side errors or server-side errors? Why?
Answer:
Server-side
Is the WebApp fully compatible with the server OS?
Are system files, directories, and related system data created correctly when the WebApp is operational?
Do system security measures (e.g., firewalls or encryption) allow the WebApp to execute and service users without interference or performance degradation?
Has the WebApp been tested with the distributed server configuration (if one exists) that has been chosen?
Is the WebApp properly integrated with database software? Is the WebApp sensitive to different versions of database software?
Do server-side WebApp scripts execute properly?
Have system administrator errors been examined for their affect on WebApp operations?
If proxy servers are used, have differences in their configuration been addressed with on-site testing?
Client-side
Hardware—CPU, memory, storage and printing devices
Operating systems—Linux, Macintosh OS, Microsoft Windows, a mobile-based OS
Browser software—Internet Explorer, Mozilla/Netscape, Opera, Safari, and others
User interface components—Active X, Java applets and others
Plug-ins—QuickTime, RealPlayer, and many others
Connectivity—cable, DSL, regular modem, T1
The number of configuration variables must be reduced to a manageable number
Problem:
What elements of the WebApp can be “unit tested”? What types of tests must be conducted only after the WebApp elements are integrated?
Answer:
In the web applications, unit testing is focus on the verification of smallest unit of the software design. It should be consideration of software components (or) module.
In the module , the interface is tested to ensure that information properly follows into and out of the program unit. And the local data structure examined to boundary conditions of module operates properly.
The unit test is white box oriented and step can be conducted in parallel for multiple components.
In the web applications, the unit test is considering on
1) interface of module
2) Local data structure
3) Boundary conditions of the application.
4) Independent paths,
5) Error handling paths.
Among these, unit test cases should be designed to uncover errors, in correct comparisons or improper control flow.
Unit test environment for web application:
Integration testing is a systematic technique for constructing the program structure.
It consists of
Top down integration.
Bottom – up integration.
Smoke testing
After these testing process is completion for web application it must be conducted for validation testing. It can be defined in many ways.
Problem:
Is it always necessary to develop a formal written test plan? Explain.
Answer:
Yes, it is very necessary to develop a formal written test plan for a web application testing process.
Because, web application testing is consideration on some limitations / boundaries and test plan is consists of below parameters.
| S.No | Test case ID | Test case description | Actual value | Obtain value | Test case result |
And the web application test plan identifies the
1) a test set to be applied as testing commences
2) The work products to be produced as each test is executed.
3) The result of testing are evaluated recorder and reused.
Problem:
Is it fair to say that the overall WebApp testing strategy begins with user-visible elements and moves toward technology elements? Are there exceptions to this strategy?
Answer:
The WebApp testing strategy exercises each quality dimension by initially examining “units” of content, functionality, or navigation. Once individual units have been validated, the focus shifts to tests that exercise the WebApp as a whole. To accomplish this, many tests are derived from user’s perspective and are driven by information contained in use-cases. A Web engineering test plan is developed and identifies testing steps, work products (e.g., test cases), and mechanisms for the evaluation of test results.
Problem:
Is content testing really testing in a conventional sense? Explain.
Answer:
Yes, in conventional way. The content testing is necessary.
Content testing focus on various categories of content in the web application it has three important objectives.
1) To uncover syntactic errors in text – based document, graphical representations and other media.
2) To uncover semantic errors in any content object presented as navigation occurs
3) To find the errors in the structure of content that is presented to the end – user. And these content objections are exist with in an architecture with specific style. At this time, the structure and organization of the content architecture is tested to ensure that required content is presented to the end – user in the proper order.
Problem:
Describe the steps associated with database testing for a WebApp. Is database testing predominantly a client-side or server-side activity?
Answer:
592-20-9P SA CODE: 4475
SR CODE: 4467
Database testing of a Web-based application would basically deal with Integrity checks as well updating checks of the Database.Database testing is done mainly for five reasons: 1.relevance of search results 2.data integrity 3.data validity 4.data recovery 5.security
For instance we have an web-application dealing with Order booking, we might have different functionalities associated with the e like - 1) Registered User ID2) Number of Items Ordered3) The pricing of each Item4) Total calcualtion etc.
For each of these operations different tables in the DB would be getting updated. DB testing would basically involve - 1) Taking a snapshot of the Database tables in terms of number of records present before carrying out the transaction2) Taking a snapshot after functionality is carried out using the UI.The number of records count should have gone up3) checking if Relational dependency of tables is properly getting executed.In sense when Orders table is updated, the inventorty table of the item should reflect a decrease in that specified item4) checking if intermediate updations occur in the DB prior to completion of the specified functionality.
Database testing is the server side activity.
End
Problem:
What is the difference between testing that is associated with interface mechanisms and testing that addresses interface semantics?
Answer:
Interface mechanisms are tested using unit testing strategies and Each interface mechanism is tested in the context of a use-case of navigation semantic unit (e.g., thread) for a specific user category.
Tests conform to a checklist of functionality and features that are defined by the interface mechanism.
When users interact with a WebApp the interaction occurs through one or more Interface Mechanisms.
Those are
• Links (each link is listed and tested)
• Forms (check labels, field navigation, data entry, error checking, data transmission, meaningful error messages)
• Client-side scripting (black box testing and compatibility tests)
• Dynamic HTML (correctness of generated HTML and compatibility tests)
• Client-side pop-up windows (proper size and placement of pop-up, working controls, consistent with aesthetic appearance of Web page)
• CGI scripts (black box, data integrity, and performance testing)
• Streaming content (demonstrate existence, accuracy, and control over content display)
• Cookies (check that server constructs cookie correctly, cookie transmitted correctly, ensure proper level of persistence, check to see WebApp attaches the correct cookies to server requests)
• Application specific interface mechanisms
Where as in the interface semantics
Once each interface mechanism has been “unit” tested the focus of interface testing changes to a consideration of interface semantics.
This testing “ evaluates how well the design takes care of users, offers clear direction, delivers feedback, and maintains consistency of language and approach”.
Problem:
Assume that you are developing an online pharmacy (YourCornerPharmacy.com) that caters to senior citizens. The pharmacy provides typical functions, but also maintains a database for each customer so that it can provide drug information and warn of potential drug interactions. Discuss any special usability tests for this WebApp.
Answer:
4633-20-11P SA: 9420
SR: 6376
Usability tests for an on-line pharmacy WebApp:
Usability tests are designed to determine the degree to which the WebApp interface makes the user’s life easy. Various usability test categories are:
• Interactivity
• Layout
• Readability
• Aesthetics
• Display characteristics
• Time sensitivity
• Personalization
• Accessibility
As the WebApp is an online pharmacy WebApp and especially designed for senior citizens, the tests should be developed in that regard. Following are the usability tests designed specially for an on-line pharmacy WebApp.
• Interactivity: Are the interaction mechanisms easily understandable and user friendly?
• Layout: Are the navigation keys designed easily and to important pages?
• Readability: Does the site have proper readability with good font type and size?
• Aesthetics: is the color and the design are comfortable to the users? Are they not affecting the vision?
• Display characteristics: Does it have proper size and resolution?
• Time sensitivity: Are the responses being given in a timely manner? Does it interact continuously all the time without failure?
• Personalization: Can it update some updates regarding new drugs and warning?
• Accessibility: Is the WebApp accessible even to those who do not have proper vision and have any disabilities?
Problem:
Assume that you have implemented a drug interaction checking function for YourCornerPharmacy.com (Problem 20.11). Discuss the types of component-level tests that would have to be conducted to ensure that this function works properly. (Note: A database would have to be used to implement this function.)
Answer:
4633-20-12P SA: 9420
SR: 6376
Component level testing means module testing. That means we test each module or component differently in our WebApp (YourCornerPharmacy.com). It focuses on a set of tests that attempt to uncover errors in WebApp functions.
The WebApp can be tested using the techniques such as,
(1) Black/White box testing
(2) Integration testing
(3) System testing.
Different types of component level tests that would have to be conducted to ensure that the WebApp is functioning properly. They are
• Equivalence partitioning: The input domain of the function is divided into input categories or classes from which test cases are derived. The input form is assessed to determine what classes of data are relevant for the function. Test cases for each class of input are derived and executed, while the other classes of input are held constant.
• Boundary value analysis: Forms data are tested at their boundaries.
• Path testing: If the logical complexity of the function is high, path testing can be used to ensure that every independent path in the program has been exercised.
• Forced error testing: A technique called forced error testing is also used. The purpose of this test is to uncover errors that occur during error handling.
• Database testing: As the correct execution of our WebApp (YourCornerPharmacy.com) is tied to proper interfacing with a database, database testing, which is an integral part of the component-testing, is also to be done.
Problem:
What is the difference between testing for navigation syntax and navigation semantics?
Answer:
Navigation Testing need to ensure that all mechanisms that allow the WebApp to user to travel through the WebApp are functional and Need to validate each navigation semantic unit
Testing Navigation Syntax
In this phase, navigation testing actually begins during interface testing and mechanisms are tested to ensure that each performs is intended function. Navigation mechanism should be tested to
Navigation Links
Redirects
Bookmarks
Frames and framesets
Site maps
Internal search engines
Testing Navigation Semantics
A set of information and related navigation structure that collaborate in the fulfillment of a subset of related user requirements.
Navigation semantic units are defined by a set of pathways that connect navigation nodes
Each NSU must allows a user from a defined user category achieve specific requirements defined by a use-case.
If a function if to be executed at a node and an error in function processing occurs, can the NSU be completed?
Testing needs to ensure that each path is executed in its entity without error
Every relevant path must be tested
User must be given guidance to follow or discontinue each path based on current location in site map
Problem:
Is it possible to test every configuration that a WebApp is likely to encounter on the server side? On the client side? If it is not, how do you select a meaningful set of configuration tests?
Answer:
592-20-14P SA Code: 4467
SR Code: 4578
On the server side, configuration test cases are designed to verify that the projected server configuration (i.e., WebApp server, database server, and operating_system, and firewall software, concurrent applications) can support the WebApp without error. In the essence, the WebApp is installed within the server-side environment and tested to ensure that it operates without error. As server-side configuration tests are designed, the Web engineer should consider each component of the server configuration.
On the client side, configuration tests focus more heavily on WebApp compatibility with configurations that contain one or more permutation. To design client-side configuration tests, the Web engineering team must reduce the number of configuration variables to a manageable number (this may mean restricting the number of configurations supported for the WebApp).
Problem:
What is the objective of security testing? Who performs this testing activity?
Answer:
Security testing
Security testing improves the understanding of the processes of the test vector generation, test code generation, results analysis and attempt to devise test cases that subvert the program's security checks.
Due to increasing concern about privacy, many programs have specific security objectives.
Security testing Activity is performed by security test engineer.
Objectives:
• To formulate the test cases that subverts an operating system's memory - Protection mechanism.
• To subvert a data-base-management system's data-security mechanisms.
• To devise test cases to study the known security problems in similar systems and generate test cases that attempt to demonstrate similar problems in the system at hand.
Identification of architectural, design, and implementation risks:
• Risk-driven test creation
• Dependency attacks
• User Interface attacks
• File system attacks
• Design attacks
• Implementation attacks
• Penetration testing
• Static vulnerability scanning
• Test coverage
• Test depth analysis
Problem:
YourCornerPharmacy.com (Problem 25.11) has become wildly successful, and the number of users has increased dramatically in the first two months of operation. Draw a graph that depicts probable response time as a function of number of users for a fixed set of server-side resources. Label the graph to indicate points of interest on the “response curve.”
Answer:
The response time or throughput of a server can be calculated by analyzing the following computation:
P = N*T*D
Here, the throughput P, can be analyzed in terms of N, i.e. number of users when other server-side resources are kept under a limit. T is the average number of transactions happening online per unit of time and D represents the data which is processed by the server for each transaction. To observe the variation in the response time of the server as a function of N, it can be clearly observed that the graph would be a linear graph. Keeping T and D constants, the response time increases with the increasing value of N, linearly as shown in the graph below:
As can be observed from the above graph the response time would definitely increase with increasing number of users if the server-side resources are fixed.
Problem:
In response to it, success YourCornerPharmacy.com (Problem 25.11) has implemented a special server solely to handle prescription refills. On average, 1000 concurrent users submit a refill request once every two minutes. The WebApp downloads a 500-byte block of data in response. What is the approximate required throughput for this server in megabits per second?
Answer:
The throughput of a server can be calculated by using the following computation:
P = N*T*D
Here, the throughput P, can be analyzed in terms of N, i.e. number of users when other server-side resources are kept under a limit, T i.e. the average number of transactions happening online per unit of time and D represents the data which is processed by the server for each transaction.
The number of concurrent users is 1000. Therefore, N = 1000.
A request is submitted once in two minutes i.e. T = ½ or 0.5.
Server processes 500-btye data in response. Therefore, D = 500 byte or 500 * 8 (=4000)bits
Thus 4KB data is processed for each request.
The throughput can thus be calculated as follows:
P = (1000*0.5*4KB)/60 sec
= 33.33 KB/sec
Therefore, the throughput is 0.033 MB/sec.
Problem:
What is the difference between load testing and stress testing?
Answer:
592-20-18P SA CODE: 4475
SR CODE: 4467
Load testing and Stress testing both are uses for check the performance of a web application.
Load testing attempts to determine how the WebApp and its server-side environments respond to various load conditions.
Where as Stress testing is a continuation of load testing that seeks to have the number of user, transactions, and data load meet and exceed operational limits.
Load Testing:
It is determined that how the Web App and its server side environment will respond to various loading conditions. And Load Testing determine combinations of N, T, and D that cause performance to degrade here, N = number of concurrent users T = number of on-line transactions per unit of time D = data load processed by server per transaction
Overall through put is computed using the equation P = N * T * D
Where as Stress Testing is continuation of load testing and variables N,T, and D are forced to meet and then exceed operational limits.
Stress Testing considering the following queries.
What values of N,T,and D force the server environment to fail? How does failure manifest itself? Are automated notifications sent to technical support staff at server site?
->Does system degrade gracefully?
->Are users made aware that they cannot reach the server?
->Does server queue resource requests during heavy demand and then process the queue ->when demand lessens?
->Are transactions lost as capacity is exceeded?
->Is data integrity affected when capacity is exceeded?
->How long till system comes back on-line after a failure?
->Are certain Web App functions discontinued as capacity is reached?
Solution: Chapter 25: TESTING WEB APPLICATIONS
25.1 No. The goal of WebApp testing is to exercise each of the many dimensions of WebApp quality with the intent of finding errors or uncovering issues that may lead to quality failures. Testing focuses on content, function, structure, usability, navigability, performance, compatibility, interoperability, capacity, and security and incorporates reviews that occur as the WebApp is designed and tests that are conducted once the WebApp has been implemented. The WebApp testing strategy exercises each quality dimension by initially examining “units” of content, functionality, or navigation. Once individual units have been validated, the focus shifts to tests that exercise the WebApp as a whole.
25.2 Testing is the process of exercising software with the intent of finding (and ultimately correcting) errors. This fundamental philosophy does not change for WebApps. In fact, because Web-based systems and applications reside on a network and interoperate with many different operating systems, browsers (or other interface devices such as PDAs or mobile phones), hardware platforms, communications protocols, and “backroom” applications, the search for errors represents a significant challenge for Web engineers. To understand the objectives of testing within a Web engineering context, we must consider the many dimensions of WebApp quality.
25.3 Compatibility is tested by executing the WebApp in a variety of different host configurations on both the client and server sides. The intent is to find errors that are specific to a unique host configuration.
25.4 Because WebApps reside within client/server architecture, errors can be difficult to trace across three architectural layers: the client, the server, or the network itself. However, server-side errors are more likely to affect a large number of users and may not give any indication and error has occured (other than failure to deliver content).
25.5 Once each interface mechanism has been “unit” tested, the focus of interface testing changes to a consideration of interface semantics. Interface semantics testing “evaluates how well the design takes care of users, offers clear direction, delivers feedback, and maintains consistency of language and approach.” The element sof a WebApp that can be unit tested are the components (e.g. scripts) that deliver WebApp functional units.
25.6 Testing shouldn’t wait until the project is finished. Start testing before you write one line of code. Test constantly and effectively, and you will develop a much more durable Web site. Since analysis and design models cannot be tested in the classical sense, the Web engineering team should conduct formal technical reviews as well as executable tests. The intent is to uncover and correct errors before the WebApp is made available to its end-users. A Web engineering test plan identifies (1) the task set to be applied as testing commences; (2) the work products to be produced as each testing task is executed, and (3) the manner in which the results of testing are evaluated, recorded, and reused when regression testing is conducted. Hence Formal written test plan is usually necessary for crucial applications.
25.7 The WebApp testing strategy exercises each quality dimension by initially examining “units” of content, functionality, or navigation. Once individual units have been validated, the focus shifts to tests that exercise the WebApp as a whole. To accomplish this, many tests are derived from users perspective and are driven by information contained in use-cases. A Web engineering test plan is developed and identifies testing steps, work products (e.g., test cases), and mechanisms for the evaluation of test results. The testing process encompasses seven different types of testing.
25.8 Content testing is more like proof reading (and reviews) in that it focuses on various categories of content. The intent is to uncover both semantic and syntactic errors that affect the accuracy of content or the manner in which it is presented to the end-user. Interface testing exercises the interaction mechanisms that enable a user to communicate with the WebApp and validate aesthetic aspects of the interface. The intent is to uncover errors that result from poorly implemented interaction mechanisms, or omissions, inconsistencies or ambiguities in interface semantics.
25.9 Database testing for WebApps is complicated by a variety of factors:
1)The original client-side request for information is rarely presented in the form (e.g., structured query language, SQL) that can be input to a database management system (DBMS). Therefore, tests should be designed to uncover errors made in translating the user’s request into a form that can be processed by these DBMS.
2)The database may be remote to the server that houses the WebApp.
3)Raw data acquired from the database must be transmitted to the WebApp server and properly formatted for subsequent transmittal to the client.
4)The dynamic content object(s) must be transmitted to the client in a form that can be displayed to the end-user. Considering these four factors, test case design methods should be applied for each of the “layers of interaction”.
25.10 Interface testing exercises interaction mechanisms and validates aesthetic aspects of the user interface. The intent is to uncover errors that result from poorly implemented interaction mechanisms or omissions, inconsistencies or ambiguities that have been introduced into the interface inadvertently. Testing of interface semantics focuses on making that user can navigate to correct content without any problems.
25.11 Answers will vary
25.12 Boundary test the minimum and maximum number of items that can be placed in the shopping cart.
- Test a “check out” request for an empty shopping cart.
- Test proper deletion of an item from the shopping cart.
- Test to determine whether a purchase empties the cart of its contents.
- Test to determine the persistence of shopping cart contents (this should be specified as part of customer requirements).
- Test to determine whether the WebApp can recall shopping cart contents at some future date (assuming that no purchase was made).
25.13 The first phase of navigation testing actually begins during interface testing. Navigation mechanisms are tested to ensure that each performs its intended function. Some of the tests can be performed by automated tools (e.g., link checking) while others are designed and executed manually. The intent throughout is to ensure that errors in navigation mechanics are found before the WebApp goes on-line. ]. Each Navigation Semantic unit (NSU) is defined by a set of navigation paths (called ‘ways of navigating’) that connect navigation nodes (e.g., Web pages, content objects, or functionality). Taken as a whole, each NSU allows a user to achieve specific requirements defined by one or more use-cases for a user category. Navigation testing exercises each NSU to ensure that these requirements can be achieved.
25.14 On the server side, configuration test cases are designed to verify that the projected server configuration (i.e., WebApp server, database server, operating system(s), and firewall software, concurrent applications) can support the WebApp without error. In essence, the WebApp is installed within the server-side environment and tested to ensure that it operates without error. As server-side configuration tests are designed, the Web engineer should consider each component of the server configuration. On the client side, configuration tests focus more heavily on WebApp compatibility with configurations that contain one or more permutation. To design client-side configuration tests, the Web engineering team must reduce the number of configuration variables to a manageable number (this may mean restricting the number of configurations supported for the WebApp).
25.15 Security testing incorporates a series of tests designed to exploit vulnerabilities in the WebApp and its environment. The intent is to demonstrate that a security breach is possible. A Web engineer who has access to the client’s environment would need to do security testing.
25.16 Answers will vary
25.17 Answers will vary
25.18 Load testing—real-world loading is tested at a variety of load levels and in a variety of combinations. The intent of load testing is to determine how the WebApp and its server-side environment will respond to various loading conditions. As testing proceeds, permutations to the following variables define a set of test conditions: N, the number of concurrent users, and the number of on-line transactions per unit of time D, the data load processed by the server per transaction.
Stress testing—loading is increased to the breaking point to determine how much capacity the WebApp environment can handle. Stress testing is a continuation of load testing, but in this instance the variables, N, T, and D are forced to meet and then exceed operational limits.