Software Testing using Visual Studio 2010

A step by step guide to understand the features and concepts of testing applications using Visual Studio.

• Master all the new tools and techniques in Visual Studio 2010 and the Team Foundation Server for testing applications
• Customize reports with Team foundation server.
• Get to grips with the new Test Manager tool for maintaining Test cases
• Take full advantage of new Visual Studio features for testing an application’s User Interface
• Packed with real world examples and step by step instructions to get you up and running with application testing

Software testing in Visual Studio 2010

Before getting into the details of the actual testing using Visual Studio 2010 let us find out the different tools provided by Visual Studio 2010 and their usage and then we can execute the actual tests. Visual Studio 2010 provides different tools for testing and management such as the Test List Editor and the Test View. The test projects and the actual test files are maintained in Team Foundation Server (TFS) for managing the version control of the source and the history of changes. Using Test List Editor we can group similar tests, create any number of Test Lists, and add or delete tests from a Test List.

The other aspect of this article is to see the different file types generated in Visual Studio during testing. Most of these files are in XML format, which are created automatically whenever a new test is created.

For the new learners of Visual Studio, there is a brief overview on each one of those windows. While we go through the windows and their purposes, we can check the Integrated Development Environment (IDE) and the tools integration into Visual Studio 2010.

Testing as part of the Software Development Life Cycle

The main objective of testing is to find the defects early in the SDLC. If the defect is found early, then the cost will be lower than when the defect is found during the production or implementation stage. Moreover, testing is carried out to assure the quality and reliability of the software. In order to find the defect as soon as possible, the testing activities should start early, that is in the Requirement phase of SDLC and continue till the end of the SDLC.

In the Coding phase various testing activities take place. Based on the design, the developers start coding the modules. Static and dynamic testing is carried out by the developers. Code reviews and code walkthroughs are conducted by the team.

Once the coding is complete, then comes the Validation phase, where different phases or forms of testing are performed:

• Unit Testing: This is the first stage of testing in the SDLC. This is performed by the developer to check whether the developed code meets the stated functionality. If there are any defects found during this testing then the defect is logged against the code and the developer fixes it.
  The code is retested and then moved to the testers after confirming the code without any defects for the purpose of functionality. This phase may identify a lot of code defects which reduces the cost and time involved in testing the application by testers, fixing the code, and retesting the fixed code.
• Integration Testing: This type of testing is carried out between two or more modules or functions together with the intention of finding interface defects between them. This testing is completed as a part of unit or functional testing and, sometimes, becomes its own standalone test phase. On a larger scale, integration testing can involve putting together groups of modules and functions with the goal of completing and verifying that the system meets the system requirements. Defects found are logged and later fixed by the developers. There are different ways of integration testing such as top-down and bottom-up.
  • The Top-Down approach is intended to test the highest level of components and integrate first to test the high level logic and the flow. The low level components are tested later.
  • The Bottom-Up approach is exactly opposite to the top-down approach. In this case the low level functionalities are tested and integrated first and then the high level functionalities are tested. The disadvantage of this approach is that the high level or the most complex functionalities are tested later.
  • The Umbrella approach uses both the top-down and bottom-up patterns. The inputs for functions are integrated in the bottom-up approach and then the outputs for functions are integrated in the top-down approach.
• System Testing: This type of testing compares the system specifications against the actual system. The system test design is derived from the system design documents and is used in this phase. Sometimes system testing is automated using testing tools. Once all the modules are integrated, several errors may arise. Testing done at this stage is called system testing. Defects found in this type of testing are logged by the testers and fixed by the developers.
• Regression Testing: This type of testing is carried out in all the phases of the testing life cycle, once the defects logged by the testers are fixed by the developers or if any new functionality changes due to the defects logged. The main objective of this type of testing is testing with the intention of determining if bug fixes have been successful and have not created any new defects. Also, this type of testing is done to ensure that no degradation of baseline functionality has occurred and to check if any new functionality that was introduced in the software caused prior bugs to resurface.

Types of testing

Visual Studio provides a range of testing types and tools for testing software applications. The following are some of those types:

• Unit test
• Manual test
• Web Performance Test
• Coded UI Test
• Load Test
• Generic test
• Ordered test

In addition to these types there are additional tools provided to manage, order the listing, and execution of tests created in Visual Studio. Some of these are the Test View, Test List Editor, and Test Results window. We will look at the details of these testing tools and the supporting tools for managing testing in Visual Studio 2010.

Unit test

Unit testing is one of the earliest phases of testing the application. In this phase the developers have to make sure the code is producing the expected result as per the stated functionality. It is extremely important to run unit tests to catch defects in the early stage of the software development cycle. The main goal of unit testing is to isolate each piece of the code or individual functionality and test if the method is returning the expected result for different sets of parameter values.

A unit test is a functional class method test which calls a method with the appropriate parameters, exercises it, and compares the results with the expected outcome to ensure the correctness of the implemented code. Visual Studio 2010 has great support for unit testing through the integrated automated unit test framework, which enables the team to write and run unit tests.

Visual Studio has the functionality to automatically generate unit test classes and methods during the implementation of the class. Visual Studio generates the test methods or the base code for the test methods but it remains the responsibility of the developer or the team to modify the generated test methods and to include the code for actual testing. The generated unit testing code will contain several attributes to identify the Test Class, Test Method, and Test Project. These attributes are assigned when the unit test code is generated from the original source code. Here is a sample of the generated unit test code.

A Unit test is used by developers to identify functionality change and code defects. We can run the unit test any number of times and make sure the code delivers the expected functionality and is not affected by new code change or defect fix.

All the methods and classes generated for the automated unit testing are inherited from the namespace Microsoft.VisualStudio.TestTools.UnitTesting.

Manual test

Manual testing is the oldest and the simplest type of testing but yet very crucial for software testing. The tester would be writing the test cases based on the functional and non-functional requirements and then testing the application based on each test case written. It helps us to validate whether the application meets various standards defined for effective and efficient accessibility and usage.

Manual testing comes to play in the following scenarios:

• There is not enough budget for automation
• The tests are more complicated or too difficult to convert into automated tests
• Not enough time to automate the tests
• Automated tests would be time consuming to create and run

The tested code hasn’t stabilized sufficiently for cost effective automation.

We can create manual tests by using Visual Studio 2010 very easily. The most important step in a Manual test is to document all the required test steps for the scenario with supporting information, which could be in a separate file. Once all the test cases are created, we should add the test cases to the Test Plan to be able to run the test and gather the test result every time we run the test. The new Microsoft Test Manager tool helps us when adding or editing the test cases to the Test Plan. The following are additional Manual testing features that are supported by Visual Studio 2010:

• Running the Manual test multiple times with different data by adding parameters
• Create multiple test cases using an existing test case to get the base test case first and then customize or modify the test
• Sharing test steps between multiple test cases
• Remove test cases from the test if not required
• Adding or copying test steps from Microsoft Excel or Microsoft Word or any other supported tool

There are a lot of other manual testing features that are supported in Visual Studio 2010.