Rational Functional Tester: Test Automation Architecture
Posted by Matt Archer on October 28, 2008
This post is part 2 of a series of 5 posts about test automation with Rational Functional Tester. I originally created the material as a white paper in 2004 when I was coaching a project that was new to GUI-level automation and also to RFT. I recently discovered it during a clean up and whilst some on my ideas have changed over the past 4 years, I’m happy that most of it still holds up as good advice. Maybe one day I’ll update it.
Identify Custom Recognition Mechanisms
IBM Rational Functional Tester provides a Test Object Map that contains a list of every Test Object that is interacted with during the recording process. Each object is assigned a unique name so that it can be referenced from within the automated test script. For the majority of tests, the Test Object Map provides a sufficient mechanism for recognising and interacting with the Test Objects within the application. However, there are some occasions when it is not desirable to use the Test Object Map and a custom recognition method is written to find and retrieve the necessary Test Object. Review the parts of the application you intend to automate and identify where a custom recognition method may be necessary. Often this activity will need to be accompanied by an investigation of how IBM Rational Functional Tester interacts with the project’s application and how Test Objects are stored within the Test Object Map.
For each custom recognition method that is identified, add its description to the project’s test automation architecture document. As a minimum, express the custom recognition method in terms of its signature and an example of when the method can be utilised to benefit the tester. It is likely that this section will evolve over the life of the project as new custom recognition needs are identified. As this occurs, ensure that you keep the test automation architecture document up-to-date.
Identify Custom Verification Mechanisms
IBM Rational Functional Tester provides three distinct methods for verifying the state of the application under test. They are Object Data, Object Property and Object Existence. For the majority of tests, the relevant verification points can be implemented using one of the standard verification methods. However, there are some occasions when the test cannot be reliably interpreted as a pass or fail and consequently a custom verification mechanism must be produced to obtain an accurate result. Review the parts of the application you intend to automate and identify where a custom verification method may be required. Often this activity will need to be accompanied by an investigation of how IBM Rational Functional Tester interacts with the project’s application and how Test Objects can be interrogated (using the three standard verification methods) to extract the application’s state.
For each custom verification point that is identified, add its description to the project’s test automation architecture document. As a minimum, express the custom verification method in terms of its signature and an example of when the method can be utilised to benefit the tester. It is likely that this section will evolve over the life of the project as new custom verification needs are identified. As this occurs, ensure that you keep the test automation architecture document up-to-date.
Identify Test Bed Components
The key to successful automated testing is a well-defined and maintained test environment. Combined with the state of the application (typically defined by the information that it contains within the application database) this is known as the test bed. Before each automated test cycle begins, it is important for the test team to reset the test bed to a known state to avoid misleading results and false failures.
Investigate the application under test and identify the different components that together form the test bed. Some of the test bed components will remain static during testing (typically, the operating system), whilst other test bed components will change as a result of the automated testing. Document each test bed component and note which components are dynamic and which are static. For each dynamic test bed component detail how it will be manually reset to a known state before automated test execution can begin. This information is typically stored in the project’s test automation architecture document.
Capture Test Bed Automation Needs
Resetting a test bed before test automation begins can be a time consuming activity. As your familiarity with IBM Rational Functional Tester increases you should consider how you can automate resetting the test bed (potentially to a variety of different configurations). The complexity of automating such a task will vary greatly depending on the application under test and the test team’s ability to acquire a self-contained test bed. Ensure that the benefits received by automating the resetting of the test bed outweigh the cost of developing the necessary code within IBM Rational Functional Tester. For each dynamic test bed component that you decide to automate the resetting of, detail how it will be achieved using IBM Rational Functional Tester. This information is typically stored in the project’s test automation architecture document.
Define Failure Recovery Routines
One of the distinct benefits of automated testing is the ability to verify the quality of an application without any human interaction. On occasions (due to test script faults, defects within the application, and incorrect test bed configurations, to mention a few) the automated tests will become unsynchronised with the application under test. That is to say, the automated test is expecting to interact with a specific test object, but that test object cannot be found as the application is displaying another window or has closed / frozen due to a serious application crash. In summary the application under test is not in the same state that the automated test is expecting. If a synchronisation error occurs it will typically leave any unexecuted automated tests incapable of running. To minimise the number of affected tests, the test team should consider incorporating some form of failure recovery into their automation testing solution. The sophistication of a failure recovery routine will depend on the ability to control the test bed using IBM Rational Functional Tester. It is recommended that a test team start with simple failure recovery routines, such as those used to handle unexpected active windows. As the test team’s experience of IBM Rational Functional Tester increases, they should consider additional failure recovery routines, such as those that are capable of completely resetting the test bed to a known baseline. As the test team identifies failure recovery routines they should be documented within the project’s test automation architecture document. As a minimum, each failure recovery routine should be described in terms of the criteria (or signal) that will trigger the routine, the test bed components affected and the state to which the test bed components will be returned.
Define Test Data Structure and Storage Needs
One of the powers of automated testing is the ability to perform the same scenario multiple times, with different variants of test data. To take advantage of this power, the test data must be stored in a maintainable format, which can be easily accessed by both the test team and the automated test script. As a rule of thumb, large volumes of hard-coded values should be avoided and the test data should be stored independently of the automated test script. The possibilities for independent data storage are endless, but a common set of options exist, including IBM Rational Datapools, Excel spreadsheets and external databases. Investigate which test data structure and type of storage best fits the test team’s needs. Document the location of the test data, its structure and any special routines required to access the test data in the project’s test automation architecture document.
Define Package Structure
As you automated testing solution grows it will quickly expand to a large number of elements, including various test scripts, datapools and verification points. To avoid your automated testing solution becoming unmanageable and difficult to maintain, it is a good idea to agree a package structure before your begin the majority of your automating testing development. The structure of a project’s packages will be determined by their specific needs. However, it is typical to subdivide test scripts into separate packages that represent functional areas, application feature or use-case. A separate temporary ‘scratch’ package to keep ideas for new techniques and approaches can also be useful. A project’s package structure should be recorded within the test automation architecture document.
Agree Naming Conventions
Maintaining an automated testing solution can be significantly aided by using a common set of naming conventions throughout the entire test team. Naming conventions will also increase readability and help other testers read your automated test scripts as if they were their own. Many organisation own naming conventions that apply to other area of the software development lifecycle. Where appropriate, the test team may also use these conventions. For elements that are specific to the test discipline, such as verification points, various test scripts, and datapools, the test team should define their own naming conventions and record them in the project’s test automation architecture document.
Identify Common-Code Owner and Policy
Reusing common-code within an automated testing solution can dramatically reduce development time and increase maintainability by providing a single point fix. IBM Rational Functional Tester supports the use of common-code through a Helper Super Class that is inherited by each test scripts. As each test script within a Functional Tester Project inherits from the Super Helper Class any changes made to the Super Helper Class may affect the way in which each test script performs its tests. Therefore any changes made to the Super Helper Class (that is to say, the common-code) should be controlled. The formality of this management will vary depending on the size and complexity of the project and the associated automated testing solution. Look within your project team and identify a person or group of people that will act as the common-code owner. It will be the common-code owner’s responsibility to review and agree any changes and additional to the Super Helper Class. Within the project’s test automation architecture document identify the project’s common-code owner and how they will handle requests to alter the Super Helper Class. Also describe how requests to the common-code are to be submitted and managed.