Introduction
The OpenClovis Test Automation Environment (TAE) is a powerful framework designed to automate software testing across various applications, especially applications related to telecom. The TAE ensures minimal impact on the final application while still providing robust self-testing capabilities. This article explores the structure, functionality, and usage of TAE.
What is the TAE?
This is the OpenClovis Test Automation Environment (TAE). It is designed to run programs written for various purposes, such as test, demo, eval, and real systems. By supplying a heavy “class” framework, most automated test frameworks’ client libraries force the application into a specific unit test mold. So the only software that can be run are test suites designed from the bottom-up to fit within the automated test framework. The TAE allows adding test-related code into an application even if it wasn’t originally designed for testing. The tests can be enabled or disabled at compile time, so they don’t necessarily impact the final application unless required. If the tests detect an issue, the application can report errors to the web server.
Organization
The TAE is intended to be used both as an individual developer’s test bench and as an enterprise-wide test automation solution. It therefore is organized in two independent parts: first, the web interface and historical database that is generally deployed on a single machine, and second, the testing engine which is often deployed on every developer’s machine and on an automated (nightly) build and test server. This allows the test automation environment to be run in a variety of different situations and by a variety of different users. For example, it could be run by an automated nightly verification system or by a developer fixing a bug. Also, tests can be run at different physical sites (or even by multiple companies cooperating on the same project) yet all the results can be uploaded to a single centralized database and web interface for project analysis.
The TAE is organized into 4 layers:
- Layer 1: Unit testing API (Clovis Test API)
The smallest layer is the API used in a program to run tests. This layer consists of a set of C macros that group and run tests. When not testing — when a “release” build is created for example — these macros are compiled out. This is very similar to unit testing packages found for many languages.
- Layer 2: System Test API
This API layer allows tests to be written in Python programming language and runs them on a machine that sits outside of the device-under-test. Through these APIs, your tests can access the entire cluster and start/stop applications (or unit tests), cause failures of processes or nodes, and access CLIs and other external interfaces.
- Layer 3: Test Execution Management
This layer consists of a separate machine (test driver) that controls the test “fixture” (the set of machines that constitute a test environment). It is capable of taking software from either a local directory, an FTP site, an HTTP site, or from a subversion repository, unpacking it (if needed), installing it on the fixture, compiling it, and running a series of tests. These tests constitute executable programs that use the Layer 1 APIs, and Layer 2 Python modules that can “drive” the progress of the test suite and inject errors and events such as process, network, and machine failures.
- Layer 4: Automated Run and Longitudinal Reports
The Automated Run and Longitudinal Reporting layer consists of a single globally accessible server and a source repository (subversion etc). The server contains a website that allows users to see which source code branches are failing and which tests on what hardware and it keeps a history of this information.
How to use the TAE
- Create your model: Create a model using the OpenClovis IDE. Your model can use any redundancy model and contain any number of components and/or service groups. It can expect any number of blades. However, the TAE can run the same test on different device (test fixture) configurations. Therefore it is recommended that your organization choose a minimum configuration (preferably one that can be run by developers) and make sure your model supports this configuration but at the same time is capable of utilizing extra blades if they exist in the test fixture.
- Add your tests: There are various ways/options to implement SAFplus Platform test cases:
- Write test case in C as part of an SAFplus-based component. Such test cases should use the C TAE Interface
- Write test cases purely in Python running within the TAE Test Executor.
- The 3rd type of test case is a mixture of the two cases above. They implement tests using the C interface but also have more than just one line of code in the Python layer (robot side).
- Run the TAE with the above model:
For example: Put some of the test cases in a file (say tests.lst) and specify the test-case-filter-file name while running TAE using the -T switch to execute test cases specified in this file.
Command: <TAEBASE>/tae/tae -T tests.lst -f fetch:stop
It will execute all the steps from fetch to stop (fetch, populate, configure, build, package, deploy, start, test, stop).
- Get the result: The result can be achieved in TAE log files or TAE Report Server(the web GUI and database that allows you to look at all test results that have ever been generated.)
Conclusion
TAE offers a comprehensive and scalable solution for automating software testing, catering to both individual developers and enterprise-wide test environments. Don’t hesitate to reach out to us at support@openclovis.org when facing any hardly with testing.