A generic interoperability testing framework and a systematic development process for automated interoperability testing

Interoperability is a prerequisite to allow users to access systems implemented by different vendors seamlessly. A good baseline to achieve interoperability is the implementation of a common set of standards. However, this is often not sufficient as different implementations of a standard are not necessarily interoperable. Therefore, different implementations of systems need to be assessed for interoperability by applying interoperability testing. In this article, we present a generic framework that enables automated interoperability testing with message checks, which assess the compliance of messages exchanged between systems. The goal of this framework is the provision of a basic functionality of interoperability test entities, the definition of a generic interoperability test environment, and guidelines for the specification of automated interoperability tests. The framework also considers aspects related to interoperability testing including verdicts, automation, and limitations of the system under test. Through the application of the framework, interoperability of systems can be assessed, systems can be validated, and standards can be improved. In addition, we present a systematic development process for automated interoperability tests to formalize the development and specification of an interoperability test system. We also consider aspects and critical issues, which are important for the development of a complete interoperability test system. The framework and the process are language and system technology independent. We present their application in a case study that includes interoperability tests for the Internet Protocol Multimedia Subsystem (IMS) using the Testing and Test Control Notation Version 3 (TTCN-3).