Nowadays, business processes and their execution are corner stones in
modern IT landscapes, as multiple process languages and corresponding engines
for these languages have emerged. In practice, it is not feasible to select the best
fitting engine, as engine capabilities are mostly hidden in the engine implementation
and a comparison is hampered by the large differences and high adoption
costs of the engines. We aim to overcome these problems by a) introducing an
abstract layer to access the functionality of the engines uniformly, b) by revealing
the engine capabilities through automated and isolated tests for typical requirements,
and c) support the user in their selection of a process engine by determining
and explaining the fitness of the engines for a single process or a given set of
processes using policy matching against previously revealed engine capabilities.
Early results show the general feasibility of our approach for BPEL engines for a
Keywords: BPM, process engines, engine selection, execution requirements,
Despite the popularity of BPEL engines to orchestrate complex and executable processes, there are still only few approaches available to help find the most appropriate engine for individual requirements.
One of the more crucial factors for such a middleware product in industry are the performance characteristics of a BPEL engine.
There exist multiple studies in industry and academia testing the performance of BPEL engines, which differ in focus and method.
We aim to compare the methods used in these approaches and provide guidance for further research in this area.
Based on the related work in the field of performance testing, we created a process engine specific comparison framework, which we used to evaluate and classify nine different approaches that were found using the method of a systematical literature survey.
With the results of the status quo analysis in mind, we derived directions for further research in this area.
Today, a plethora of enterprise middleware solutions are available, leading to the problem of choosing the right tool for a specific use case.
Automated tests can support the selection of such software by determining decision relevant metrics, like e.g., throughput or the degree of standard conformance.
To avoid side effects between tests, test isolation, i.e., to provide fresh instances of the software for each test execution, is essential.
However, middleware suites are inherently complex, provide a large range of configuration options, have tedious or sometimes manual installation procedures, and long startup times.
These idiosyncrasies aggravate the creation of fresh instances of such middleware suites, leading to slower turnaround times and increasing the cost for ensuring test isolation.
We aim to overcome these issues with methods and tools from the area of virtualization and devops.
In this work, we focus on BPEL engines which are common middleware components in Web Service based SOAs.
We applied our proposed method to the BPEL Engine Test System (betsy), a conformance test suite and testing tool for BPEL engines.
Results reveal that our method a) enables automatic creation of fresh instances of software without manual installation steps, b) reduces the time to create these fresh instance dramatically, and c) introduces only a neglectable performance overhead, therefore, reducing the overall costs of testing complex software.
It is a long-standing debate, whether software that is developed as open source is generally of higher quality than proprietary software.
Although the open source community has grown immensely during the last decade, there is still no clear answer.
Service-oriented software and middleware tends to rely on highly complex and interrelated standards and frameworks.
Thus, it is questionable if small and loosely coupled teams, as typical in open source software development, can compete with major vendors.
Here, we focus on a central part of service-oriented software systems, i.e., process engines for service orchestration, and compare open source and proprietary solutions.
We use the Web Services Business Process Execution Language (BPEL) and compare standard conformance and its impact on language expressiveness in terms of workflow pattern support of eight engines.
The results show that, although the top open source engines are on par with their proprietary counterparts, in general proprietary engines perform better.
The errors in BPEL processes that are only detected at runtime are expensive to fix. Several modelers and process engines for BPEL exist, and the standard defines basic static analysis (SA) rules as a detection mechanism for invalid processes, but the actual conformance of BPEL modelers and engines regarding these rules is unknown. We propose to develop test cases to evaluate the conformance of BPEL modelers and engines regarding static analysis. The evaluation results enable decision makers to identify and use the most conformant engine and modeler that detect errors before runtime and therefore reduce costs
Service-oriented software consists of middleware, such as application servers and runtime engines, into which service applications are deployed. This middleware is often complex and difficult to install. The deployment of services requires the crafting of deployment descriptors and packaging
of applications. As a consequence, the installation of service-oriented software systems can be a daunting task. Installability, however, is an important influencer of the portability of software. Portability in turn is one of the main goals of service orchestration languages based on open standards. In this paper, we investigate the installability of service orchestrations based on the
Systems and software Quality Requirements and Evaluation
(SQuaRE) method, the new series of software qual-
ity standards currently under development by the ISO/IEC.
We develop a measurement framework based on SQuaRE and
tailored to evaluating the installability of service orchestrations and their runtimes. We validate the measurement framework theoretically and show its applicability in a case study.
More than five years have passed since the final release of the long-desired OASIS standard of a process language for Web Services orchestration, the Web Services Business Process Execution Language (BPEL). The aim of this standard was to establish a universally accepted Web Services orchestration language that forms a core part of service-oriented architectures and, because of standardization,
avoids vendor lock-in. By now, several fully conformant engines should have arrived in the market. It is our aim to shed light on this situation and to provide a comprehensive picture of the
current state of BPEL upport. We present an evaluation of the standard conformance of five open source BPEL engines.
To obtain these results we have developed betsy, a tool that allows for a fully-automatic standard conformance testing of BPEL engines. The results demonstrate that full standard
conformance in contemporary engines is still far from given.
More than five years have passed since the final release of the long-desired OASIS standard of a process language for web service orchestration, the Web Services Business Process Execution Language (BPEL). The aim of this standard is to establish a universally accepted orchestration language that forms a core part of current service-oriented architectures and, because of standardisation, avoids vendor lock-in. High expectations, in academia and practice alike, have been set on it. By now, several fully conformant and highly scalable engines should have arrived in the market. The perception of many however, is that standard conformance in current engines is far from given. It is our aim to shed light on this situation. In this study, we present the tool betsy, a BPEL Engine Test System that allows for a fully-automatic assessment of the standard conformance of a given BPEL engine. We use it to examine the five most important open source BPEL engines available today. Betsy comes with a large set of engineindependent conformance test cases for assessing BPEL standard conformance. This enables us to give a view of the state of the art in BPEL support.