Unfortunately your submission to RTAS 2009 has been unsuccessful. The reviews for your paper are attached. Thank-you for supporting the conference with your submission. Neil Audsley Program Chair, RTAS 2009. --------------------------------------------- Paper: 99 Title: Design-time Optimization Techniques in Middleware QoS Configuration for Distributed Real-time Systems -------------------- review 1 -------------------- PAPER: 99 TITLE: Design-time Optimization Techniques in Middleware QoS Configuration for Distributed Real-time Systems The paper "Design-time Optimization Techniques for Middleware QoS Configuration in Dsitributed Real-Time Systems" describes an algorithm that optimizes a configuration for a middleware (a variant of CORBA) generated by a CASE tool. The new derived configuration leads to a better mapping of the application to this middleware. In the given example, "better" translates to less latency. Optimization is done by grouping similar requirements of software parts to one, enabling the middleware to execute them in a single container. The presented approach does show an advantage over naively mapping the model as defined in the CASE tool to a middleware configuration. The case study gives an idea of the magnitude of improvement in this special case. There are however several issues concerning the presentation of the paper. - Many, if not all key concepts used in the paper are not sufficiently explained, nor backed by a suitable publication. Examples are: "Middleware Container", "QoS configuration", "System Deployment plan", "QoS policy set", "deployment artifact", "configuration artifact". - The discussion of "optimization of QoS policy configurations" is not backed by any metric to be optimized. (The example infers that latency could be a metric) Furthermore, with QoS considering a multitude of system/middleware parameters, I would expect this to be a multi-objective optimization issue leading to pareto optimal solutions, which has not been mentioned in the paper. - The idea to use the least amount of containers in order to minimize latency, as intra-container communication is expensive seems obvious, the key parts of the algorithm - determing "similar" QoS requirements and merging the policies are not discussed. That is probably also due to the fact that neither the content of a QoS requirement, nor the structure of a policy have been described). -------------------- review 2 -------------------- PAPER: 99 TITLE: Design-time Optimization Techniques in Middleware QoS Configuration for Distributed Real-time Systems although the title speaks of a generic "optimization technique", the paper in reality focuses on a clustering algorithm for SW components subject to QoS constraints and operating under the control of a real-time middleware layer. The clustering algorithm follows the description of a modeling language that has the purpose of modeling the execution semantics of the QoS-sensitive components. although it is possible to argue on the effectiveness of the modeling language (and its suitability for representing the communication and synchronization semantics, as well as the QoS requirements of large embedde applications) this is not the innovative content. the contribution seems to be the algorithm itself at page 4, which only groups components together in clusters based on their QoS policies. although the clustering rule (or similarity rule) is not fully explained, it does not seem to be particularly original or even complex. This is actually all there is with the paper. the case study could have been worth something more, but the authors only present a very simple example, in which the claimed advantage of 70% in latency is entirely due to the simplicity and inefficiency of the initial solution. The clustering of components subject to real-time constraints in complex distributed systems is indeed much more complex than this. Also, several (actually most of the existing) references to design optimization work in this area are missing. -------------------- review 3 -------------------- PAPER: 99 TITLE: Design-time Optimization Techniques in Middleware QoS Configuration for Distributed Real-time Systems This paper mainly addresses an optimized way of specifying QOS configurations for a Distributed real time system (DRE). The paper proposes a transformation algorithm which optimizes the QOS specifications and thereby reduces the latency or end to end delay between the different subsystems in a component based DRE. The paper analyzes the shortcomings of QOS policies in the context of component middleware platforms such as LwCCm(Lightweight CORBA component model). The authors propose the use of a modeling language called CQML(Component QOS modelling Language) forproviding an easier interface for the developers to specify QOS policies when compared to the corresponding LwCCM mechanisms. In a second step, a QOS policy optimization algorithm is used to generate a modified QOS policy which utilizes the collocation property of the middleware. Points in favor + Algorithm for optimization of the collocation property of middleware architectures. Points against + The algorithm is specific to LwCCM middleware platforms. The potential of the algorithm in other middleware architectures should be researched specifically. + The main contribution is merely in the transformation algorithm which improves the QOS policy. -------------------- review 4 -------------------- PAPER: 99 TITLE: Design-time Optimization Techniques in Middleware QoS Configuration for Distributed Real-time Systems I fail to understand what the technical contribution of this paper is. In my opinion, for anyone who is not intimately familiar with the authors’ previous work, this paper will be very difficult to understand. The paper refers to too many tools and uses too many abbreviations, without concisely explaining what the problem is and the solution being proposed. Sec 3 is the main technical section. It says that the paper presents a model transformation algorithm, where the inputs are: - QoS specification - system deployment plan (i.e., which components can be placed together) The output should be a new “QoS policy set”. The obvious questions that arise are: i) how does the QoS specification look like? (ii) how does the deployment plan look like? (iii) how does the QoS policy set look like? (iv) why is computing the policy set important and difficult? (v) how is the policy set computed (I understand that this is done using Algorithm 1, although I couldn’t follow the algorithm)? On the same page it has been mentioned that QoS specifications and deployment files are specified using GME – Generic Modeling Environment and the algorithms are defined using GReAT. However, in Section 3.1 it has been mentioned that QoS modeling has been done using CQML. Then comes the LwCCM configuration mechanism … Is the core technical contribution Algorithm 1? Then it should have been better explained and more space should have been devoted to it. The case study was equally difficult to understand. It appeared to be a relatively simple setup. But how was the “QoS policy set” obtained? How did it look like in this case? The only thing that was easy to understand was that there was a reduction in average latency and standard deviation in latency for the case “with optimization” and “with optimization and PAM”. In between it was mentioned that the proposed results can also be used by frameworks like “PAM”. How was the latency reduction obtained? Why was it difficult? I did not understand. In summary, this paper should be rewritten in a form that is understandable by someone who is familiar with real-time systems, software engineering and general systems development, but might not be familiar with the authors’ previous research.