Real-Time DDS Examination & Evaluation Project (RT-DEEP)

RT-DEEP is a evaluation project conducted by DOC group at Vanderbilt University. It  is aimed at examing different flavors of DDS implementations and using the empirical benchmarking results to evaluate their performance and compare them with other publish/subscribe mechanisms.

Table of Contents



1. DDS Overview

The Object Management Group's (OMG) Data Distribution Service(DDS) specification (OMG DDS formal/04-12-02) provides a new data distribution interoperability standard  for a plethora of diverse real-time computing environments ranging from small networked embedded systems to large-sclae information backbones. Based on a  publisher-subscriber architecture, the specification defines clear interfaces that enable applications running on heterogeneous platforms to communicate with each other simply by publishing and subscribing the information they need. By pervasive use of  Quality of Service (QoS) parameters, DDS also allows applications to configure the publish-subscribe system in a highly flexible manner,  thus providing performance predicability and resource control required by misson-critical environments.Compared to CORBA and DCOM which are client-server communication model, Publish-Subscribe is more efficient in both latency and bandwith for periodic data exchange because it is designed as a pure data-centric model. Figure 1 gives a overall architacture of DDS.


figure 1 architecture of DDS

2. RT-DEEP Overview

The goal of RT-DEEP is to get familar with different flavors of COTS DDS implementations, examine the capabilities they provide, and evaluate real-time DDS performance by comparing different DDS implementations as well as by comparing DDS with other pub/sub mechanisms.

Comparing different DDS Implementations - Right now there are several primary COTS DDS middleware vendors. Open-source DDS development projects include OpenDDS . OpenDDS provides an open-source C++ implementation of the DDS specification based ACE. After a year of beta testing, OpenDDS is now being used in production systems and has achieved a formal version 1 release.

As other DDS implementations become available users will need to choose the right one(s) to meet their application QoS requirements. Different implementations may address certain parts of the DDS specifications with different methods, thus affecting the QoS of those part in different ways. It is the goal of RT-DEEP to investigate these variants and discover the architectural differences in various products that may affect ability of products to scale to large number of processors & processes.

Comparing DDS with other pub/sub mechanisms - As a powerful communication pattern, Publish/Subscribe mechanism has been commonly offered by many distrbuted systems as core system services. Examples include Java Messaging ServiceWeb Service Pub-Sub Notification, and CORBA Notificaton Service. Sun's Java Messaging Service (JMS) is a message-oriented middleware standard that defines common ways for Java programs to create, send, receive and read enterprise messaging system's messages. One of the most common messaging models JMS provides is a Pub/Sub model in which Java objects/components that represent publishers publish enterprising messages to a topic, and all the subscribers (Java objects/components) to the topic will receive the messages sent to that topic. Publish-Subscribe Notification for Web Services (WS-pubsub) is part of Web Service Resource Framework which defines standard web services approach to publish/subscribe pattern. In the WS-PubSub model, a publisher is represented by a web service NotifiationProducer, and a subscriber by a web service NotificationConsumer. In a typical application scenario, a service requestor, as a subscriber, sends a subscription to NotificationProducer webservice, indicating the interest topic and the address of a NotifciationConsumer.When certain situation is triggered, the NotificationProducer will send out NotificaitionMessages associated with a topic to the interst parties. CORBA notification service is a OMG event service standards which allows multiple event suppliers to send events to multiple consumers. It decouples event suppliers from event consumers by using event channels. As we can see, all the three services mentioned above provides similar pub/sub mechanisms for real-time applications, and both JMS and WS-PubSub reply on topics to connect publishers to subscribers and vice versa, which is simialr to DDS. So it is also the goal of RT-DEEP to compare DDS with all the existing pub/sub mechanisms.


3. Examination & Evaluation objectives

The goal of RT-DEEP has driven the following objectives for the benchmarking
            We are running experiments that evaluate the black-box metric for different DDS implementations. Currently we are primarly focused on the latency performance. The following table gives the list of parameters we are looking at in our tests.

name
description
round-trip latency
time between the call to publish data operation and the return of the read/take operation
best-effort latency
latency with Best Effort publish
reliable latency
latency with Reliable publish


4. Benchmark Design


4.1 Evaluation targets

Benchmarked DDS implementations are:

DDS implementations
test platform
P1
GNU/Linux
P2
GNU/Linux
P3
GNU/linux

Benchmarekd other pub/sub mechanisms are:

Pub/Sub mechanisms
test platform
Java Messaging Service
GNU/Linux
Pub/Sub Notification for Web Service
GNU/Linux
CORBA Notification Service
GNU/Linux

4.2 Benchmark Scenario


The first cut of our benchmark simulates a handshake protocol similar to TCP/IP, only that we will use two-way instead of three-way. Figure 2 demonstrates the simple benchmark scenario of performance evaluation for different DDS implementations. Our test application are composed by one publish node and one subscriber node communicating with each other. The pub_node sends data (configurable sequence of bytes) to sub_node that subscribes to the data. Upon receiving the data, sub_node sends a Ack message back to pub_node indicating the receipt of data, thus finishing a round-trip.



The throughput test will deploy 1 publisher and multiple subscribers on different blade nodes. The publisher will keep sending samples and we measure the time that it takes for each subscriber to get the designated number of samples and average the throughput from our subscribers. In order to caluclate exact number of messages in spite of packet loss, we need to oversend samples at the publisher side.

4.3 Testbed overview

The testbed we are using for our benchmark test are located at ISISLab in which we have over 50 IBM blade nodes installed with FC4. For more information, please see http://wiki.isis.vanderbilt.edu/support/isislab.htm.


5. Emperical Results

see results

References

1. A Publish/Subscribe Mechanism for Web Services, http://webservices.sys-con.com/read/39676.htm
2. WS-PubSub, http://www-128.ibm.com/developerworks/library/ws-pubsub/WS-PubSub.pdf


Last modified $Date: 2007-07-13 15:26:08 -0500 (Fri, 13 Jul 2007) $ by $Author: mxiong $