Real-Time Event Channel Configuration

Configuring the Event Channel at Compilation Time

TAO's Real-Time Event Channel can be configured at compilation time to use a different payload than its default. To do so there are several options:

To eliminate the CORBA::Any field in the event payload you should define the TAO_LACKS_EVENT_CHANNEL_ANY macro when invoking the IDL compiler. You can do as follows:
```
$ cd $TAO_ROOT/orbsvcs/orbsvcs
$ make TAO_IDLFLAGS=-DTAO_LACKS_EVENT_CHANNEL_ANY
```

Similarly you can eliminate the octet sequence field in the event payload using:

$ cd $TAO_ROOT/orbsvcs/orbsvcs
$ make TAO_IDLFLAGS=-DTAO_LACKS_EVENT_CHANNEL_OCTET_SEQUENCE

Finally you can provide your own event payload, to do this you should Replace the contents of the RtecDefaultEventData.idl and define some IDL structure named RtecEventData. For example, you could define you own event type as follows:
```
// Replace RtecDefaultEventData.idl with this:

struct RtecEventData
{
  sequence<long> my_event_payload;
};
```

Run-time Configuration

The new implementation of the real-time event channel uses a factory to build all the objects and strategies it requires. The factory can be dynamically loaded using ACE Service Configurator, this is extremely convenient because the factory can also parse options in the Service Configurator script file.

There are currently several implementations of the factory.

The default factory supports all of the options below and most of the potential values, except as specifically indicated.
The sched factory defines new option values as indicated below that allow the RTES to integrate with the Real-Time Scheduling Service. Use of the sched factory requires that you link in the TAO_RTSchedEvent library.
The tpc (thread per consumer) factory defines new options values as indicated below. This strategy is similar to the MT dispatching strategy in that it separates the thread that receives the push() from a supplier from the thread that does the eventual push() to the consumer(s), but it differs from MT in that MT has a pool of threads pulling from a single queue and ultimately potentially servicing every consumer, whereas TPC has a queue for each consumer and, thus, a single thread dedicated to pushing to that consumer. This insures that a badly-behaved consumer CANNOT have an adverse effect on the channel or supplier.
To use this new strategy you will have to replace the call in your code for TAO_EC_Default_Factory::init_svcs() with TAO_EC_TPC_Factory::init_svcs().
The basic and null factories are hard-coded factories that do not support any configuration options. They both define simple configurations that may enable some applications to use the RTES in a smaller footprint. See the files $TAO_ROOT/orbsvcs/orbsvcs/Event/EC_Null_Factory.h and $TAO_ROOT/orbsvcs/orbsvcs/Event/EC_Basic_Factory.h for details.

Special Topic: Queue Configuration

In certain configurations such as mt and tpc, the RTES implementation uses something called TAO_EC_Queue, which is an in-memory queue that separates threads that receive push() invocations from suppliers from the threads that dispatch push() invocations to consumers. A user of the RTES can configure some behaviors of this queue as well as behaviors when the queue reaches certain trigger points. The configuration of these behaviors is a combination of compile-time and run-time specification.

The queue itself has two trigger points for the number of invocations in the queue:

low-water mark (LWM), a lower threshold below which the queue allows more invocations to be placed into the queue;
high-water mark (HWM), an upper threshold above which the queue prevents more invocations from being added to the queue

The values for these can be set at compile time via TAO_EC_QUEUE_LWM and TAO_EC_QUEUE_HWM for the low-water mark and high-water mark, respectively.

In addition, an application can specify what the RTES should do if a queue reaches the HWM, i.e., fills up. This behavior is encapsulated in a derivation of an ACE_Service_Object, called TAO_EC_Queue_Full_Service_Object. The RTES looks for a service object of a particular name (the default for this name is in macro TAO_EC_DEFAULT_QUEUE_FULL_SERVICE_OBJECT_NAME, which is "EC_QueueFullSimpleActions") and invokes queue_full_action() on the instance. See class TAO_EC_Simple_Queue_Full_Action in file $TAO_ROOT/orbsvcs/orbsvcs/Event/EC_Dispatching_Task.h for an example. This particular example can be configured to either wait or discard new invocations on reaching the HWM.

The configuration file

The real-time event channel uses the same service configurator file that the ORB uses, the default name for this file is svc.conf, but the ORB option -ORBSvcConf can be used to override this. The format of the file is described in detail in the service configurator documentation, but the relevant section for the event channel looks like this:

# Comments go here...
# More comments if you want to...
static EC_Factory "-ECFiltering basic ....."
# ...and optionally...
static EC_QueueFullSimpleActions "[wait | discard]"
# ...or you can define your own TAO_EC_Queue_Full_Service_Object,
#    give it whatever name you want and use -ECQueueFullServiceObject
#    (see below) to specify that object's service configurator tag.

The table below describes all of the options (and values) supported by these factories.

All the event service factory options start with -EC

The options

Option Description

-ECDispatching dispatching_strategy Select the dispatching strategy used by the real-time event service. The reactive strategy uses the same thread that received the event from the supplier to push the event to all the consumers.
The priority strategy (sched factory only) uses a prioritized pool of threads and queries the scheduling service to select the thread that dispatches each event.
The mt strategy also uses a pool of threads, but the thread to dispatch is randomly selected.
Does not apply to the tpc factory.

-ECDispatchingThreads number_of_threads Select the number of threads used by the mt dispatching strategy.
Does not apply to the tpc factory.

-ECDispatchingThreadFlags thread_flags:thread_priority Provide flags and priority used for creation of dispatching threads.
thread_flags is a set of THR_ flags separated by the vertical bar ('|'), e.g., THR_BOUND|THR_NEW_LWP|THR_SCOPE_SYSTEM.
thread_priority is a numeric value indicating the desired priority at which the thread should be created. No range checking is performed on the value prior to passing to the thread creation function. Also note that priorities are platform-specific, so a valid value for a given set of flags on one platform may not work with the same flags on another platform.

-ECQueueFullServiceObject service_object_name Used to specify the name/tag in the service configurator through which the RTEC should look for a "queue full service object".
Will only have an effect on dispatch strategies that use TAO_EC_Queue.

-ECFiltering consumer_filtering_strategy Select the filtering strategy used by the consumers. The null filtering strategy will build trivial filters for all consumers. The basic filtering strategy supports disjunction, conjunction, logical and, negation, bitmask, and timeout filters based on the subscriptions passed by the consumer during the connect call. It only supports limited nesting of filter groups. The prefix filtering strategy supports all filter types that the basic strategy does and allows unlimited nesting of filter groups. When using the prefix strategy, the source field of the group filter must specify the number of children in the group. The priority filtering strategy (sched factory only) supports the same features as the basic filtering strategy, but it also collaborates with the scheduling service to build the dependency graph.

-ECSupplierFiltering supplier_filtering_strategy The event channel can also perform some filtering close to the suppliers by minimizing the number of consumers that are tested for given event. If the strategy is null then a global collection of consumers is maintained and each event is filtered by each consumer. If the strategy is per-supplier then the EC uses the supplier publications and the consumer subscriptions to determine which consumers could potentially be interested in events for each supplier. This set of consumers is usually smaller than the complete set and it is thus faster to traverse it, but keeping more collections of consumers increases the connection and disconnection time as well as the memory requirements.

-ECTimeout timeout_strategy A consumer can request that the event channel generate periodic or oneshot timeout events. This option controls the strategy to generate the timeouts, using reactive the same reactor used for the ORB is used by the event service. The priority strategy (sched factory only) uses a prioritized group of threads, timeouts with higher rate are generated by threads of higher priority.
NOTE: The priority strategy is not implemented

-ECObserver observer The event channel uses the Observer pattern to report changes in the subscriptions and publications of its suppliers and consumers; this is especially useful in the implementation of event channel gateways. The user can disable this feature by selecting the null strategy; whereas the basic strategy provides a simple, yet efficient implementation. reactive provides a strategy in which an observer that is unreachable is removed from the observer list.
NOTE: The default is to have the feature disabled.

-ECScheduling scheduling_strategy The event channel can collaborate with the scheduling service to build the dependency list between the consumers and the suppliers. If the null scheduling strategy is selected this feature is disabled, the priority strategy (sched factory only) enables this feature.
NOTE: The default is to have the feature disabled.

-ECProxyConsumerLock lock_type Select the lock type (null, thread or recursive) to synchronize access to the ProxyPushConsumer state.

-ECProxySupplierLock lock_type Select the lock type (null, thread or recursive) to synchronize access to the ProxyPushSupplier state.

-ECUseORBId orbid Set the name of the ORB used by the event service, only useful in applications that create multiple ORBs and activate the event service in one of them.

-ECConsumerControl policy Select the consumer control policy (null or reactive) to detect and discard broken consumers.

-ECSupplierControl policy Select the supplier control policy (null or reactive) to detect and discard broken suppliers.

-ECConsumerControlPeriod period Set the period (in microseconds) used by the reactive consumer control policy (set ECConsumerControl to reactive) to poll the state of the consumers. When set to 0 there is no polling but the reactive strategy is in place.

-ECSupplierControlPeriod period Set the period (in microseconds) used by the reactive supplier control policy (set ECSupplierControl to reactive) to poll the state of the suppliers. When set to 0 there is no polling but the reactive strategy is in place.

-ECConsumerControlTimeout timeout Set the timeout period (in microseconds) used by the reactive consumer control policy (set ECConsumerControl to reactive) to detect a timeout when polling the state of the consumers.

-ECSupplierControlTimeout timeout Set the timeout period (in microseconds) used by the reactive supplier control policy (set ECSupplierControl to reactive) to detect a timeout when polling the state of the suppliers.

-ECProxyPushConsumerCollection flag[:flags]
Configure the data structure and strategies used to implement collections of ProxyPushConsumers. The argument is a colon separated list of flags, with the following semantics:

Flag Description

MT Use regular mutexes and/or condition variables for serialization.

ST Use null mutexes and/or condition variables for serialization.

LIST Implement the collection using an ordered list, fast for iteration (i.e. during event dispatching), but slow for insertion and removal (i.e. when clients connect and disconnect from the EC).

RB_TREE Implement the collection using a Red-Black tree, slow for iteration (i.e. during event dispatching), but fast for insertion and removal (i.e. when clients connect and disconnect from the EC).

IMMEDIATE Threads block until they can execute a change on the data structure, the system must use other approaches to guarantee that the iterators are not invalidated during event dispatching. For example, use a separate dispatching thread. Using this option with the reactive values for any of the -ECSupplierControl, -ECConsumerControl, or -ECDispatching options may cause deadlocks.

COPY_ON_READ Before initiating an iteration to dispatch events (or similar tasks) a copy of the complete collection is performed. This solves most of the synchronization problems, but introduces a significant source of overhead and priority inversions on the critical path.

COPY_ON_WRITE Similar to the previous one, but the copy is only performed when needed.

DELAYED Threads that need to change the collection can detect if that change will invalidate iterators used by other threads. If so, the thread posts the change on a queue that is executed once the collection is no longer in use.

-ECProxyPushSupplierCollection flag[:flags]
Configure the data structure and strategies used to implement collections of ProxyPushSupplier objects. Use the same arguments as with the ECProxyPushConsumerCollection option.

-ECConsumerValidateConnection validate_connection: 0 | 1 When this option is set the connection from the Event Channel to the consumer will be created during the connect of the consumer, just before the consumer is added to the consumer list. By default the Event Channel will create the connection to the consumer at the moment the first event is pushed to the consumer. When the network is heavily loaded during this connect and a new event is pushed before the connection is established, a new create connection request is done. When having very high connection establishment time (for example running on a WAN) and an interval that is much smaller than the connection establishment time it is wise to set this option. Else you get dozens of new connection request that can result in a stack overflow. Another option is to use BiDIR GIOP so that there is only one connection between Rtec and consumer but for this the Rtec and the consumer must be changed to set the BiDIR Policy, this is not implemented at this moment.

The constructor

The TAO_EC_Event_Channel class implements the RtecEventChannelAdmin::EventChannel interface; this class takes one mandatory and two optional parameters in its constructor:

  TAO_EC_Event_Channel (const TAO_EC_Event_Channel_Attributes& attributes,
                        TAO_EC_Factory* factory = 0,
                        int own_factory = 0);

The factory is an optional parameter to override the default strategy factory used by the event channel, the event channel will destroy the factory if the own_factory argument is true.

The attributes parameter can be used to fine tune some of the algorithms and strategies used by the event channel, the default values are probably OK for most applications. Notice that the attributes include the POA used to activate the ConsumerAdmin, SupplierAdmin, ProxyPushConsumer and ProxyPushSupplier objects; these POAs must have the IMPLICIT_ACTIVATION and the SYSTEM_ID policies (as the RootPOA does).

Attribute Description

consumer_reconnect If the attribute is not zero then the same consumer can call connect_push_consumer on its ProxyPushSupplier multiple times to change its subscriptions; this is usually more efficient that disconnecting and connecting again.

supplier_reconnect If the attribute is not zero then the same supplier can call connect_push_supplier on its ProxyPushConsumer multiple times to change its publications; this is usually more efficient that disconnecting and connecting again.

busy_hwm When the delayed flag is set on proxy collections, this flag controls the maximum number of threads that can simultaneously iterate over the collection before blocking. It can be used to avoid starvation in delayed updates on the collection.

max_write_delay When the delayed flag is set on proxy collections, this flag controls the maximum number of threads that will initiate dispatching after a change has been posted. Any thread after that is blocked until the operations are performed. It can be used to completely stop starvation of delayed updates on the collection.

scheduler Most configurations of the real-time event channel do not require an scheduler. If any of the strategies that require an scheduling service is selected then this attribute should be set appropriately.

supplier_poa The POA used by the event channel to activate SupplierAdmin and SupplierProxy objects.

consumer_poa The POA used by the event channel to activate ConsumerAdmin and ConsumerProxy objects.

disconnect_callbacks If not zero, then the event channel sends disconnect callbacks when a disconnect method is called on a Proxy. For example, if a consumer calls disconnect_push_supplier() on its proxy the EC invokes disconnect_push_consumer() on the consumer. An analogous call is also made for suppliers when this attribute is set. By default, these callbacks are not made in this situation.

Attribute	Description
`consumer_reconnect`	If the attribute is not zero then the same consumer can call `connect_push_consumer` on its ProxyPushSupplier multiple times to change its subscriptions; this is usually more efficient that disconnecting and connecting again.
`supplier_reconnect`	If the attribute is not zero then the same supplier can call `connect_push_supplier` on its ProxyPushConsumer multiple times to change its publications; this is usually more efficient that disconnecting and connecting again.
`busy_hwm`	When the delayed flag is set on proxy collections, this flag controls the maximum number of threads that can simultaneously iterate over the collection before blocking. It can be used to avoid starvation in delayed updates on the collection.
`max_write_delay`	When the delayed flag is set on proxy collections, this flag controls the maximum number of threads that will initiate dispatching after a change has been posted. Any thread after that is blocked until the operations are performed. It can be used to completely stop starvation of delayed updates on the collection.
`scheduler`	Most configurations of the real-time event channel do not require an scheduler. If any of the strategies that require an scheduling service is selected then this attribute should be set appropriately.
`supplier_poa`	The POA used by the event channel to activate SupplierAdmin and SupplierProxy objects.
`consumer_poa`	The POA used by the event channel to activate ConsumerAdmin and ConsumerProxy objects.
`disconnect_callbacks`	If not zero, then the event channel sends disconnect callbacks when a disconnect method is called on a Proxy. For example, if a consumer calls disconnect_push_supplier() on its proxy the EC invokes disconnect_push_consumer() on the consumer. An analogous call is also made for suppliers when this attribute is set. By default, these callbacks are not made in this situation.

Configuring the IIOP Gateway

The TAO_EC_Gateway_IIOP_Factory is the factory for the TAO_EC_Gateway_IIOP class. This factory can be used to configure the settings of the IIOP Gateway using the svc.conf file. The contents looks like this:

# Comments go here...
# More comments if you want to...
static EC_Gateway_IIOP_Factory "-ECGIIOPConsumerECControl null ....."

All the IIOP Gateway factory options start with -ECGIIOP

Option Description

-ECGIIOPConsumerECControl policy Select the consumer ec control policy (null or reactive or reconnect) to detect and handle broken consumer ec's.

-ECGIIOPConsumerECControlPeriod period Set the period (in microseconds) used by the reactive and reconnect consumer ec control policy (set ECGIIOPConsumerECControl to reactive or reconnect) to poll the state of the consumer ec. When set to 0 there is no polling but the reactive strategy is in place.

-ECGIIOPConsumerECControlTimeout timeout Set the timeout period (in microseconds) used by the reactive and reconnect consumer ec control policy (set ECGIIOPConsumerECControl to reactive or reconnect) to detect a timeout when polling the state of the consumer ec.

-ECGIIOPUseORBId orbid Set the name of the ORB used by the event gateway, only useful in applications that create multiple ORBs and activate the gateway in one of them.

-ECGIIOPUseTTL use_ttl: 0 | 1 Set whether the IIOP Gateway uses the TTL or not. This is 1 by default. When set to 0 make sure that no recursive structure is build between ec's, else an event can keep looping for ever. Useful when there are no recursive structures and we want the gateway to transfer all events independent of their TTL because suppliers can be unaware about the maximum number of hops. In case this option is 0, the TTL in the event isn't decremented by the gateway also.

-ECGIIOPUseConsumerProxyMap use_consumer_proxy_map: 0 | 1 Set whether the IIOP Gateway uses a map with a proxy push consumer for each unique source id or not. This is 1 by default meaning that a map is used. When set to 0 the gateway will just use one proxy push consumer to push events to the consumer ec. This options is useful when a lot of different source id's are used but there is no need for a different proxy push consumer for each source id. Setting this option to 0 then improves the performance.

Option	Description
`-ECGIIOPConsumerECControl` policy	Select the consumer ec control policy (null or reactive or reconnect) to detect and handle broken consumer ec's.
`-ECGIIOPConsumerECControlPeriod` period	Set the period (in microseconds) used by the reactive and reconnect consumer ec control policy (set ECGIIOPConsumerECControl to reactive or reconnect) to poll the state of the consumer ec. When set to 0 there is no polling but the reactive strategy is in place.
`-ECGIIOPConsumerECControlTimeout` timeout	Set the timeout period (in microseconds) used by the reactive and reconnect consumer ec control policy (set ECGIIOPConsumerECControl to reactive or reconnect) to detect a timeout when polling the state of the consumer ec.
`-ECGIIOPUseORBId` orbid	Set the name of the ORB used by the event gateway, only useful in applications that create multiple ORBs and activate the gateway in one of them.
`-ECGIIOPUseTTL` use_ttl: 0 \| 1	Set whether the IIOP Gateway uses the TTL or not. This is 1 by default. When set to 0 make sure that no recursive structure is build between ec's, else an event can keep looping for ever. Useful when there are no recursive structures and we want the gateway to transfer all events independent of their TTL because suppliers can be unaware about the maximum number of hops. In case this option is 0, the TTL in the event isn't decremented by the gateway also.
`-ECGIIOPUseConsumerProxyMap` use_consumer_proxy_map: 0 \| 1	Set whether the IIOP Gateway uses a map with a proxy push consumer for each unique source id or not. This is 1 by default meaning that a map is used. When set to 0 the gateway will just use one proxy push consumer to push events to the consumer ec. This options is useful when a lot of different source id's are used but there is no need for a different proxy push consumer for each source id. Setting this option to 0 then improves the performance.

The commandline arguments

The Event_Service executable has several commandline arguments you can use. The following arguments are available.

Option Description

-n service_name The name with which the Event Service registers itself in the Naming Service.

-o ior_file_name The file to which the IOR of the Event Service will be written.

-p pid_file_name The process id of the Event Service will be written to this file

-q ec_object_id The object ID for the Event Channel that will be used when a Persistent IOR is generated

-b Use BiDIR GIOP

-x Don't register the Event Service to the Naming Service

-s global|local|none Which scheduler should be used. The default is none. With local an in process scheduler is created. With global a scheduler is retrieved from the Naming Service

Option	Description
`-n` service_name	The name with which the Event Service registers itself in the Naming Service.
`-o` ior_file_name	The file to which the IOR of the Event Service will be written.
`-p` pid_file_name	The process id of the Event Service will be written to this file
`-q` ec_object_id	The object ID for the Event Channel that will be used when a Persistent IOR is generated
`-b`	Use BiDIR GIOP
`-x`	Don't register the Event Service to the Naming Service
`-s` global\|local\|none	Which scheduler should be used. The default is `none`. With `local` an in process scheduler is created. With `global` a scheduler is retrieved from the Naming Service

Back to the TAO components documentation. Last modified: Tue Oct 18 13:13:54 CDT 2005