% $Id: challenges.tex 4523 2006-11-29 04:06:54Z gokhale $

\section{Case Study to Motivate Network QoS Provisioning Requirements}
\label{sec:challenges}

This section describes (1) the structure/\-functionality of
a representative DRE system case study, (2) the key network traffic
and QoS requirements in the DRE system case study, and 
(3) the challenges in provisioning the network QoS requirements
for such a DRE system using QoS-enabled component middleware.

% This section describes the challenges faced in raising the level of
% abstraction of network QoS provisioning to the level of model-driven
% QoS-enabled component middleware, where the provisioning decisions can
% be made declaratively.  To best motivate these challenges we first
% present a DRE system case study outlining the network-centric QoS
% requirements of a representative DRE application.  We then illustrate
% the challenges that exist in the realm of model-driven QoS-enabled component
% middleware to enable DRE application developers to declaratively
% provision network QoS for the DRE applications.


\subsection{DRE System Case Study}
\label{sec:office-casestudy}

Figure~\ref{fig:office_arch} illustrates a representative 
DRE system case study, that involves a modern office/enterprise. The
enterprise consists of data centers hosting the computing facilities,
desktops throughout the enterprise accessing data and compute cycles
in the computing facilities,  
videoconferencing facilities throughout the enterprise, laboratories
hosting expensive and sensitive instruments, and safety/security
related hardware throughout the premises.  The security and
safety-related hardware artifacts of the network and facilities
operations include sensors, such as fire and smoke sensors, audio
sensors and video surveillance cameras that monitor sensitive parts of
the office premises for detecting intruders and unauthorized
accesses.  All these sensors are  connected to  the office network
where the data is streamed across potentially multiple but a small
number of subnets to the networking and facilities  
operation center. 

%\begin{floatingfigure}{0.6\columnwidth}
\begin{wrapfigure}{r}{0.6\columnwidth}
%\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.6\columnwidth,keepaspectratio]{figs/office-scenario}
  \vspace{-0.1in}
  \caption{An Office Environment and Network Setup}
  \label{fig:office_arch}
%\end{figure}
\end{wrapfigure}
%\end{floatingfigure}

Each sensor is associated with a software controller that is physically 
connected to the sensor via cables.  The sensory information sent by the 
hardware sensors is filtered and aggregated by their software 
controllers, and relayed to the monitoring systems at the network and 
facilities operations center via the office network.  These monitoring 
systems are assumed to consist of a wide variety of hardware, such as 
displays, monitors and analyzers.  These sensor hardware units and monitoring systems 
are all driven by software controllers, which are also implemented as 
CIAO~\cite{Schmidt:04w} components.  There is essentially one Layer-3
IP network for all the network traffic including email,
videoconferencing as well as the sensory traffic.  The underlying
network technology is itself a typical enterprise Layer-2 technology
such as Fast Ethernet that is IP-aware, for instance as supported by
CISCO 6500 switches.

\subsubsection{Network Traffic and QoS Requirements in Office Scenario}
\label{challenges-network-qos}

The traffic across the office network is diverse and has specific QoS
requirements.  For example, the traffic between sensors that monitor
factors like smoke or fire send periodic events of small or
moderate sized data and require low and predictable
latencies.   On the other hand, video surveillance cameras send a
constant stream of images back to the monitoring system.  
These data streams consume more bandwidth and while
latencies may not be critical, reliability of data is a critical
factor.  The normal business-related operations use both TCP and
UDP.  For example, email traffic using TCP requires best-effort QoS,
while applications, such as videoconferencing and other DRE
applications using UDP require appropriate latency, jitter and
bandwidth guarantees though they may tolerate some losses.  These
requirements demonstrate the need for differentiated QoS over
different transport protocol for different applications.  

The sensor traffic also has different importance levels.  For
example, a fire or smoke alarm will have higher priority over
sensors that track air conditioning temperature control within the
office environment.   Thus, certain traffic flows within the office
network need to take preference over others. 

The office scenario described here presents a wide variety of
characteristics that are representative of many DRE applications,
specifically from the network QoS requirements perspective.  The DRE
applications demand different degrees of reliability,
timeliness/\-latency, jitter and
importance/priority.\footnote{Although higher priority translates to
  lower latency, to guarantee a specific latency, the network
  utilization has to be kept  very low, i.e., priority and latency are
  non-overlapping dimensions of network QoS.}  
Even within a process, these characteristics typically vary for 
different flows, as illustrated here for the office case study.

\subsection{Challenges Provisioning Network QoS in QoS-enabled Component Middleware}
\label{sec:mw-challenges}

As shown in Figure~\ref{fig:office_arch}, a modern office environment
could have many different data flows between the same or different
sets of components, and those communications could have different
network QoS requirements. To support this scenario, the modern
office environment requires a QoS-enabled component middleware
that can support the following requirements:

{\bf 1: Specifying network QoS requirements for component interactions:}  
Component-based applications require specific levels of QoS to be honored 
for the inter-component communications e.g.,
bandwidth amount, desired latency, and priority.  For example,
different flows within the office environment have varied network QoS
requirements.  Bandwidth amount is often specified as  the tuple
$<$rate, burst size$>$.  In layer 3 networks, the network protocol
(e.g., UDP, TCP) is often a parameter used to distinguish a flow.

Such network QoS requirements must be collected for all the dataflows
involved in the system. Section~\ref{sec:NetQoSMDE} describes how NetQoPE
framework supports this requirement.

% Since the DRE systems of interest to us are moderate to large scale, 
% it is infeasible for application developers to programmatically 
% specify the QoS needs of components and their interactions on a 
% per flow basis.  The number of flows to be specified is in the order
% of thousands in one of the military applications that we have dealt
% with.  Moreover, the specifications are often made on a trial and
% error basis, i.e., the specification-implementation
% (simulation)-testing cycle has to be repeated several times.  A
% desired feature therefore is an easy to use, declarative and scalable
% means of specifying these QoS requirements.  
% 
% Design and development processes for large systems often use different
% views of the system to glean away information not essential for the
% problem being addressed.   Unfortunately, there are many cases when
% some system concerns crosscut multiple such views.  For example,
% consider the deployment view and a network QoS specification view of a
% complex DRE system as in our office environment where software
% controllers are deployed on hardware nodes that are connected to
% specific network ingress points.  Any changes made to the deployment
% of the components, will affect their network QoS depending on how
% the network resources will be consumed and provisioned. Metadata is used
% to reconcile adding QoS specific information to deployment specific
% information in QoS-enabled component middleware. Manually writing 
% the metadata that reconciles deployment and QoS
% specification is an extremely error prone and tedious  task.  The
% manual way of writing metadata does not scale as the complexity of the
% system grows.   Any automation to generate the metadata from the QoS
% specification is what is needed.

% Section~\ref{sec:NetQoSMDE}
% describes a model-driven engineering (MDE)~\cite{Schmidt:06a}
% approach we have developed that provides declarative
% capabilities to capture QoS requirements.  Our approach alleviates the
% aforementioned accidental complexities and automates the process of
% generating metadata for network QoS requirements.  It comprises
% extending our existing MDE capabilities that enable us to define the
% component dataflows by providing mechanisms to specify network QoS on
% a per flow basis.

{\bf 2: Allocating resources to meet inter-component flow
  QoS requirements:} 
Satisfying the network QoS requirements of flows between components
involves: (1) identifying which hardware nodes the application
components are to be deployed onto, (2) determining how much bandwidth
is available in every network connection between the hardware nodes
hosting the application components, (3) committing network resources
to be allocated for every network link between the two nodes, and 
(4) configuring the switches and routers with appropriate
policing and mark down behaviors for the flows.

Determining how much bandwidth is available on each link for a flow
between two components requires discovery of the path the flow will
take.  Section~\ref{sec:resource_reserve} 
describes how NetQoPE framework supports this requirement.

% Network management and provisioning elements, such as our Bandwidth
% Broker, also require the source and destination host information to
% ensure bandwidth availability on the appropriate links a flow will
% traverse.   In QoS-enabled component middleware, such deployment
% decisions  are usually made using intelligent component placement
% algorithms~\cite{Rajkumar:05,Schmidt:06m}, based on details including
% component CPU resource profiles.  Hence, network QoS allocation
% decisions and provisioning mechanisms must work with such component
% placement algorithms.  A well-integrated network allocation and
% placement algorithm provides an opportunity to retry a component
% placement decision if the network QoS cannot be met with a particular
% placement decision.
% 
% Section~\ref{sec:resource_reserve} describes enhancements we made to 
% the planning portions of our resource allocation and control engine
% (RACE)~\cite{Schmidt:05u}, which acts as a mediator between application
% components and low-level network elements, specifically switches and
% routers, to convey the application's network QoS requirements to these
% elements through the Bandwidth Broker, request resource reservations 
% across the hardware nodes decided by the component placement
% algorithms, and configure these networking elements.  The results of
% these interactions are captured within the application's deployment
% metadata, which describes how different application components will be
% deployed and configured within the computing infrastructure.

{\bf 3: Configuring applications and their flows to meet
  network-level QoS requirements:} 
Once the desired network level resources for various flows are
reserved, and the edge routers are configured for appropriate
policing and mark down behaviors for the flows, the application
communications need to be made with appropriate QoS settings
so that the routers can provide the right packet forwarding behavior
for flows traversing through those devices.

QoS settings for achieving network QoS are usually added to IP
packets, when an application communicates with another application.
In a QoS-enabled component middleware, such low-level details for
working with IP packet headers are hidden by the ORB middleware.
Hence middleware must provide mechanisms for interacting with
applications, to configure them with appropriate QoS settings, so that
such QoS settings can be carried onto to the underlying network
subsystem, which can then provision network QoS for those
applications.  Section~\ref{sec:policy} describes how NetQoPE
framework supports this requirement.

% We need a network policy  or automation framework that will accomplish
% this.  This automation is consistent with our philosophy for
% component-based systems that enable the application developers to only
% focus on the  business logic and not be bothered about complex details
% on how the middleware and underlying network subsystems are configured
% to ensure the QoS requirements.  In general, the applications
% developers should be alleviated from the responsibility of writing
% code to work with end system-specific interface and QoS requirements,
% thereby increasing opportunities for code reuse.  
% 
% Referring to the office case study in Figure~\ref{fig:office_arch},
% regardless of which host originates a flow/remote call and which
% routers (and operating system) the flow traverses/uses and the QoS
% used for the flow, the code should ideally be the same for better
% reuse.  In other words, our goal is ``write once, but deploy multiple
% times'' in different targets.  The office scenario illustrates the
% need for the underlying QoS-enabled middleware capabilities to
% facilitate different QoS guarantees to the same software written and
% deployed under multiple deployment targets and network environments. 
% 
% Section~\ref{sec:policy} describes a network QoS policy framework we
% have developed within our CIAO component middleware framework to
% transparently add platform-specific QoS settings when inter-component
% remote calls are made thereby honoring the network-level QoS 
% management decisions made at deployment time. 
