% $Id: Resolution.tex,v 1.53 2005/10/06 17:46:14 arvindk Exp $
\section {Resolving Middleware Generality via Context-spe\-cific Specializations}
\label{mw-spl-solution}
%%%%
% Arvind: Perhaps the most unexplained paragraph in the whole paper!
% Most of the reviewers did not understand how we are applying all these
% techniques. We mention here.
%%%
In this section we examine techniques that
focus on an important dimension of optimizing DRE systems: {\em the
use of context-specific specializations to enhance the QoS of
PLA-based DRE systems by alleviating excessive generality in
middleware implementations}.
%%%
% Arvind: I am making clear that this space encompasses all of the
% specializations mentioned below. Not what we do!
%%%
The different dimensions of Context-specific specialization includes
techniques such as (1) {\em partial evaluation}, which creates a
specialized version of a general program that is more optimized for
time and/or space than the
original~\cite{Jones:1993:PartialEvaluation}, (2) {\em
code-refactoring and weaving}~\cite{zhang03refactoring,Fowler:99},
which uses aspect-oriented programming mechanisms to factor out and
weave cross-cutting concerns, and (3) {\em language mechanisms}, such
as program optimization techniques~\cite{Itkin:1983:OnPartial:Russian}
and program generation~\cite{Thiemann:1997:ProgramGeneration}.

% Arvind, hum, then can you please explicitly mention each of these
% things in the writeup below?  Right now, I don't see any explicit
% mention of memo-ization, constant propagation/folding, etc.
%%%
% Arvind: I will explain them individually in each of the subsections
% below.
%%%
The middleware specializations described in this paper consists of
constant propagation/folding, memo-ization, code-refactoring, and
aspect weaving.  Following previous work in program
specialization~\cite{marlet99efficient}, these specializations are
driven by \emph{invariant properties}, which are particular attribute
settings, parameter values, and internal/\-external contexts that are
fixed for a length of time during a system's execution.  In simple
cases, an invariant property manifests itself in the form of the
method call to a method {\tt m()} where one or more of the parameters
of the method always is bound to the same value.  Our program
specializations strategies push invariant data through the middleware
code and simplify along the way, {\em e.g.}, by creating a specialized
version of {\tt m()} where the parameters with fixed values are
removed and the body of {\tt m()} is simplified according to the
information provided by the fixed parameter values.  It is crucial
that (1) the specialization techniques not compromise the standard
interfaces provided by the middleware implementation and (2) the
effort of specifying and applying these specializations to production
middleware and PLAs for DRE systems be minimized.

To evaluate our middleware specializations in a
realistic context, we compare it to TAO~\cite{Schmidt:97q}, which is a
widely-used, open-source implementation of Real-time CORBA that
contains many general-purpose ORB
optimizations~\cite{Schmidt:99h,Schmidt:02r}.  As shown below, we use
TAO as a baseline to quantify the benefits of five specializations
from~\cite{daugherty:04} that help resolve the challenges for PLAs
described in Section~\ref{mw-spl-challenges}.  The specializations and
the quantitative results presented below help to assess the extent
to which specializations are beneficial in the context of production
middleware implementations to discover what payoffs (in terms of
speed-up, footprint reduction, and jitter reduction) can be expected
for the type of comprehensive specialization framework we
envision~\cite{daugherty:04}.  The constant propagation and code
refactoring techniques described in the paper
were automated using the conditional compilation technique in TAO while
the memo-ization, layer folding and aspect weaving were automated via
the automation framework described in Section~\ref{FOCUS-approach}.

%This section presents the results of a feasibility study that (1)
%locates key specialization points within the context of a
%widely-deployed Real-time CORBA implementation~\cite{RTCorba:02}, (2)
%manually identifies instances of the types of specialization
%opportunities and context information that we aim to automatically
%accumulate as our work progresses, and (3) manually transforms the
%code to assess the extent to which specializations are beneficial in
%the context of production middleware implementations to discover what
%payoffs (in terms of speed-up, footprint reduction, and jitter
%reduction) can be expected for the type of comprehensive
%specialization framework we envision~\cite{daugherty:04}.

\sqztiny
\subsection {Analyzing General-purpose Middleware}
To specialize general-purpose Real-time CORBA middleware for PLA-based
%%%%
% Reviewer comment:
% Page 7, Section 3.1: the definition of code path is a little bit weak
% (sequence of steps), although the example is clear. Code path is the
% crucial part for evaluating QoS. I would suggest that you provide a more
% precise definition.
%%%%
DRE systems, we first analyzed the end-to-end critical code path
(sequence of steps) of the following synchronous two-way CORBA request
in TAO:

{\tt result = object$\rightarrow$operation (arg1, arg2)} \\
%%%
% Arvind: Response to the comment.
%%%%
\noindent We define the critical code path to be the sequence of
activities that are always necessary in TAO to process any request
or response.  This code path is the same for processing the
\url{get_data()} two-way operation and \url{data_avail} and {\tt
timeout} events in the {\em BasicSP} scenario described in
Section~\ref{DRE-case-study}.  This code path provides a baseline for
comparing our context-specific specialization approach to quantify the
number of steps specialized along the critical request/\-response
processing path shown by the numbered bullets in
Figure~\ref{client-server-trace}.
\begin{figure}[htpb]
\centerline{\includegraphics[width=3in]{figures/client-server-trace.eps}}
  \bfcaption{End-to-End Request Processing Path}
  \label{client-server-trace}
\end{figure}
Below, we describe the steps involved when a client invokes a
synchronous two-way operation on an object residing at a server.
After the connection from client to server has been established, the
following activities are performed by the TAO client ORB when a client
application thread invokes an operation on an object reference that
designates a particular target object on a TAO server
ORB:\footnote{This discussion has been generalized using the Reactor,
Acceptor-Connector, and Leader/Followers patterns~\cite{Schmidt:00a},
which are used in many popular CORBA ORBs, such as e*ORB, ORBacus, and
TAO. }
\begin{figure}[htpb]
\centerline {\includegraphics[width=6cm]{figures/Specialization-Summary.eps}}
  \bfcaption{Specialization Points for Real-time CORBA Middleware}
  \label{fig:spl-points}
\end{figure}
\sqztiny
\begin{enumerate}\crunchlist

\item [1.] Determine if the target object is remote or collocated. If
collocated, then goto Step 7 of server-side steps; else, goto Step 2.

\item [2.] Allocate a buffer from a memory pool to create a
request and marshal the parameters in the operation invocation.

\item [3.] Wait for a reply from the server on the same connection on
which the data was sent.
\end{enumerate}

\sqztiny
\noindent The server ORB activities for processing a request are
described below:

\sqztiny
\begin{enumerate}\crunchlist

\item [4.] Server waits in the reactor's event loop for new client
connection and data events.

\item [5.] When new data arrives, the request header is read to
determine the request size.

\item [6.] Allocate a buffer from a memory pool to hold the request
and read the request data into the buffer. Use appropriate
message parser to demultiplex the request.

\item [7.] Find the target portable object adapter
(POA)~\cite{Schmidt:98o}, servant, and skeleton -- then dispatch the
designated upcall to the servant after demarshaling the request
parameters if the request is not collocated.

\item [8.] In the remote case, send the reply (if any) to the client on
connection $C$.

\item [9.] Go to step 4.
\end{enumerate}

\noindent Having described the activities at the client and server,
the remainder of this section describes our approach for specializing
TAO using invariance assumptions and quantitatively compares the
end-to-end predictability, latency, and throughput improvements
stemming from our approach.  To ensure portability and
interoperability, these specializations comply with the Real-time
CORBA specification and do not modify any standard interfaces.
Figure~\ref{fig:spl-points} illustrates where we applied these
specializations to the Real-time CORBA architecture.  All
specializations are applied along the critical request/response
processing path and affect end-to-end QoS.  Our approach
specializes most of the steps (1 -- 2 on the client and 4 -- 8 on the
server).

\sqztiny
\subsection {Designing and Evaluating Middleware Specializations}
\label{sec:spec_eval}

We now describe how we specialized TAO to resolve the challenges for
PLAs described in Section~\ref{mw-spl-challenges}.  For each
challenge, we quantify the gains observed by applying our techniques.
All experiments were performed on an Intel Pentium III 851 Mhz
processor with 512 MB of main memory running on Linux
2.4.7-timesys-3.1.214 kernel.  The TAO middleware used for the
experiments was version 1.4.7, which was compiled with gcc version
3.2.2.

TAO provides a wide range of configuration options\footnote{A detailed
description of all the ACE+TAO configuration settings is available
from \url{www.cs.wustl.edu/~schmidt/ACE_wrappers/TAO/docs/Options.html}.}
For this analysis, we used the following configuration: (1) portable
interceptors are not used, (2) servants inherit statically from
\url{org::omg::PortableServer::Servant}, {\em i.e.}, we do not
consider CORBA's dynamic invocation/skeleton features, (3) native C++
exception support was disabled, (4) no proprietary policies were used
in the ORB, and (5) TAO's general-purpose optimizations ({\em e.g.},
active demultiplexing, perfect hashing, \url{Thru_POA} collocation
optimization and buffer caching strategies) were enabled for all
experiments.  These assumptions are representative of ways in which
DRE systems commonly apply Real-time CORBA
middleware~\cite{Wendy:03,Sharp:03}.

To showcase our results, we performed the following types of
experiments for each specialization presented in
Sections~\ref{sec:resolve_1} through~\ref{sec:resolve_5}:
\begin{enumerate}\crunchlist
\item {\bf End-to-end latency metrics}, which measure the differences
in end-to-end latency/throughput between general-purpose and
specialized versions of TAO.  For each experiment, timers on the
client collected end-to-end measurement data used for analysis.

\item {\bf Path specialization metrics}, which compare latency
measures for specialized path vs. general-purpose paths.  For each
experiment, timers within TAO's ORB core measured latency improvements
for the code path specialized within TAO.

\item {\bf Cumulative metrics}, which measure the end-to-end latency
and predictability improvements accrued by applying all
specializations.  For each experiment, we define {\em predictability}
as the measure of standard deviation of the data points.
\end{enumerate}

\noindent A sample size of 100,000 data points was used to generate results for
each experiment.  The term predictability has different connotations in
different disciplines.  For example, in real-time scheduling theory,
a predictable system means that each task always meets its deadline.
For these experiments, we define {\em predictability} as the measure of
standard deviation of the data points.

% Guys, this is very cool, but I don't think we'll have room!
\iffalse
In all the middleware specialization experiments described below, we
use the following format to describe our approach:
\begin{itemize}\crunchlist
  \item {\bf Intent,} which provides the purpose for applying the
  specialization technique under consideration.
  \item {\bf Invariance assumption,} which indicates the assumptions
  on the invariant parts of the system that we leverage for the
  specializations.
  \item {\bf Steps specialized,} which indicates the steps shown in
  Figure~\ref{client-server-trace} that we focus on for our specialization
  techniques.
  \item {\bf Specializations,} which describe the result of our
  specializations to the middleware.
  \item {\bf Standards compliance,} which indicates how our
  specializations have affected the standards compliance of the
  middleware.  Our goal is to remain maximally compliant with the
  standards in order for our specializations to be widely applicable
  to situations demanding similar optimizations.
  \item {\bf Empirical Results,} which illustrate the empirical
  benchmarking results for the specializations we applied to the
  middleware.
\end{itemize}
\fi

\sqztiny
\subsubsection {Specialize for Target Object Location}
\label{sec:resolve_1}
\iffalse
This experiment evaluates a middleware specialization technique that
focuses on alleviating the overhead of target object location, which
addresses Challenge 1 in Section~\ref{mw-spl-challenges}.
\fi
\sqzlarge
\paragraph* {\bf Intent} Specialize the code path through the
middleware for certain collocated components that do not require
remoting.

\sqzlarge
\paragraph* {\bf Invariance assumption} Collocated components in a
product variant or applications-specific context that need no remoting
capabilities.

\sqzlarge
\paragraph* {\bf Steps specialized}
This specialization corresponds to step 1 in the client side of
Figure~\ref{client-server-trace}.

\sqzlarge
\paragraph* {\bf Specialization} The middleware hosting the {\tt
Airframe} and \url{Nav_Display} components need not test for remote
calls in the generated stub.
% Arvind, what is the difference between *stub* and *proxy*?  Don't
% you mean ``skeleton'' rather than ``proxy''?
%%%
% Arvind: I have changed this accordingly.
%%%
Similarly, this test can be removed in the skeleton classes.  To reduce
footprint, all classes that involve connection handling capabilities
({\em e.g.}, opening connections, marshaling data, resolving and
parsing IORs) can be removed entirely from the middleware.
%%%
% Arvind: Say what kind of a technique is this.
%%%
This specialization is an example of code-refactoring technique
which can help improve both performance and footprint.

\sqzlarge
\paragraph* {\bf Applicability and CORBA compliance} This
specialization is applied above and beyond the standard collocation
optimization supported in a Real-time CORBA implementation and
completely eliminates remoting tests (the generality) in the
collocation optimization.  This specialization is compatible with the
CORBA specification since no changes are made to the CORBA interfaces
and all collocation invocations go through the POA, thereby honoring
application developer-designated CORBA policies.

\sqzlarge
\paragraph* {\bf Empirical results}
Figure~\ref{fig:target-loc-results} shows the footprint and throughput
improvements accrued by the target object location specialization.
\begin{figure}[htpb]
\begin{center}
{\includegraphics[width=7cm]{figures/Collocation.eps}}
  \bfcaption{Results for Target Object Location Specialization}
  \label{fig:target-loc-results}
\end{center}
\end{figure}
As shown in the figure, footprint for a collocated application
improves by $\sim$40 kiloBytes, which is a 12\% improvement over the
general-purpose TAO implementation.  This specialization also removes
redundant tests along the critical path, which improves end-to-end
throughput by $\sim$7\% over the general-purpose collocation
optimization implemented by TAO.  These results show how eliminating
redundant remoting functionality can improve size and performance of
general-purpose middleware.

\sqztiny
\subsubsection {Specialize Request Creation/Initialization}
\label{sec:resolve_2}
\iffalse
This experiment evaluates a middleware specialization technique that
focuses on alleviating the overhead of request creation and/or
initialization, which addresses Challenge 2 in
Section~\ref{mw-spl-challenges}.
\fi

\sqzlarge
\paragraph* {\bf Intent} Rather than creating a new CORBA
request repeatedly for each invocation, create/\-initialize a request
once and only update its state that changes.

\sqzlarge
\paragraph* {\bf Invariance assumption} Most operation parameters
and/or context information in the request do not change across
invocations.

\sqzlarge
\paragraph* {\bf Steps specialized} This specialization corresponds to
step 2 in the client side of Figure~\ref{client-server-trace}.

\sqzlarge
\paragraph* {\bf Specialization}
Figure~\ref{fig:request-spl} illustrates the structure of a two-way
CORBA request using GIOP version 1.2.
\begin{figure}[ht]
\centerline {
  \includegraphics[width=8cm]{figures/CORBA-Request-Spl}}
\bfcaption {Specialization opportunities for Request Creation Specialization}
\label{fig:request-spl}
\end{figure}

\begin{figure}[htpb]
  \centerline{\includegraphics[width=8cm]{figures/CORBA-Request-spl-results.eps}}
  \bfcaption{Results for Request Creation/Initialization Specialization}
  \label{fig:request-spl-results}
\end{figure}
As shown in the figure, every request has three components defined by
the CORBA specification: (1) a request header that indicates the type
of CORBA request ({\em i.e.}, GIOP version 1.0, 1.1, or 1.2) and the
total size of the message, (2) a request-specific header containing
the object key that uniquely identifies the servant and service
context information containing service-specific information, such as
the required priority and transaction/security contexts, and (3)
optional parameters that were passed as arguments to the operation.

Figure~\ref{fig:request-spl} also shows three types of {\em
incremental specializations} that can be applied.  First, only the
request header can be specialized, {\em i.e.}, its contents are held
constant, updating only the total size of the message.  Second, both
the request and the request-specific headers can be held constant,
updating only the payload.  Third, the entire CORBA request can be
reused wholesale across multiple request invocations.
This specialization is an example of memo-ization technique, where
a result (in our case, the CORBA request components) are
precomputed and saved rather than computing it each time.

\sqzlarge
\paragraph* {\bf Applicability and CORBA compliance}
Specializing the entire request is possible only if the request does
not change, which is the case for control messages sent between {\tt
Timer} and {\tt GPS} components.  Specializing the request and
request-specific header is possible if only the contents change
between requests, which is the case for the \url{get_data()}
operation.  This specialization can be applied for the Real-time CORBA
{\sc server\_declared} priority model, where the priority information
is set {\em a priori} during object reference creation.  Specializing
only the request header is applicable to all requests, though it has
the least payoff in terms of improvements in performance since it
represents a relatively small portion of the request.

All three approaches comply with the CORBA specification since they do
not change the type of the CORBA request message.  The third approach,
however, does not update the request identifier, which is used to
uniquely identify the client thread processing the response when
multiplexed connections are used.

\sqzlarge
\paragraph* {\bf Empirical results}
Figure~\ref{fig:request-spl-results} illustrates the end-to-end and
code path specialization improvements that result from applying the
request creation/initialization specialization on the request and
request-specific CORBA header.  The average end-to-end latency
measures improved by $\sim$8$\mu$sec (4\%), while the path
specialized improved by 25\%.  This discrepancy shows how much our
specialized code path influences end-to-end latency.  The dispersion
measures do improve but not significantly by applying this
specialization.  Both 99\% and
worst-case measures improve, which show this specialization
improves predictability.
% Arvind, the word ``contribution'' appears twice in this sentence.
% Can you please revise this so that a different term is used?
%%%
% Arvind: I have reworded the sentence below.
%%%
These results show how the end-to-end path specialization results
are influenced by the contribution from the actual path specialized.

\sqztiny
\subsubsection {Specialize Dispatch Resolution}
\label{sec:resolve_3}
\iffalse
This experiment evaluates a middleware specialization technique that
focuses on alleviating the overhead of request dispatch resolution,
which addresses Challenge 3 in Section~\ref{mw-spl-challenges}.
\fi
\sqzlarge
\paragraph* {\bf Intent} Resolve the target request
% Arvind, I changed ``processor'' to ``dispatcher'' here since it
% seems more consistent with the terminology and also avoid confusing
% people by thinking that by ``processor'' you mean ``CPU''!
%%%
% Arvind: Ok.
%%%
dispatcher once for the first request and use it to service all other
requests sent over the same dedicated connection.

\sqzlarge
\paragraph* {\bf Invariance assumptions} The same operation
or operations in the same IDL interface are invoked on a multiplexed
connection.

\sqzlarge
\paragraph* {\bf Steps specialized} This specialization corresponds to
step 7 to step 1 in the server side of
Figure~\ref{client-server-trace}.

\sqzlarge
\paragraph* {\bf Specialization} Figure~\ref{fig:dispatch-spl} shows
a normal layered demultiplexing path through a CORBA server, {\em
i.e.}, the ORB core locates the target POA, which locates the servant,
which locates the skeleton, which dispatches the request to an
application-defined method.
\begin{figure}[ht]
  \centerline{\includegraphics[width=6cm]{figures/Dispatch-Spl}}
  \bfcaption{Specializing Request Dispatching} \label{fig:dispatch-spl}
\end{figure}

\begin{figure}[htpb]
  \centerline{\includegraphics[width=8cm]{figures/Dispatch-spl-results.eps}}
  \bfcaption{Results for Dispatch Resolution Specialization}
  \label{fig:dispatch-spl-results}
\end{figure}
Rather than navigating this layered path, the specialized
implementation caches the skeleton servicing the request and invokes
the method on the skeleton directly.  A similar approach can be
applied to cache the target POA and servant.
%%%
% Arvind: Say what kind of a technique is this.
%%%
This specialization
is an example of layer folding plus memo-ization,
where an answer (in our case the dispatcher)
is saved for later use than recomputing it each time.  This enables
multiple middleware layers to be collapsed during request processing.

\sqzlarge
\paragraph* {\bf Applicability and CORBA compliance} This
specialization applies to the \url{get_data()} operation in the {\em
BasicSP} scenario where the same operation is invoked repeatedly.
Caching the target servant and skeleton sacrifices some CORBA
compliance since thread-specific state ({\em e.g.}, CORBA Current and
POA Current are not maintained.  However, this context information is
often unnecessary, {\em e.g.}, the POA current interface is used
primarily when the POA is associated with a \url{Default_Servant}
(where one servant handles all invocations) or \url{Servant_Manager}
(which creates a servant dynamically to handle requests).  Since these
dynamic CORBA policies are rarely used in DRE systems, the impact on
CORBA compliance is negligible in practice.  Moreover, this
specialization can be applied only when circumstances allow.

\sqzlarge
\paragraph* {\bf Empirical Results}
Figure~\ref{fig:dispatch-spl-results} illustrates the end-to-end and
code path specialization improvements.  The average end-to-end latency
measures improved by $\sim$30$\mu$secs, which is $\sim$16\% better
than the general-purpose TAO implementation.  For
% Arvind, I don't understand how speeding things up by 73 usec in the
% code path would only lead to a 50 msec improvement end-to-end.  Can
% you explain this?!
%%%
% Arvind: This is wrong. I checked again, and it is 43 mu secs
% which is ~ 30%. Fixed this...
%%%
the actual code path specialized this translates to $\sim$40\%
improvement in latency.
The dispersion measures for end-to-end latencies improved by a factor
of $\sim$1.5, while those for the specialized path were twice as good
as those for the general-purpose path.  The 99\% measures are similar
to the dispersion measures, indicating improvement in predictability.
The worst-case measures improved by 20\% when applying the
dispatch resolution specialization to the specialized path and by
$\sim$14\% for the end-to-end results.  These results show that
folding middleware layers improves predictability and latency
considerably.

\sqztiny
\subsubsection {Specialize Request Demarshaling}
\label{sec:resolve_4}
\iffalse
This experiment evaluates a middleware specialization technique that
focuses on alleviating the overhead of request demarshaling, which
addresses Challenge 4 in Section~\ref{mw-spl-challenges}.
\fi

\begin{figure}[ht]
  \centerline{\includegraphics[width=8cm]{figures/Demarshal-spl-results.eps}}
  \bfcaption{Results for Request Demarshaling Specialization} \label{fig:demarshal-spl}
\end{figure}
\iffalse
\begin{figure}[htpb]
  \centerline{\includegraphics[width=8cm]{figures/Demarshal-Incremental.eps}}
  \bfcaption{Results for Demarshaling {\tt sequence<long>}} \label{fig:demarshal-inc-results}
\end{figure}
\fi
\sqzlarge
\paragraph* {\bf Intent} Eliminate redundant tests for byte order
when demarshaling a CORBA request. Do not align the individual
components within the CORBA request.

\sqzlarge
\paragraph* {\bf Invariance assumptions} The communicating entities
are on nodes with the same byte order. The same (de)marshaling
mechanism exists on both the client and server.

\sqzlarge
\paragraph* {\bf Steps specialized} This specialization corresponds to
steps 5 and 6 in the server side of Figure~\ref{client-server-trace}.

\sqzlarge
\paragraph* {\bf Specialization} A CORBA ORB is required to test for
byte order compatibility for every part of a CORBA request (not just
the payload), including all fields in the CORBA request and
request-specific headers.  Figure~\ref{fig:request-spl} shows the
different parts of a CORBA request.  For a typical request with a few
basic types (such as {\tt long}, {\tt short}, and {\tt octet}
parameters), these tests translate to $\sim$15--20 tests.  Removing
these redundant tests can improve demarshaling efficiency
significantly, particularly as the data type complexity increases.
Similarly while marshaling a CORBA request, ORB implementations
align the individual components, {\em e.g.}, request size, id and
objectkeys, to their natural boundaries.
For a typical request with basic types, all the $\sim$15--20 components
need to be aligned.  Ignoring alignment would improve marshaling
efficiency and eliminate padding thereby reducing request size.
\iffalse
Unaligned data, however, forces a copy  at the recipient.  However
most ORBs always incur the overhead of a copy
(irrespective of alignment) between the memory managed by the ORB to
the memory managed by the application.
\fi
These specialization are an example of constant propagation technique
where the byte-order, is propagated along the request
to the recipient where the byte order is checked to ensure the validity of the
invariance assumption.  Similarly, un aligned data is sent along the
request to the recipient where demarshaling fails if data is expected
to be aligned.

\sqzlarge
\paragraph* {\bf Applicability and CORBA compliance} Elimination
of byte-order checks and ignoring alignment
specializations are applicable to deployments on homogeneous
environments {\em i.e.}, (1) nodes with same byte
order, {\em e.g.}, {\em NodeA} and {\em NodeB} in our {\em BasicSP}
scenario and/or (2) same middleware implementations used at sender and
receiver.
These specializations are not CORBA compatible. However, a middleware
implementation can add recovery mechanisms, such as checking for
byte order within the request before using the aforementioned
specializations.  Recovery mechanisms also violates our invariance
assumption.
% Arvind, I'm not really sure I undertstand what you are saying here,
% can you please explain it further?  I think that this optimization
% is NOT compliant with the CORBA spec since it wouldn't work properly
% if an ORB running on a node with a different byte order sent a
% message to an ORB that had this optimization enabled...
%%%
% Yes. I have changed this.
%%%

\sqzlarge
\paragraph* {\bf Empirical results}
Figure~\ref{fig:demarshal-spl} illustrates the end-to-end and path
specialization improvements.  The specialized path for this experiment
started when a server demarshaled a request until the response was
returned to the client.  Applying the specialization of ignoring
alignments improved end-to-end latencies by $\sim$8$\mu$secs (a
4\% improvement over general-purpose TAO implementation) while
eliminating byte order checks improved byte order checks by
$\sim$9$\mu$sec (a 4\% improvement over general-purpose TAO
implementation).  Path specialization results improved by
$\sim$ 4 -- 5$\mu$sec (a 10\% improvement) for both the cases.
Although the general-purpose TAO implementation performs tests on the
client and server for all fields in a CORBA request header, our
specialization improvements were relatively small since our initial
experiment sent a single {\tt long} data type, which only requires a
few byte order tests.

To quantify the improvements from this specialization for more complex
data types, we conducted another experiment that sent records of a
Library structure (IDL described in Appendix~\ref{library-example})
over the network.  The library structure consisted of a sequence of
\url{struct}s composed of basic types, structs and sequence of
\url{struct}.  The \url{stucts} where both simple (bounded length)
structures and complex (unbounded length) structures.
\iffalse
books each having
an ISBN (a fixed bounded struct), title (a simple struct), author
(a struct with sequences of primitive types), characters (sequence of
structs), available (a boolean) and an id (a long).  In this example
byte order tests need to be performed at multiple levels (for each
book, for each ISBN, for the number within the ISBN ....). Similarly,
padding and alignment checks need to be performed at multiple
levels, for example, padding and alignment has to be done for writing
length of name, born, dead and recommends files in the Author
structure. In the example, alignment operations are required for
each component in the book structure.
\fi
We ran the end-to-end latency measures for the library example with
both the specializations enabled.   The average end-to-end latency measures
with the specializations improved by $\sim$14\%\footnote{We do not
present detailed results for this example as it is not representative
of a DRE application.}.  These results underscore the fact that
specializations depend on the use cases that exercise the specialized
code.
% Arvind, you need to make this datatype MUCH more complicated, i.e., it
% should be some really complex type, e.g., a sequence of structs with
% fields of many different types, including other structs/sequences,
% etc.  You're still not really putthing things through the paces.
%%%%
% Arvind: I have used the library example and my work in Qualcomm
% to automate the marshaling and demarshaing from XSC to take the
% above results.
%%%%
\iffalse
a {\url{sequence<long>} over the network.  The length of the sequence
ranged from 64 to 2,048 in powers of 2 and end-to-end results were
gathered for each case.  To extract each member of the sequence, a
byte-order test would normally be performed.  Eliminating redundant
demarshaling therefore removes these byte order tests.

Figure~\ref{fig:demarshal-inc-results} shows how our specialized
version improves end-to-end latencies as the length of the sequence
increases.  In particular, latency improves only by $\sim$2\% for a
sequence of size 64, where as for sequence of size 2,048, it improves
by $\sim$8\%.  These results underscore that specializations depend on
the use cases that exercise the specialized code, as well as on the
underlying middleware implementation.
\fi

\sqztiny
\subsubsection {Specialize Middleware Framework Implementation}
\label{sec:resolve_5}
\iffalse
This experiment evaluates a middleware specialization technique that
focuses on alleviating the overhead of overly extensible middleware
framework implementations, which addresses Challenge 5 in
Section~\ref{mw-spl-challenges}.
\fi
\sqzlarge
\paragraph* {\bf Intent} Eliminate indirections and dynamic
dispatching in middleware components along the critical
request/\-response processing path.  This specialization can be
applied to many ORB frameworks, such as transport protocols, request
demultiplexing, and concurrency models.  For our feasibility study, we
choose to specialize TAO's Reactor~\cite{Schmidt:02a} framework, which
is responsible for demultiplexing connection and data events to their
corresponding GIOP event handlers and TAO's pluggable protocol
framework~\cite{Schmidt:99x} which allows the ORB to transparently use
different protocol implementations such as TCP/IP, VME, or shared memory for
communication.

\sqzlarge
\paragraph* {\bf Invariance assumptions} Once a Reactor framework
implementation is selected, it does not change during the lifetime of
the ORB.  Similarly, once a protocol implementation is selected it
does not change during the lifetime of the ORB.

\sqzlarge
\paragraph* {\bf Steps specialized} This specialization corresponds to
step 3 in the client side and steps 4 and 9 in the server side of
Figure~\ref{client-server-trace}.

\sqzlarge
\paragraph* {\bf Specialization} Figure~\ref{fig:reactor-spl} shows
different Reactor implementations.
\begin{figure}[ht]
  \centerline{\includegraphics[width=6cm]{figures/Reactor-Spl}}
  \bfcaption{Reactor Specialization} \label{fig:reactor-spl}
\end{figure}
The \url{Select_Reactor}, which uses the single-threaded {\tt
select()}-based event demuxer, \url{Thread_Pool_Reactor}, which is a
multi-threaded {\tt select ()}-based event demuxer, and
\url{WFMO_Reactor}, which uses the Windows
\url{WaitForMultipleObjects()} event demuxer.  To work transparently
across all Reactor implementations, extensible ORBs like TAO use an
abstract base class ({\em e.g.}, a generic \url{Reactor_Impl}) that
delegates to concrete subclasses via virtual method calls.
Specializing the Reactor framework with a concrete subclass ({\em
i.e.}, a subclass with no virtual methods) eliminates the indirection
(generality) by using the concrete reactor instance directly.

\begin{figure}[ht]
  \centerline{\includegraphics[width=5cm]{figures/Pluggable-Protocol}}
  \bfcaption{Protocol Specialization} \label{fig:pluggable-protocol}
\end{figure}
The pluggable protocol implementation in TAO uses the template method
pattern~\cite{Vlissides:94} to dynamically plug and play different
protocol
implementations.  As shown in figure~\ref{fig:pluggable-protocol},
TAO's protocol framework consists of protocol independent
components such as the \url{Transport}\footnote{The transport class
encapsulates
a connection and provides a protocol independent way of
sending/receiving data.} class, that provide template methods or
hooks (\url{send ()} and \url{recv ()}).  Protocol specific
components, classes such as the \url{IIOP_Transport} class, override these
hooks to implement protocol specific functionality.  The \url{Transport}
class in turn interacts with other framework components, such as
the \url{Profile} class\footnote{Profile class encapsulates addressing
information in TAO.}, which in turn uses the template method pattern
to work across multiple protocol implementations.
Specializing the template methods with the concrete protocol
implementation eliminates the indirection (virtual hook methods) by filling
in the protocol specific behavior within the template methods.
%%%
% Arvind: Specialization example.
%%%
The aforementioned
specializations are an example of aspect weaving technique, where
generality (virtual-functions and indirections) which crosscuts
different classes and files is tailored to be more specific.

\sqzlarge
\paragraph* {\bf Applicability and CORBA compliance} This
specialization is applicable to all ORB frameworks that use virtual
methods for extensibility, yet can be customized via a single concrete
instance late in the system lifecycle, {\em e.g.}, during deployment
or initialization.  This
specialization is CORBA-compliant since the reactor is part of TAO's
ORB core implementation, not part of the public API defined by the
Real-time CORBA specification.  Similarly the specialization of the
protocol framework does not modify any of the specification defined
API but affects only the template methods which are specific to TAO.

\sqzlarge
\paragraph* {\bf Empirical results}
\begin{figure}[htpb]
  \centerline{\includegraphics[width=8cm]{figures/Reactor-spl-results.eps}}
  \bfcaption{Results for Reactor \& Protocol Specializations}
  \label{fig:reactor-spl-results}
\end{figure}
Figure~\ref{fig:reactor-spl-results} illustrates the improvements to
end-to-end latency by our middleware framework specialization.  Since
reactors and protocols are used by both client and server ORB,
we present the most representative end-to-end results.
Our specialization results in average latency improvements of
$\sim$8$\mu$secs (4\%) for the reactor
case and in $\sim$10$\mu$sec (5\%) for the protocol case.  This
specialization also minimizes dispersion measures for both
the specializations, though not
appreciably.  The 99\% and worst-case measure also decrease, thereby
showing how removing virtual method indirection enhances
predictability.
These results show how minimizing dynamic dispatch along the critical
path can improve performance.  Similar approaches can be applied to
specialize other frameworks used within an ORB implementation.  Based
on the results obtained in this feasibility study, our future work will
apply middleware framework specializations more broadly to improve
performance more significantly.

\sqztiny
\subsubsection {Applying Specializations Cumulatively}
\label{cumulative}

Figure~\ref{fig:optimisitic-spl} illustrates the QoS improvements
accrued by applying all of the middleware specializations discussed above
to a remote\footnote{Naturally, we exclude the specialization for
target location since that only applies for collocated requests.}
CORBA request.  The average end-to-end latency dropped by
$\sim$61$\mu$secs, which is a $\sim$35\% improvement.  Dispersion
measures were nearly twice as good as those for the general-purpose TAO
implementation, indicating considerable improvement in predictability,
which is essential for DRE systems.

\begin{figure}[htpb]
\centerline{\includegraphics[width=7cm]{figures/Optimistic-Optimization.eps}}
  \bfcaption{Results for Cumulative Specialization Application}
  \label{fig:optimisitic-spl}
\end{figure}

Similarly, the 99\% bound values
for our specialized approach are closer to the average than those for
the general-purpose TAO implementation, indicating higher
predictability.  Worst-case measures also improved by
$\sim$150$\mu$secs, which is a 45\% improvement over the
general-purpose TAO implementation.  End-to-end throughput
improved by $\sim$2500 calls/sec representing a gain of
$\sim$50\%.
\iffalse
Table~\ref{cumulative-table}
shows that end-to-end throughput improved by $\sim$50\%.
{
\begin{table}[htpb]
\begin{center}
\footnotesize
\begin{tabular}{|c|c|}\hline
{\bf Scenario} & {\bf Throughput calls/sec}
\tabularnewline \hline
    General & 5326 \tabularnewline \hline
    Specialized & 8160 \tabularnewline \hline
\end{tabular}
\caption {Cumulative Specialization Throughput Improvements}
\label{cumulative-table}
\end{center}
\vspace{-0.5cm}
\end{table}
}
\fi
As we apply other specializations
-- particularly the middleware framework specializations discussed in
Section~\ref{sec:resolve_5} -- we expect to find even better latency,
jitter, and throughput improvements.

\sqztiny
\subsection {Summary, Evaluation \& Lessons Learned}
\iffalse
{
\begin{table}[htpb]
\begin{center}
\scriptsize
\begin{tabular}{|c|c|}\hline
{\bf Specialization } & {\bf Technique}
\tabularnewline \hline
 Collocation & Code refactoring \tabularnewline \hline
 Request creation & Memo-ization \tabularnewline \hline
 Demarshaling checks & Code refactoring \tabularnewline \hline
 Dispatch resolution & Memo-ization + Layer folding \tabularnewline \hline
 Framework generality & Aspect weaving \tabularnewline \hline
\end{tabular}
\caption {Summary of Specialization Techniques}
\label{spl-summary}
\end{center}
\vspace{-0.5cm}
\end{table}
}

Table~\ref{spl-summary} describes the different specialization techniques
that have been applied to resolve middleware generality challenges.  The
memo-ization, layer folding and aspect weaving techniques have been automated
using our specialization framework described in Section~\ref{FOCUS-approach}
while the code refactoring technique was automated using a conditional
compilation technique in TAO.  The qualitative and quantitative results
from our specialization techniques illustrated that:
\fi

The specialization techniques discussed in this section are targeted
towards PLA architects who during the SCV analysis phase identify the
applicability of the different specialization techniques to the
family, {\em i.e.}, the PLA or to a specific product, {\em i.e.}, a
variant.  Among the specializations examined and implemented in this
paper, memo-ization and layer folding, lead to the greatest
improvement in QoS followed by aspect weaving technique while code
refactoring technique enabled both performance and footprint
improvements.  These results show how by passing middleware layers at
the server side in TAO can improve performance significantly.  Our
future work will examine how layering can be eliminated at the client
side as well. Middleware frameworks also provide opportunities for
applying aspect weaving specialization technique to other components
such as the concurrency framework, and the different strategies for
connection management and purging.  Eliminating dynamic dispatching to
all components along the critical request/response path would improve
performance further.  Our future work will examine more of these
techniques.  The remainder of this section discusses the 
consiequences/implications of our specialization techniques.

\sqzlarge\paragraph* {\bf Implications on QoS} The improvement in QoS by
applying the specializations cumulative is largely {\em additive},
{\em i.e.}, the 35\% improvement is roughly the sum of the individual
specializations.  This property stems from the fact that these
specializations were carefully chosen not to overlap in the end-to-end
code path and have no dependencies.  As middleware and system
architects develop a catalog of specializations it will be necessary
to document the interplay between the specializations and analyze the
implications on mixing and matching different specializations.
Similarly, not all the specializations will be applicable to every PLA
application scenario.  A PLA architect in conjunction with middleware
developers would need to determine the applicability of the different
product variants.  This is done during the SCV analysis phase.

Quantitative results show that improvements from applying
our specializations are scenario specific.  For example, the
demarshaling results showed how a complicated structure benefited
more from the specialization than a simple type.  Therefore,
when the specialized path is traversed more often, its influence
on end-to-end performance is more significant.

\sqzlarge\paragraph* {\bf Implications on applicability to different middleware Implementations}
The improvements to QoS from our specialization are implementation
specific. Preliminary results~\cite{daugherty:04} of applying our
specializations to ZEN~\cite{ZEN} which is a Java based CORBA
middleware implementation has resulted in performance improvements
that are better than the results discussed in this paper.  To enable
other middleware implementors to analyze and implement the
specializations we are working on developing a comprehensive CORBA
\emph{specialization model} based on~\cite{daugherty:04} that --
independent of a particular CORBA implementation -- identifies (1)
points in an ORB architecture where specialization is beneficial and
(2) API extensions to the architecture that provide {\em hooks} for
achieving effective specialization.

Another observation is that the strength general-purpose optimization
influences the improvements to performance from the specializations.
For example, our dispatch resolution optimization improved latency
measures by 16\% over and above the active demultiplexing and perfect
hashing general-purpose optimizations.  These optimizations ensure
$O(1)$ lookup time bound for all dispatch operations in TAO for the
general case and incur the least overhead compared to other
strategies~\cite{Schmidt:98v}.  Naturally, using any other techniques
such as linear search, which incurs $O(n)$ time bound would lead to a
greater pay off from using our specializations.

\sqzlarge\paragraph* {\bf Implications on recovery mechanisms}

\sqzlarge\paragraph* {\bf Implications on Scalability}

\iffalse
$\bullet$ Certain specialization techniques rely and enhance
general-purpose middleware optimizations while others
are independent. For example, the collocation specialization
requires the middleware implementation to provide the collocation
optimization, while the request caching optimization does not
require a buffer caching scheme to be implemented by the ORB.  This
shows that some specializations may be applicable to more mature,
{\em i.e.}, feature rich, middleware implementations while others
can be applied universally.

$\bullet$ The context-specific specializations use TAO implementation
artifacts, which limits their applicability, even though the
specializations can conceptually be applied to other CORBA ORBs.
To enable other middleware implementors to analyze and implement
the specializations, we have described the specializations using
a template that brings out the intent (purpose),
invariance assumptions (conditions
under which to apply the specializations), steps (where in the request
path specializations apply) and type (specialization technique) to
implement the specialization.  We are also working developing a
comprehensive CORBA \emph{specialization model}
based on~\cite{daugherty:04} that -- independent of a particular CORBA
implementation -- identifies (1) points in an ORB architecture where
specialization is beneficial and (2) API extensions to the
architecture that provide {\em hooks} for achieving effective
specialization.
\fi
