% $Id: mdd.tex,v 1.8 2005/12/09 12:57:25 gokhale Exp $
\section{Scaling and Automating the Performability Modeling Process}
\label{sec:mdd}

The previous sections described how a SRN model for performability of
the desired application can be developed.  The process described until
now focusses on manually developing these models and the associated
input scripts used by the model solvers, such as SPNP.  As the
application complexity grows, however, it becomes tedious and error
prone to develop these models manually since the accidental
complexities involved in modeling the system performance
characteristics and the input script sizes for the model solver grow
substantially.  As explained earlier there could be several different event
types that an event demultiplexing and handling framework may have to
handle.  Moreover, it should be possible for a software architect, who
is a non expert in the tools and techniques of performance analysis to
be able to use the model.  To enable these dual objectives, it is
necessary to encapsulate the performance modeling approach described
in the earlier sections into user-friendly tools that enable model
scalability to large systems.  In this section, we describe the
model-driven development (MDD)~\cite{Janos:97,MDA:01} approach, which
allows the user to scale the smaller base model and automate the
process of performance analysis.

\subsection{Modeling Languages for Performance Analysis}
\label{sec:dsml}
In this section we describe the ideas based on a
model-driven~\cite{MDA:01} generative programming~\cite{Czarnecki:00}
framework we are developing to address the aforementioned challenges.
Our modeling framework comprises modeling languages we have developed
using the Generic Modeling Environment (GME)~\cite{GME:01}.  GME is a
tool that enables domain experts to develop visual modeling languages
and generative tools associated with those languages.  The modeling
languages in GME are represented as metamodels.  A metamodel in GME
depicts a class diagram using UML-like constructs showcasing the
elements of the modeling language and how they are associated with
each other.  The GME environment can then be used by application
developers to model examples that conform to the syntax and semantics
of the modeling language captured in the metamodels.  

We have developed two modeling languages that provide the visual
syntactic and semantic elements required to model the systems
compositions and their performance models.\footnote{Formal
  descriptions of these modeling languages is beyond the scope of
  this paper.}  The first language is called POSAML (Patterns-Oriented
Software Architecture Modeling Language), which models the patterns
described in the POSA~\cite{Schmidt:00a} pattern language.  The POSA
pattern language is a vocabulary describing a set of related patterns
used to develop network services.  The other modeling language is
called SRNML (Stochastic Reward Net Modeling Language), which enables
a user to model the behavior of individual patterns as a SRN. 

Figures~\ref{fig:posa-meta-bench} and~\ref{fig:posa-meta-conf}
illustrate two different views of the metamodel for the
POSAML language.  Although the formal description of this language is
beyond the scope of this paper, we describe the important features
supported by this language.  POSAML allows a user to describe the
middleware as a collection of patterns.  Progress to date on POSAML
enables users to model the Reactor, Acceptor and Connector patterns.
POSAML decouples the middleware structural composition i.e, the way
individual patterns are associated with each other from the
variability modeling in each pattern.  The variability in each pattern
stems from the allowable configurations of each pattern, which can be
modeled using the configuration capabilities of the language.  For
example, an Acceptor pattern can be configured with the transport
protocol and the end point it will use to listen for incoming
requests.  Yet another dimension of modeling permitted by POSAML is
related to benchmarking, a capability enabled by the benchmarking view
of the language.  This view of the system enables users to model the
workload characterization for the composed system.  For example, it is
possible to model the input arrival patterns of requests, the service
rates of the event handlers in the systems, and sizes of thread pools.

\begin{figure}[!htb]
\centering\includegraphics[width=\columnwidth]{eps/benchmarking_meta.eps}
\caption{POSAML Metamodel: Benchmarking View}
\label{fig:posa-meta-bench}
\end{figure}

\begin{figure}[!htb]
\centering\includegraphics[width=\columnwidth]{eps/features_meta.eps}
\caption{POSAML Metamodel: Configuration View}
\label{fig:posa-meta-conf}
\end{figure}

Built in constraint checking in POSAML enables correct by construction
structure of system compositions since any erroneous associations
between different patterns are flagged as error at modeling time.
Moreover, the feature modeling view and benchmarking view of the
language also flags errors when a modeler commits errors.  For
example, if a modeler choses a single threaded reactor demultiplexing
strategy in the feature view but provides a thread pool configuration
in the benchmarking view, then the constraint checker flags this
condition as an error.

Figure~\ref{fig:posa-example} illustrates an example middleware
composition comprising three patterns of the POSAML language.  The
figure illustrates the configuration view of the system.   Such a
configuration will be useful in generating the artifacts for the
empirical benchmarking tests as well as simulations.

\begin{figure}[!htb]
\centering\includegraphics[width=4in]{eps/posaml-example.eps}
\caption{POSAML GME Model for the Reactor Pattern}
\label{fig:posa-example}
\end{figure}

A second language we have developed is the Stochastic Reward Net
Modeling Language (SRNML).  Parts of SRNML are shown in
Figure~\ref{fig:srn-meta}.  SRNML provides the artifacts necessary to
model a system as a SRN.

\begin{figure}[!htb]
\centering\includegraphics[width=4in]{eps/srn-meta.eps}
\caption{SRN Metamodel}
\label{fig:srn-meta}
\end{figure}

Figure~\ref{fig:srn-example} illustrates the SRN model we have used
for our case study and evaluation described in
Section~\ref{sec:eval_perf}. 

\begin{figure}[!htb]
\centering\includegraphics[width=4in]{eps/srn-example.eps}
\caption{SRN GME Model of the reactor pattern}
\label{fig:srn-example}
\end{figure}

\subsection{Generative Tools and Model Scalability}
\label{sec:gen}

The GME environment allows metamodel developers to plugin model
interpreters that can provide various capabilities, such as code
generation or configuration information.

In POSAML to date we have included two model interpreters with
generative capabilities.  The benchmark interpreter described here
traverses all the elements of the model (i.e. Pattern, Feature and
Benchmarking), extracting relevant information and generating an XML
file to drive the simulations and benchmarking.  The relevant
information captured in the synthesized metadata includes:
\begin{enumerate}
  \item the Reactor type (extracted from the Feature Aspect), such as
  a single threaded or multi-threaded.
  \item the desired metrics, such as whether the measurements should
  include latency or throughput or both.
  \item the number of data exchanges (which is basically the number of
  times one client thread generates events for the reactor).
  \item the number of client threads (or connections).
  \item the number of event handlers (extracted from the Pattern Aspect)
  \item the service time required by a handler specified as uniform
  distribution or exponential distribution.
  \item arrival pattern of requests modeled as Poisson or periodic
  arrivals. 
\end{enumerate}

Following is a snippet of the XML file generated by the benchmark
interpreter: 
\begin{verbatim}
<benchmark_inputs>
   <connections>5</connections>
   <data>ABCDEF</data> 
   <data_exchanges>200</data_exchanges>
   <reactor_inputs>
   <reactor_type>wfmo</reactor>
      <handlers>2</handlers>
   </reactor_inputs>
 </benchmark_inputs>
\end{verbatim}


The other interpreter in POSAML is the configuration interpreter,
which synthesizes the configuration parameters for each pattern
modeled in the system.

\iffalse
% We will use this later sometime

The benchmarking program is thus "configured" with the help of these
parameters and used for performance analysis.  

This benchmarking program does the following things:
\begin{enumerate}
  \item The client and server are in the same program.
  \item The server creates the reactor depending on the type of
  reactor it is (wfmo, select\_st, etc).
  \item The program creates client threads and and for each thread
  establishes a connection with the server using the
  acceptor-connector pattern.  It uses the command line arguments
  "-ORBEndpoint iiop://127.0.0.1:9000" to specify host address, port
  and protocol. (Right now it just takes a default value, but I can
  implement it).
  \item The client thread sends data to the server for
  <data\_exchanges> no. of times.
  \item The reactor (which is on the server side), notifies a read
  event handler whenever data is received from the client.
  \item The read handler just reads the data, but doesn't do anything
  about it in this example program. (I still have to implement the
  <service\_time> distribution part.)
  \item When a thread finishes sending data its connection is closed.
  \item The program then prints out the total elapsed time starting
  from the first client send call.  
\end{enumerate}

\begin{verbatim}
Service Configurator file will look like this:
static Advanced_Resource_Factory " -ORBReactorType tp -ORBReactorThreadQueue
LIFO"
static Server_Strategy_Factory "-ORBConcurrency reactive"

and Script file will look like this:

benchmark_test -ORBEndpoint iiop://127.0.0.1:9000
\end{verbatim}
\fi


Model scalability is addressed using a model replicator tool we have
developed previously called C-SAW (Constraint-Specification Aspect
Weaver)~\cite{Gray:01}.  The model replication and scalability was
discussed in an earlier work of ours~\cite{Gray-MoDELS05a}. 
We have designed C-SAW to provide support for modularizing
crosscutting modeling concerns as well as scaling models in the GME.
This weaver operates on the internal representation of a model
(similar to an abstract syntax tree of a compiler).  GME provides a
framework that allows metamodel developers to register custom actions
and hooks with the environment.  These hooks can read and write the
elements of a model during the modeling stage.  GME also provides an
introspection API, which provides knowledge about the types and
instances of a model, without {\em a priori} knowledge about the
underlying metamodel.  Utilizing this feature of GME, we have
implemented C-SAW as a ``plug-in,'' which is GME terminology for a
metamodel independent hook.  Thus, the benefits of C-SAW are
applicable across a whole spectrum of metamodels.  C-SAW is thus
useful to scale the models such as those modeled by POSAML and
SRNML.  


\iffalse

\subsection{Scaling the Models via Model Replicators}
\label{sec:replicator}

Section~\ref{sec:dsml} describes the modeling languages we have
developed to model the system compositions and the stochastic reward
net models of systems.  


\subsection{Unresolved Challenges for Model-driven Frameworks} 
\label{sec:mdd-challenges}
In the above sections, we describe how the base model which captures the
essential concepts could be easily scaled using a model-driven framework. 
The scaled model essentially embodies the same characteristics of the 
original base model. In practice, however, it may be necessary to
vary certain characteristics of the base model, in order to create 
``variants'' of the pattern.  For example, some applications that
must provided differentiated levels of service must handle events in a
prioritized manner, and the implementation of such applications will
require a reactor pattern which provides prioritized handling. 
Another possibility is to handle events
concurrently in multiple threads. In the future, we propose
to develop the model-driven framework further, so that creating 
the variants of the base model will be facilitated, in addition
to scaling the models with minimal manual intervention. 

\fi

%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "issre-2005"
%%% End: 
