%% $Id$
\section{Introduction}
Society is increasingly reliant on the services provided by 
distributed, performance-sensitive software systems. These systems 
demand multiple simultaneous quality of service (QoS) properties 
including predictability, controllability, and adaptability of 
operating characteristics for applications with respect to such
features as time, throughput, accuracy, confidence, security 
and synchronization. A key enabler in recent successes in the 
development of such systems is {\em QoS-enabled middleware} 
(\cite{Schmidt:01o}). 
Middleware comprises software layers that provide platform-independent
execution semantics and reusable services that coordinate how application
components are composed and interoperate. The flexibility and configurability
offered by middleware is manifested in the large number of reusable software
building blocks and their configuration options, which can be used to
compose and build large systems end-to-end. These building blocks embody good
design practices called patterns (\cite{Vlissides:94,Schmidt:00a}). 
The choice of the patterns
and their configuration options is driven by the context of the
application. These choices have a profound impact on 
the overall performance of the provided service. 

Current {\em ad hoc} techniques based on manually choosing the right
set of building blocks and their configuration options are error-prone
and may adversely impact performance, system costs and schedules, since
most errors are caught very late in the lifecycle of the system development. 
It is desirable to have the ability to analyze the performance of 
individual building blocks and the composed system much earlier in the
system lifecycle, thereby significantly lowering system testing costs 
as well as improving the correctness of the final developed system. 

To address the challenge of system performance evaluation in the 
design phase, a systematic performance analysis methodology is necessary.
This methodology would consist of the performance models of the
individual building blocks and their compositions. The performance 
models may be based upon well-known analytical/\-numerical modeling 
paradigms (\cite{evol-spn,mrspn,fspn-1}) and 
simulation techniques (\cite{NS-2}). As a first step
towards the development of such a methodology, this paper presents a 
model of the {\em Reactor pattern} (\cite{Vlissides:94,Schmidt:00a}), 
which provides important synchronous demultiplexing and dispatching 
capabilities to network services and applications. The model is based
on the Stochastic Reward Net (SRN) modeling paradigm (\cite{evol-spn}). 
We illustrate how the model can be used to obtain estimates
of the performance metrics of a Virtual Private Network (VPN) 
service provided by a Virtual Router (VR) (\cite{Knight:04}).  

\noindent {\bf Paper organization:}
The paper is organized as follows: Section~\ref{sec:perf_model} 
presents the performance model of the reactor pattern. Section~\ref{sec:ill}
illustrates how the SRN model of the reactor pattern 
is used to obtain the performance metrics of a VPN service provided 
by a VR. Section~\ref{sec:concl} offers concluding
remarks and directions for future research. 

\section{Performance Model of the Reactor Pattern}
\label{sec:perf_model}
In this section, we first provide an overview of the reactor 
pattern followed by the SRN model of the reactor pattern. 
We then describe how the different performance
metrics can be obtained from the SRN model.

\subsection{Reactor Pattern in Middleware Implementations}
\label{sec:reactor}

Figure~\ref{fig:mw-demux} depicts a typical event demultiplexing and
dispatching mechanism documented in the reactor pattern. The
application registers an event handler with the event demultiplexer
and delegates to it the responsibility of listening for incoming
events. On the occurrence of an event, the demultiplexer dispatches the
event by making a callback to its associated application-supplied event
handler. This is the idea behind the reactor pattern, which provides
synchronous event demultiplexing and dispatching capabilities.

\begin{figure}[htb]
\begin{center}
\includegraphics[width=2.3in]{../Sigmetrics-2005/eps/demux.eps}
\bfcaption{Event Demultiplexers in Middleware}
\label{fig:mw-demux}
\end{center}
\end{figure}

Figure~\ref{fig:reactor_dynamics} illustrates the reactor pattern
dynamics. We categorize these dynamics into two phases described below.

\begin{enumerate}
  \item {\bf Registration phase:} In this phase all the event handlers
  register with the reactor associating themselves with a particular
  event type they are interested in.  Event types usually supported by
  a reactor are input, output, timeout and exceptions.  The reactor
  will maintain a set of handles corresponding to each handler
  registered with it.
  \item {\bf Snapshot phase:} Once the event handlers have completed
  their registration, the main thread of control is passed to the
  reactor, which in turn listens for events to occur. A snapshot is
  then an instance in time wherein a reactor determines all the event
  handles that are enabled at that instant.  For all the event handles
  that are enabled in a given snapshot, the reactor proceeds to
  service each event by invoking the associated event handler.  There
  could be different strategies to handle these events. For example, a reactor
  could handle all the enabled events sequentially in a single thread
  or could hand it over to worker threads in a thread pool. After all
  the events are processed, the reactor proceeds to take the next
  snapshot of the system.
\end{enumerate}

\begin{figure}[htb]
\begin{center}
\includegraphics[width=5in]{eps/reactor_dynamics.eps}
\bfcaption{Reactor Pattern Dynamics}
\label{fig:reactor_dynamics}
\end{center}
\end{figure}

The reactor pattern could be implemented in many different ways
depending on the event demultiplexing capabilities provided by the
operating systems and the concurrency requirements of the
applications. For example, the demultiplexing capabilities of a
reactor could be based on the {\tt select ()} or {\tt poll ()} system calls
provided by POSIX-compliant operating systems, or {\tt
WaitForMultipleObject ()} as found in the different flavors of Win32 operating
systems. Moreover, the handling of the event in the event handler
could be managed by the same thread of control that was listening for
events leading to a single-threaded reactor
implementation. Alternatively, the event could be delegated to a pool of
threads to handle the events leading to a thread-pool
reactor. 


\subsection{Characteristics of the Reactor Pattern}
\label{sec:reactor_modeling}
To simplify the description of our methodology, we consider a
single-threaded, {\em select}-based implementation of the reactor
pattern with the following characteristics:

\begin{itemize}%\crunchlist
\item The reactor receives two types of input events with one
      event handler for each type of event registered with the reactor.
\item Each event type has a separate queue, which holds the incoming
      events of that type. The buffer capacity for the queue of type
      \#1 events is denoted $N_1$ and of type \#2 events is denoted $N_2$.
\item Event arrivals for both types of events follow a Poisson
      distribution with rates $\lambda_1$ and $\lambda_2$, while the
      service times of the events are exponentially distributed with
      rates $\mu_1$ and $\mu_2$. 
\item In a snapshot, an event of type \#1 is serviced with a higher
      priority over an event of type \#2. In other words, when event handles 
      corresponding to both event types are enabled in a snapshot, the 
      event handle corresponding to type \#1 is serviced with a 
      priority that is higher than the event handle of type \#2. 
\end{itemize}

\subsection{Desired Performance Metrics}
\label{sec:reactor_metrics}

The following performance metrics are of interest for each one of
the event types in the reactor pattern described in
Section~\ref{sec:reactor_modeling}: 

\begin{itemize}%\crunchlist
\item {\bf Expected throughput} -- which provides an estimate of the number
      of events that can be processed by the single threaded event
      demultiplexing framework. These estimates are important for
      many applications, such as telecommunications call processing.
\item {\bf Expected queue length} -- which provides an estimate of the
      queuing for each of the event handler queues. These estimates are
      important to develop appropriate scheduling
      policies for applications with real-time requirements.
\item {\bf Expected total number of events} -- which provides an estimate of
      the total number of events in a system. These estimates are also
      tied to scheduling decisions. In addition, these
      estimates will determine the right levels of resource
      provisioning required to sustain the system demands.
\item {\bf Probability of event loss} -- which indicates how many events
      will have to be discarded due to lack of buffer space. These
      estimates are important particularly for safety-critical
      systems, which cannot afford to lose events. These
      also provide an estimate on the desired levels of resource
      provisioning. 
\item {\bf Expected response time} -- which indicates the time taken 
      to service an event. These estimates are important for real-time 
      services. 
\end{itemize}

\subsection{SRN Model}
\label{sec:srn}
In this section we present the Stochastic Reward Net (SRN) model of
the reactor pattern.  
A Stochastic Reward Net (SRN) substantially extends the modeling 
power of Generalized Stochastic Petri Nets (GSPNs) (\cite{evol-spn}), 
which are an extension of Petri nets (\cite{peterson81}). A SRN is a modeling
technique that is concise in its specification and closer
to a designer's intuition about what a model should look like. 
Stochastic reward nets (SRNs) can be solved using Stochastic
Petri Net Package (SPNP)~\cite{spnp-new} and
they have been extensively used for performance,
reliability and performability analysis of a variety of
systems (\cite{corba-perf,avail-vax,poll-perf,man-perf,queue-perf,rel-srn}).
The work closest to the proposed research is reported by Ramani {\em
et al.} (\cite{corba-perf}), where SRNs are used for the performance
analysis of the CORBA event service. A detailed overview of SRNs
can be obtained from (\cite{evol-spn}). 

\begin{figure*}[htb]
\centering
\includegraphics[width=4.5in]{../Sigmetrics-2005/eps/srn.eps}
\bfcaption{SRN model for the reactor pattern}
\label{fig:srn}
\end{figure*}

\paragraph*{Description of the net:} Figure~\ref{fig:srn} shows the 
SRN model for the reactor pattern with the characteristics described 
in Section~\ref{sec:reactor_modeling}. The net on the left-hand 
side models the arrival, queuing and service of the two types 
of events. Transitions $A1$ and $A2$
represent the arrival of the events of type \#1 and \#2, respectively. 
Places $B1$ and $B2$ represent the queue for the two
types of events. Transitions $Sn1$ and $Sn2$ are immediate transitions
that are enabled when a snapshot is taken. Places $S1$ and $S2$ 
represent the enabled handles of the two types of events, whereas 
transitions $Sr1$ and $Sr2$ represent the execution of
the enabled event handlers of the two types of events. An inhibitor
arc from place $B1$ to transition $A1$ with multiplicity $N1$ prevents 
the firing of transition $A1$ when there are $N1$ tokens in 
place $B1$. The presence of $N1$ tokens in place $B1$ indicates
that the buffer space to hold the incoming input events of the first
type is full, and no additional incoming events can be accepted.
The inhibitor arc from place $B2$ to transition $A2$ achieves
the same purpose for type \#2 events. The inhibitor arc from place 
$S1$ to transition $Sr2$
prevents the firing of transition $Sr2$ when there is a token in 
place $S1$. This models the prioritized service for an event of
type \#1 over event of type \#2 in a given snapshot.

The net on the right of Figure~\ref{fig:srn} models the process of 
taking successive snapshots and prioritized service of the event handle 
corresponding to type \#1 events in each snapshot. Transition $Sn1$ 
is enabled when there is a token in place $StSnpSht$, 
at least one token in place $B1$, and no tokens in place $S1$. 
Similarly, transition $Sn2$ is enabled when there is a token 
in place $StSnpSht$, at least one token in place $B2$, and 
no tokens in place $S2$. Transition $T\_SrvSnpSht$ is enabled 
when there is a token in either one of the places $S1$ and 
$S2$, and the firing of this transition deposits a token in
place $SnpShtInProg$. 

The presence of a token in the place 
$SnpShtInProg$ indicates that the event handles that were enabled 
in the current snapshot are being serviced. After these event
handles complete execution, the current snapshot is 
complete and it is time to take another snapshot.
This is accomplished by enabling the transition $T\_EndSnpSht$.
Transition $T\_EndSnpSht$ is enabled when there are no tokens
in both places $S1$ and $S2$. Firing of the transition $T\_EndSnpSht$
deposits a token in place $StSnpSht$, indicating that the service
of the enabled handles in the present snapshot is complete,
which marks the initiation of the next snapshot. Table 1
summarizes the enabling/\-guard functions for the transitions in the net.    

\begin{table*}[htb]
\begin{center}
\begin{tabular}{|c|c|} \hline
Transition & Guard function \\ \hline 
$Sn1$ & $((\#StSnpShot == 1) \&\& (\#B1 >= 1) \&\& (\#S1 == 0)) ? 1:0$ \\ \hline 
$Sn2$ & $((\#StSnpShot == 1) \&\& (\#B2 >= 1) \&\& (\#S2 ==0)) ? 1:0$ \\ \hline 
$T\_SrvSnpSht$ & $((\#S1 == 1) || (\#S2 == 1)) ? 1:0$ \\ \hline 
$T\_EndSnpSht$ & $((\#S1 == 0 \&\& (\#S2 == 0)) ? 1:0$ \\ \hline 
\multicolumn{2}{c}{$^1$ Enabling/\-Guard Functions}
\end{tabular}
\end{center}
\end{table*}

\paragraph*{Dynamic evolution of the net:}
We now describe how the process of taking a single snapshot is
modeled by the SRN model presented in Figure~\ref{fig:srn}. 
We consider a scenario where there is one token 
in each one of the places $B1$ and $B2$, and there is a token in the place 
$StSnpSht$. Also, there are no tokens in places $S1$ and $S2$.
In this scenario, transitions $Sn1$ and $Sn2$ are enabled. Both of
these transitions are assigned the same priority, and any
one of these transitions can fire first. Also, since these 
transitions are immediate, their firing occurs instantaneously. 
Without loss of generality, it can be assumed that transition $Sn1$ 
fires before $Sn2$, which deposits a token in place $S1$. 

When a token is deposited 
in place $S1$, transition $T\_SrvSnpSht$ is enabled. In addition, 
transition $Sn2$ is already enabled. If transition $T\_SrvSnpSht$ were
to fire before transition $Sn2$, it would disable transition
$Sn2$, and prevent the handle corresponding to the second event
type from being enabled. In order to prevent transition $T\_SrvSnpSht$
from firing before transition $Sn2$, transition $T\_SrvSnpSht$
is assigned a lower priority than transition $Sn2$. Because transitions
$Sn1$ and $Sn2$ have the same priority, this also implies that 
the transition $T\_SrvSnpSht$ has a lower priority than transition
$Sn1$. This ensures that in a given snapshot, event handles corresponding
to each event type are enabled when there is at least one event in
the queue. 

After both event handles are enabled, transition 
$T\_SrvSnpSht$ fires and deposits a token in place $SnpShtInProg$. 
The presence of a token in the place 
$SnpShtInProg$ indicates that the event handles that were enabled 
in the current snapshot are being serviced.
The event handle corresponding to type \#1 event is serviced first,
which causes transition $Sr1$ to fire and the removal of the token
from place $S1$. Subsequently, transition $Sr2$ fires and the event
handle corresponding to the event of type \#2 is serviced. This
causes the removal of the token from place $S2$. After both events
are serviced and there are no tokens in places $S1$ and $S2$, transition
$T\_EndSnpSht$ fires, which marks the end of the present snapshot and
the beginning of the next one. 

\paragraph*{Assignment of reward rates:} 
The performance measures described in
Section~\ref{sec:reactor_metrics} can be computed  
by assigning reward rates at the net level as summarized in 
Table 2. The throughputs of events of type 
\#1 and \#2 denoted $T_1$ and $T_2$ respectively are 
given by the rate at which transitions $Sr1$ and $Sr2$ fire. 
The queue lengths of the events denoted 
$Q_1$ and $Q_2$ are given by the number 
of tokens in places $B1$ and $B2$ respectively. The total number of 
events of type \#1 denoted $E_1$ is given by the sum of the number of tokens 
in places $B1$ and $S1$.
Similarly, the total number of events of type \#2 denoted 
$E_2$ is given by the sum of the
number of tokens in places $B2$ and $S2$. The loss 
probability of type \#1 events denoted $L_1$ is given by 
the probability of $N1$
tokens in place $B1$. Similarly, the loss probability of type 
\#2 events denoted $L_2$ is given by
the probability of $N2$ tokens in place $B2$.  

\begin{table*}[htb]
\begin{center}
\begin{tabular}{|c|c|} \hline
Performance & Reward rate \\ 
metric &                  \\ \hline 
$T_1$ & return rate($Sr1$) \\ \hline 
$T_2$ & return rate($Sr2$)  \\ \hline 
$Q_1$ & return ($\#B1$)  \\ \hline 
$Q_2$ & return ($\#B2$)  \\ \hline 
$L_1$ & return ($\#B1 == N1 ? 1:0$) \\  \hline 
$L_2$ &  return ($\#B2 == N2 ? 1:0$) \\ \hline 
$E_1$ & return($\#B1 + \#S1$) \\ \hline 
$E_2$ & return($\#B2 + \#S2$) \\ \hline 
$R_1$ & return($\#B1 < N1 ?$ \\
      &         $(1/\mu_1*(\#S1+\#B1+1) + 1/\mu_2*(\#S2+\#B1)):0$) \\ \hline 
$R_2$ & return($\#B2 < N2 ?$ \\
      & $(1/\mu_2*(\#S2+\#B2+1) + 1/\mu_1*(\#S1+\#B2+1)):0$) \\ \hline 
\multicolumn{2}{c}{$^2$ Reward assignments to obtain performance measures}
\end{tabular}
\end{center}
\end{table*}

We obtain the response times of the events denoted $R_1$ and 
$R_2$ using the tagged customer
approach (\cite{melamed84}). In the tagged customer approach, an arriving event
is tagged and its trajectory through the system is followed from entry to
exit. The response time of the tagged event is then determined conditional
to the state in which the system lies when the event arrives. The 
unconditional response time can be obtained as the weighted sum of
the conditional response times, with the weights given by the 
steady state probabilities of being in each one of the states. 
Typically, the response time of an event consists of two pieces; namely,
the time taken to service the event hereafter referred to as the 
``service time,'' and the time that the event must
wait in the system before its service commences, hereafter referred to
as ``waiting time''. In our case, the average service time of
an incoming type \#1 and type \#2 event 
is given by $1/\mu_1$ and $1/\mu_2$, irrespective of
the state in which the system lies when the event arrives. The waiting
time, however, will depend on the system state. Next, we discuss
how the conditional waiting time of each event type is determined.

The conditional waiting time for a tagged event
of type \#1 will depend on the state of the system,
where the state is given by the number of tokens or markings of 
places $S1$, $S2$, $B1$ and $B2$. Of these four places, the markings
of the places $S1$ and $S2$ determine the progress of the current
snapshot, whereas, the markings of places $B1$ and $B2$ determine
the state of the queue. The time taken to complete the current
snapshot is given by the sum of two terms, the first term is 
the product of the number of tokens in place $S1$ and $1/\mu_1$,
and the second term is the product of the number of tokens in
place $S2$ and $1/\mu_2$. Even if there are no additional events 
in the queues, the current snapshot must be completed before
the service of an incoming event of type \#1 can begin. Hence, 
the time taken to complete the current snapshot contributes to the 
waiting time of the incoming or tagged type \#1 event. In order to obtain
the entire waiting time of a tagged type \#1 event, the contribution 
of the queued events of type \#1 and type \#2 needs to be determined. 

Let $n_1$ be the number of events of type \#1 in the queue, 
and $n_2$ be the number
of events of type \#2 in the queue, when the tagged event of type \#1 
arrives. This implies that after $n_1$ snapshots the tagged event
will be serviced. The following three possibilities arise between
the relative values of $n_1$ and $n_2$. 
If $n_1 < n_2$, then only $n_1$ of the type \#2 events
need to be serviced before the service of the tagged type 
\#1 event can commence,
and hence the waiting time is given by $n_1(1/\mu_1 + 1/\mu_2)$. If 
$n_1 = n_2$, then $n_1$ events of type \#1 and type \#2 need to be serviced
before the service of the incoming type \#1 event can commence, 
and hence the waiting time is given by $n_1(1/\mu_1 + 1/\mu_2)$. 
If $n_1 > n_2$, then in the optimistic case, 
$n_1$ events of type \#1 and $n_2$ events of type \#2 need to be 
serviced before the service of the tagged event can commence. The
optimistic case assumes that no additional events of type \#2 arrive in
the first $n_1$ snapshots. In the pessimistic case, however, $n_1 - n_2$ 
events of type \#2 will arrive while the first $n_2$ events 
are being serviced. Thus,
in the optimistic case, the waiting time will be  $n_1/\mu_1 + n_2/\mu_2$,
and in the pessimistic case, the waiting time will be $n_1(1/\mu_1 + 1/\mu_2)$.
We consider the pessimistic case since that provides an upper bound on
the response time. The pessimistic contribution of the queued events 
to the waiting time is thus given by the product of the number of tokens in
place $B1$ and the sum of the reciprocals of $\mu_1$ and $\mu_2$.
Thus, the overall response time of the tagged event will be given
by the sum of two terms, the first term is $1/\mu_1$
times the sum of the tokens in places $S1$, $B1$ and $1$, and the 
second term is given by the product of $1/\mu_2$ and the sum
of the number of tokens in place $S2$ and $B1$. 
The contribution of the queued events to the waiting time of
the tagged event of type \#2 can also be determined using
similar reasoning, with an additional consideration given to
the prioritized service provided to event of type \#1 over an 
event of type \#2 in each snapshot. 
The reward rates to obtain the performance measures 
of the events of type \#1 and type \#2 are summarized in Table 2.

\subsection{Model Variations}
In the model of the reactor pattern described above, the arrival 
and the service time distributions are assumed to be 
exponential for the sake of illustration. 
For certain types of applications, this assumption
may not hold. For example, for safety-critical applications, 
events may occur at regular intervals, in which case the arrival
process is deterministic. In addition to the deterministic
distribution, the arrival and service times may also follow
any other non-exponential or general distribution. 
There are two ways to consider non-exponential distributions in
the SRN model. In the first method, a non-exponential 
distribution can be approximated 
using a phase-type approximation (\cite{evol-spn}), and the resulting SRN 
model can then be solved using SPNP.  In the second method, the model can be 
simulated using discrete-event simulation incorporated in 
SPNP (\cite{spnp-new}).  

\section{Case Study: VPN Service using Virtual Router}
\label{sec:ill}
In this section we describe how the SRN model of the reactor pattern
presented in Section~\ref{sec:srn} is used to estimate the response 
time of a Virtual Private Network (VPN) service provided by a 
Virtual Router (VR). 

Figure~\ref{fig:vpn-arch} illustrates the architecture of a
provider-provisioned virtual private network
(PPVPN) (\cite{Nagarajan:04}) using a VR. A VR is a
software/\-hardware component that is 
part of a physical router called the provider edge (PE) router. A VR
contains the mechanisms to provide highly scalable, differentiated
levels of services in VPN architectures. Multiple VRs can reside on a
PE device. VRs can be arranged in a hierarchical fashion within a
single PE as shown in Figure~\ref{fig:vpn-arch}. Moreover, an entity
acting as a service provider for an end customer might itself be a
customer of a larger service provider. VRs may also use different
backbones to improve reliability or to provide differentiated levels
of service to customers.  

\begin{figure}[ht]
\center{\includegraphics[width=\columnwidth]{../../proposals/NSF_CyberTrust_VU_UConn05/eps/VPN-VR-arch.eps}}
%\steal{0.2in}
\bfcaption{VPN Architecture using Virtual Routers}
\label{fig:vpn-arch}
\end{figure}

Customer edge (CE) devices wishing to join a VPN connect to a VR on
the PE device. A VR can multiplex several distinct CEs belonging to
the same VPN session. A VR may use tunneling mechanisms to use
multiple routing protocols and link layer protocols, such as IPSec,
GRE, and IP-in-IP, to connect with the CEs. A totally different set of
protocols and tunneling mechanisms could be used for inter-VR or
VR-backbone communication. These tunneling mechanisms can also be the
basis for differentiated levels of service as well as to provide
improved reliability. A VR also comprises firewall capabilities.

We consider a scenario where a VR is used to provide VPN services
to two organizations, with each organization having a customer edge 
(CE) router connected to the VR. The employees of each organization
issue VPN set up and tear down service requests to the VR via CEs. 
Also, the VR offers a differentiated level of service, with organization \#1
receiving prioritized service over organization \#2. From the 
point of view of the employees of the organizations, it 
is necessary that the service requests be handled in a 
reasonable amount of time. Also,
the probability of denying the service requests should be minimal.
From the point of view of the VPN service provider, the rate at
which the service requests are processed or the throughput is
important. The throughput will determine determine the revenue 
collected by the provider.

In order to implement the VPN service, a reactor pattern with
the characteristics described in Section~\ref{sec:reactor_modeling}  
can be used to demultiplex the events. The SRN model of the 
reactor pattern can thus be used for the performance
analysis of the VPN service provided by the VR. 
In order to use the SRN model, we designate the requests 
originating from organization \#1 as events of type \#1 and requests
originating from organization \#2 as events of type \#2. 

In the early stages of application development lifecycle, it is
necessary to analyze the impact of design  
choices and configurations on the performance metrics.  Also, in these
stages, it is rarely the case that the values of the
input parameters can be estimated with certainty, which makes
it imperative to analyze the sensitivity of
the performance metrics to the variations in the input parameters. 
Sensitivity analysis will enable the provider to determine the 
regions of operation during which service performance is acceptable.
In the subsequent subsections we demonstrate how the 
SRN model can be used to explore alternative
design configurations, as well as to facilitate sensitivity analysis.

\subsection{Impact of Buffer Space}
\label{sec:n1n2}
The buffer space available to hold the incoming events of each type
is a configuration parameter of the reactor pattern and
the VPN service that uses the pattern. 
This choice will have a direct impact 
on all the performance metrics. Most importantly, 
from the employees' perspective, the
buffer space will influence the probability of denying the service
requests.

We analyze the impact of the buffer capacities on the 
performance measures. The values of the 
remaining parameters (except for the buffer capacities) are reported 
in Table 3. We consider two values of buffer capacities 
$N_1$ and $N_2$. In the first case, the buffer capacity is 
set to $1$ for both types
of events, whereas in the second case the buffer 
capacity of both types of events is set to $5$. The performance metrics for 
both these cases are summarized in Table 4.
Because the values of the parameters of the service requests
from organization \#1 ($\lambda_1$, $\mu_1$ and $N_1$) are the same 
as the values of the parameters
for the service requests from organization \#2 ($\lambda_2$, 
$\mu_2$, and $N_2$), the
throughputs, queue lengths, and the loss probabilities are the same
for both types of service requests for each one of the buffer capacities 
as indicated in Table 4.
It can be observed that the loss probabilities are significantly higher 
when the buffer capacity is $1$ 
compared to the case when the buffer space is $5$. Also, due to the
higher loss probability, the throughput is slightly lower when
the buffer capacity is $1$ than when the buffer capacity is $5$. 

For both buffer capacities, the total number of service requests 
from organization \#2, denoted $E_2$, is slightly higher than the 
total number of requests from organization \#1, 
denoted $E_1$. Because the requests from organization \#1
are provided prioritized service over the requests from organization \#2,
on an average it takes longer to service a request from organization \#2 
than it takes to service a request from organization \#1. This results
in a higher total number of requests from organization \#2 than 
the total number of requests from organization \#1. This also results in a 
higher response time for service requests from organization \#2 
as compared with the service requests from organization \#1, as 
shown in Table 4.

\begin{table}[htb]
\begin{center}
\begin{tabular}{|c|c|c|} \hline 
Event type & Arrival rate & Service rate \\ \hline 
\#1 & $\lambda_1$ = $0.400$/sec. & $\mu_1$ = $2.000$/sec. \\ \hline 
\#2 & $\lambda_2$ = $0.400$/sec. & $\mu_2$ = $2.000$/sec. \\ \hline 
\multicolumn{3}{c}{$^3$ Parameter values}
\end{tabular}
\end{center}
\end{table}

\begin{table}[htb]
\begin{center}
\begin{tabular}{|c|c|c|} \hline 
Performance measure & \multicolumn{2}{|c|}{Buffer space} \\ 
                    & $N_1=1$, $N_2=1$ & $N_1=5$, $N_2 = 5$ \\ \hline 
$T_1$ & $0.37$/sec. & $0.39$/sec. \\ \hline 
$T_2$ & $0.37$/sec.  & $0.39$/sec. \\ \hline 
$Q_1$ & $0.06$ & $0.12$ \\ \hline 
$Q_2$ & $0.06$ & $0.12$ \\ \hline 
$E_1$ & $0.25$ & $0.31$ \\ \hline 
$E_2$ & $0.27$ & $0.34$ \\ \hline 
$L_1$ & $0.06$ & $0.00024$ \\ \hline 
$L_2$ & $0.06$ & $0.00024$ \\ \hline
$R_1$ & $0.63$ sec. & $0.83$ sec. \\ \hline 
$R_2$ & $1.10$ sec. & $1.33$ sec.  \\ \hline 
\multicolumn{3}{c}{$^4$Impact of buffer capacity on performance measures}
\end{tabular}
\end{center}
\end{table}
% Swapna, the table number was missing. I added the $4$ above, which I
% hope is the right thing.
% Okay, I just made a subscript out of it. 

\subsection{Impact of Service Rates}
\label{sec:mu}
Another design parameter that will impact the service
performance is the service time of the event handlers.
Obviously, a faster rate of service will improve performance,
but from the provider's perspective, increasing the service
rate may require higher levels of resource provisioning thereby increasing 
the cost. Therefore, it is necessary to determine what service rate would be
``good enough'' to provide acceptable service performance. This
can be accomplished by analyzing the sensitivity of
the performance metrics with respect to the service rates, namely, 
$\mu_1$ and $\mu_2$.

The results reported in Section~\ref{sec:n1n2} indicate that
the loss probability is significantly lower when the buffer space
is $5$ as compared to the loss probability when the buffer space is 
$1$. As a result, for the purpose of sensitivity analysis 
with respect to $\mu_1$ and $\mu_2$, we 
set the buffer capacity for both service requests to $5$. 
We vary the service rates $\mu_1$ and $\mu_2$
in the range of $0.4$/sec. to $2.0$/sec. roughly in steps of $0.025$, one 
at a time, to obtain the expected performance
measures by solving the SRN model shown in Figure~\ref{fig:srn}. 
The remaining parameters are held at the values reported in 
Table 3.

The plots in the left column of Figure~\ref{fig:sens_mu} 
show the variation of the performance measures with
respect to $\mu_1$, while the plots in the right column of 
Figure~\ref{fig:sens_mu} show the variation with respect
to $\mu_2$. The topmost plot in the left column shows the 
variation of throughputs of service requests from organizations
\#1 and \#2 with respect to $\mu_1$. The figure 
indicates that as $\mu_1$ decreases, 
the throughputs of requests from both organizations decrease; however,
the magnitude of decrease in the throughput of requests from
organization \#1 is higher than the magnitude of decrease in the throughput
of the requests from organization \#2. The second and the fourth 
plots in the same column 
show the variation of the total number of requests and the queue length 
for requests from both organizations as a function of $\mu_1$.
It can be observed that both the total number of requests and the queue 
length increase as $\mu_1$ decreases, and the magnitude of 
increase for service requests from organization \#1 is higher 
than the magnitude of increase for requests from organization \#2.
Similar trends can be observed in
the variation of the loss probabilities and response times 
in the third and the fifth plots.
  
\begin{figure*}[!hp]
\centerline{\hbox{
\psfig{file=eps/tputmu1.ps,height=2.5in,angle=270}
\hspace*{0.0in}
\psfig{file=eps/tputmu2.ps,height=2.5in,angle=270}
}}
\centerline{\hbox{
\psfig{file=eps/totmu1.ps,height=2.5in,angle=270}
\hspace*{0.0in}
\psfig{file=eps/totmu2.ps,height=2.5in,angle=270}
}}
\centerline{\hbox{
\psfig{file=eps/lossmu1.ps,height=2.5in,angle=270}
\hspace*{0.0in}
\psfig{file=eps/lossmu2.ps,height=2.5in,angle=270}
}}
\centerline{\hbox{
\psfig{file=eps/quemu1.ps,height=2.5in,angle=270}
\hspace*{0.0in}
\psfig{file=eps/quemu2.ps,height=2.5in,angle=270}
}}
\centerline{\hbox{
\psfig{file=eps/respmu1.ps,height=2.5in,angle=270}
\hspace*{0.0in}
\psfig{file=eps/respmu2.ps,height=2.5in,angle=270}
}}
\caption{Sensitivity of performance metrics to service rates $\mu_1$ and $\mu_2$}
\label{fig:sens_mu}
\end{figure*}

It can be noted that as $\mu_1$ decreases, the 
performance measures degrade (loss probability, queue
length, total number of events and response time increases,
and throughput decreases). Also, as $\mu_1$ decreases below 
$0.8$/sec., the performance metrics deteriorate rapidly.  
With decreasing $\mu_1$, the time taken
to service a request from organization \#1 increases due to the 
additional time taken to complete each snapshot. As a result, 
indirectly, this also increases the time taken to service a 
request from organization \#2 in each snapshot. Thus, the performance 
of requests from both organizations are adversely impacted,
even though only the service time for the requests 
from only one of the organizations increases.  
The variation of the performance metrics with respect to
$\mu_2$ shows the same trend, with the role of event types 
reversed. 

\subsection{Impact of Request Arrival Rates}
\label{sec:lam}
The arrival rates of service requests, namely, $\lambda_1$ and $\lambda_2$,
constitute the input parameters. In this section we analyze the 
the sensitivity of the performance metrics to these input
parameters.

Based on the results and analysis described in Section~\ref{sec:n1n2}
and Section~\ref{sec:mu}, we set $N_1$ and $N_2$ to $5$, and
we set $\mu_1$ and $\mu_2$ to $2.00$/sec for sensitivity analysis
with respect to $\lambda_1$ and $\lambda_2$.
The parameters $\lambda_1$ and $\lambda_2$
were varied one at a time in the range of $0.4$/sec to 
$2.0$/sec. roughly in steps of $0.025$ to obtain the 
expected performance metrics
by solving the SRN model shown in Figure~\ref{fig:srn}. 

\begin{figure*}[!hp]
\centerline{\hbox{
\psfig{file=../Sigmetrics-2005/eps/tputlam1.ps,height=2.5in,angle=270}
\hspace*{0.0in}
\psfig{file=../Sigmetrics-2005/eps/tputlam2.ps,height=2.5in,angle=270}
}}
\centerline{\hbox{
\psfig{file=../Sigmetrics-2005/eps/totlam1.ps,height=2.5in,angle=270}
\hspace*{0.0in}
\psfig{file=../Sigmetrics-2005/eps/totlam2.ps,height=2.5in,angle=270}
}}
\centerline{\hbox{
\psfig{file=../Sigmetrics-2005/eps/losslam1.ps,height=2.5in,angle=270}
\hspace*{0.0in}
\psfig{file=../Sigmetrics-2005/eps/losslam2.ps,height=2.5in,angle=270}
}}
\centerline{\hbox{
\psfig{file=../Sigmetrics-2005/eps/quelam1.ps,height=2.5in,angle=270}
\hspace*{0.0in}
\psfig{file=../Sigmetrics-2005/eps/quelam2.ps,height=2.5in,angle=270}
}}
\centerline{\hbox{
\psfig{file=../Sigmetrics-2005/eps/resplam1.ps,height=2.5in,angle=270}
\hspace*{0.0in}
\psfig{file=../Sigmetrics-2005/eps/resplam2.ps,height=2.5in,angle=270}
}}
\caption{Sensitivity of performance metrics to arrival rates $\lambda_1$ and $\lambda_2$}
\label{fig:sens_lam}
\end{figure*}

Figure~\ref{fig:sens_lam} shows the performance measures as a
function of $\lambda_1$ and $\lambda_2$. The plots in the left
column show the variation of the performance metrics with
respect to $\lambda_1$, whereas the plots in the right column
show the variation of the performance metrics with respect
to $\lambda_2$. Referring to the topmost figure in the left 
column, it can be observed that initially, 
the throughput of service requests from
organization \#1 is nearly the same as the
arrival rate of the requests, indicating that the requests are serviced 
at the same rate at which they arrive. However, as 
$\lambda_1$ increases, the throughput starts lagging the arrival rate,
which indicates that the service rate is not sufficiently
high to process the requests at the rate at which they arrive. This 
may cause the queue for requests from organization \#1 to operate at 
full capacity for an extended period of time, which results in a 
rejection of the incoming requests from organization \#1.

It can be observed that a decrease in the 
rate at which the throughput increases is accompanied by an increase in
the loss probability of the requests (as shown in the third plot in 
the left column) and an increase in 
the queue length (as shown in the fourth plot in the left column) 
in Figure~\ref{fig:sens_lam}. The left plot in the second
row represents the total number of service requests
from each organization as a function
of $\lambda_1$. The plot indicates that the total number of 
requests from organization \#2
in the system increases as $\lambda_1$ increases. When $\lambda_1$
increases, the probability of having to service a request
from organization \#1 prior to servicing a request from 
organization \#2 in a given snapshot increases. Because requests
from organization \#1 have priority over requests from organization \#2, 
requests from organization \#2 tend to reside longer in the
system as $\lambda_1$ increases. Thus, although the throughput of 
requests from organization \#2 is unchanged with respect to $\lambda_1$, the 
response time of the requests from organization \#2 increases 
as shown in the fifth plot in the figure. At approximately $\lambda_1=1.0$, 
the expected response times for both types of events is close. 
However, as $\lambda_1$ increases beyond $1.0$/sec,
the expected response time of type \#1 events is higher than the 
expected response time of type \#2 events. Thus, effectively,
requests from organization 
\#2 are receiving better service than requests from organization \#1. 

The plots in the right column of Figure~\ref{fig:sens_lam} indicate
that similar trends can be observed for all the performance 
measures except response time with respect to $\lambda_2$, 
except that the roles of the two types of 
requests are reversed. The response time, however, follows a 
different trend. The response time of both the
event types increases, however, the response time for the events of
type \#1 is always lower than the response time for the events of
type \#2, for the entire range of variation of $\lambda_2$. 

\section{Conclusions and future research}
\label{sec:concl}
In this paper we presented a performance model of the reactor pattern
which offers the important synchronous demultiplexing and dispatching
capabilities in middleware. The model was based on the 
Stochastic Reward Net (SRN) modeling paradigm. We 
illustrated how the performance model is  
used to obtain estimates of the performance metrics of a VPN service 
provided by a Virtual Router (VR). Our future research 
consists of empirically validating 
the performance estimates obtained from the performance model. Developing
and validating the performance models of other middleware building blocks
and the composition of these building blocks is also a topic of future
research. 

\bibliographystyle{elsart-harv}
{\small\ls{0.9}
\raggedright
\bibliography{networks,analysis,perf-id}
}

\end{document}
