%\subsection{Screening Experiment Design}

\input{design.tbl}

\iffalse
We used a screening experiment because of their run sequence
efficiency. Screening experiments reduce the run sequence (i.e., number
of configurations to be tested) by aliasing the effect of lower-order
terms with higher-order ones. Since our main concern is the first-order
effects, we had the following concerns in mind when we were designing
the experiment: (1) Aliasing structure of the main effects and (2)
The run sequence efficiency of the design.
\fi

In this step, our main concern is to find the first-order effects
of configuration options; we are not interested in higher-order
(interaction) effects.  We designed the screening experiment using a
fractional factorial design as explained below:

%\begin{smallitemize}

\iffalse
\item We first created the configuration space given in
Table~\ref{options}. This configuration space contains 14 run-time options
each of which having two levels of setting. We chose these options in
particular because they all can potentially effect the performance of
the benchmarking code.
\fi

Based on the information we obtained from the ACE+\-TAO+\-CIAO
developers on the $2^{14}$ configuration space, we decided to alias no
main factors with effects involving fewer than three factors. This
decision dictated a resolution IV design, in which no effect involving
$i$ factors is aliased with effects involving less than $4-i$
factors. For example, in such a design some of the main effects are
aliased with three-factor interactions, but not with other main
effects or two-factor interactions.
% and some two-factor interactions are aliased
%with other two-factor interactions.

With resolution IV and run sequence efficiency requirements in mind,
we designed a $2_{IV}^{14-9}$ screening experiment given in
Table~\ref{design}.
%In this table (-) and (+) signs refers to the first
%and second setting of the corresponding option, respectively. 
This screening design examines 14 factors in $2^5=32$ runs, which is a
$2^{9}$ fraction of the exhaustive design.
 
To create the final design, we started with a $2^5$ all options design
(shown in the first 5 columns in Table~\ref{design}.  We computed the
remaining 9 columns by using the following design generators: o6=123,
o7=124, o8=134, o9=234, o10=125, o11=135, o12=235, o13=145, o14=245.
\iffalse
Each of these design generators is used to create an
additional column for a particular option. Take the design generator
o6=123 as an example. Using this design generator, 6th column (for o6)
is generated by multiplying the column 1, 2 and 3; that is, each entry
in the column for o6 is the same as the product of the corresponding
entries in the columns for o1, o2, and o3. This means that we alias
the effect of o6 with the effect of third-order interaction between
o1, o2, and o3. If this interaction effect is not significant, the
effect of o6 can reliably be predicted.
\fi
%\end{smallitemize}

