%
% x-kernel v3.3
%
% Copyright (c) 1996,1993,1991,1990  Arizona Board of Regents
%

\section{Another Example {\var xsim.data} File}\label{more_examples}

This section presents a more complex {\var xsim.data} file that is
included with the distribution.  The purpose of this section is to
explain this file, which can be used to test protocols in a realistic
internetwork environment, as well as to give a flavor of the complex
internetworks that can be simulated using {\xs}. One word of warning:
for a simulation of this complexity, it takes about one minute of real
time to run 1 second of simulated time on a 200MHz workstation.

The {\var xsim.data} file on which this section is based is located
in

\begin{verbatim}
     /xkernel/simulator/build/Template/example/xsim.data.ANS
\end{verbatim}

\noindent where {\var /xkernel} is the root of the {\xk} source tree.
This canned configuration file represents a subset of the ANSnet
backbone, a 45Mbps nationwide public data network.\footnote{More
information on ANSnet can be found at: http://www.ans.net/}

\begin{figure}[ht]
\centering
\leavevmode\hbox{\epsfig{file=ans.eps,height=3.0in}}
\caption{Subset of the ANS backbone}\label{ans}
\end{figure}

Figure~\ref{ans} shows a diagram of the internetwork specified in the
file.  Four routers, marked by circles in the figure, correspond to
routers located in Los Angeles, San Francisco, Chicago and Cleveland.
The boxes represent the supercomputer centers in San Diego and
Pittsburgh, and the Argonne National Labs site near Chicago; large
``bulk data'' transfers take place between these three sites.  The 16
nodes marked {\bf TS} are traffic sources, each consisting of two or
three hosts.  The resulting traffic consists of nearly a thousand new
connections being established per simulated second, where each
connection is either a TELNET, FTP, SMTP or NNTP conversation.  The
bulk data transfers between supercomputer centers must compete for
network resources with all of these traffic conversations.

The {\var xsim.data} file specifying this network is large, but easy
to understand. This section gives only snippets of that file; see the
actual file for the complete story. There are only a few types of
components being configured: 45Mbps and 100Mbps links, routers, bulk
transfer hosts, and traffic hosts.  Each type of component resembles
others of its type; in other words, all 100Mbps links look almost the
same.  The remainder of this section gives examples showing how an
individual component of each type is specified in the {\var xsim.data}
file.

\subsection{Networks}

Figure~\ref{ans} shows 100Mbps links connecting the traffic sources to
the routers, and 45Mbps links connecting the routers to each other and
to the supercomputer sites.  The 45Mbps links running between routers
are configured as point-to-point links; the 45Mbps links connecting
the supercomputing centers to the routers are parameterized Ethernets.
We call these networks ``parameterized'' Ethernets because the
parmeters we specify in the {\var xsim.data} file change the
characteristics of these networks from those of the default 10Mbps
Ethernet.

The time associated with each link in the figure represent latency
times for packets traveling across the link; this latency is
proportional to distance.  From San Francisco, it is much farther to
Chicago than to Los Angeles, so the latency to Chicago (28ms) is four
times as great as that to L.A. (7ms).

We now give an example specification for each kind of link, as it
appears in {\var xsim.data}.  The entries specifying networks are
found at the end of the file.  First, the 45Mbps point-to-point link
from San Francisco to Chicago:

\begin{verbatim}
    set prate = 45Mb;
    set pdelay1 = 28ms;

    net PP 192.0.51.0;
      connections = SF 0.0.0.1, CH 0.0.0.2;
      args = rate=$prate, delay=$pdelay1;
\end{verbatim}

This entry is straighforward.  The first line identifies it as a
point-to-point link with network address 192.0.51.0; the two routers
SF and CH attach at addresses 192.0.51.1 and 192.0.51.2, respectively.
The bandwidth and latency of the link are set by the {\var rate} and
{\var delay} parameters.

Next, we show how the parameterized Ethernets that connect the
supercomputer centers to the routers are configured.  This one
connects the router in Cleveland to the Pittsburgh Supercomputing
Center:

\begin{verbatim}
    set erateBig = 45Mbps;
    set edelay42 = 780us;
    set emaxpkt = 1500;
    set ehdrsize = 12;
    set coll = off;

    net ETH 192.0.42.0;
      connections = CL 0.0.0.100, h1n42 ethd 0.0.0.1;
      args = rate=$erateBig, delay=$edelay42, maxPacket=$emaxpkt, hdrSize=$ehdrs
             collisions=$coll;
\end{verbatim}

This network is identified as an Ethernet, with net address
192.0.42.0.  We change its bandwidth from the default 10Mbps to
45Mbps, and specify a latency of 780 microseconds.  The CL router
connects at address 192.0.42.100; host {\var h1n42}, a host at the PSC
that will engage in bulk transfers, has address 192.0.42.1.

Finally, each of the traffic sources is connected to a router by a
100Mbps parameterized Ethernet.  Here is an entry for a network
that connects a traffic source consisting of three hosts to the router
in L.A.:

\begin{verbatim}
    set erate = 100Mbps;
    set edelay1 = 1120us;
    set emaxpkt = 1500;
    set ehdrsize = 12;
    set coll = off;

    net ETH 192.0.1.0;
      connections = LA 0.0.0.100, h1n1 ethd 0.0.0.1, h2n1 ethd 0.0.0.2, 
                h3n1 ethd 0.0.0.3;
      args = rate=$erate, delay=$edelay1, maxPacket=$emaxpkt, hdrSize=$ehdrsize, 
             collisions=$coll;
\end{verbatim}

This entry is almost the same as the one given above for the 45Mbps
parameterized Ethernet.  The three hosts {\var h1n1}, {\var h2n1}, and
{\var h3n1} will generate traffic connections during the simulation;
together they make up a single ``traffic source'' in Figure~\ref{ans}.

\subsection{Routers}

There are four routers shown in Figure~\ref{ans}.  Below is the entry
that defines the router at San Francisco:

\begin{verbatim}
    set rbufs = 60;

    router FCFS SF;
      rtable = 192.0.5.0 192.0.5.0, 192.0.6.0 192.0.6.0, 192.0.7.0 192.0.7.0, 
               192.0.8.0 192.0.8.0, 192.0.1.0 192.0.50.1, 192.0.2.0 192.0.50.1, 
               192.0.3.0 192.0.50.1, 192.0.4.0 192.0.50.1, 192.0.40.0 192.0.50.1, 
               0.0.0.0 192.0.51.2;
      args = buf=$rbufs, timeInc=10000, shortTrace=20000:4,
             traceQueue=192.0.50.2, traceInOut=192.0.50.2, traceOut=192.0.50.2, 
             traceQueueLen=192.0.50.2, traceQueue=192.0.51.1, traceInOut=192.0.51.1,
             traceOut=192.0.51.1, traceQueueLen=192.0.51.1;
\end{verbatim}

The routing table given on the {\var rtable} line is large but simple.
It specifies that

\begin{itemize}
\item Packets destined for hosts on networks 192.0.5.0, 192.0.6.0,
192.0.7.0, and 192.0.8.0 are sent to the appropriate host.  The router
is directly connected to these networks.

\item Packets destined for networks 192.0.1.0, 192.0.2.0,
192.0.3.0, and 192.0.4.0 are sent out on network 192.0.50.0 to address
192.0.50.1 (this is the L.A. router).

\item All other packets are sent out on network 192.0.51.0 to
address 192.0.51.2 (this is the Chicago router).

\end{itemize}

\noindent
The first arguments listed configure the router with 60 buffers; the
{\var qmin} and {\var qmax} arguments have no effect at the present
time, but will be included in a future release.  The rest of the
arguments are trace options.  They specify that the router should
trace activity on the network interfaces with addresses 192.0.50.2 and
192.0.51.1; packets sent out on these interfaces are destined for the
routers in L.A. and Chicago.  Information that will be traced for each
interface includes the minimum, maximum, and average send queue size
in packets and bytes; the average number of bytes/second forwarded to
the interface; and the average number of bytes/second sent out on the
interface.  Also, between seconds 5 and 8 of the simulation, trace
information for each packet sent on interface 192.0.51.1 will be
saved.

All of the other routers resemble this one.  All routers trace queue
size and average bytes/second for the interfaces connected to the
point-to-point links; the packets sent on these interfaces are
destined for another router.  However, only the SF router uses the
{\var pktTrace} option to trace individual packets.

\subsection{Hosts}

There are three types of hosts in the internetwork: hosts initiating a
bulk transfer, hosts receiving a bulk transfer, and hosts generating
traffic connections.

The hosts engaged in bulk transfers are at the supercomputer sites.
Below, host {\var h1n40} is at the SDSC and will initiate a transfer;
host {\var h1n42} is at the PSC and will receive the data sent by host
{\var h1n40}.

\begin{verbatim}
    set ip = ip, ip vnet, vnet eth arp, arp eth, eth ethd, ethd sim;
    set tcpMegS = vtcp;
    set tcpMegR = btcp;
    set buf1 = 400;
    set len1 = 1024;
    set tcpTrace1 = -tcpTrace=350000:5;
    set thresh1 = 200;
    set ed = 0;
    set delay1 = 10s;
    set megTime1 = 12s;
    set db = db/junk -tcpTimerAuto ;
    set dbStr1 = -dbStr=rbufs:$rbufs;
    set dbStr2 = -dbStr=group:$group;
    set dbStr3 = -dbStr=qmin:$qmin;
    set dbStr4 = -dbStr=qmax:$qmax;

    host h1n40;
      protocols = megtest $tcpMegS, $tcpMegS $ip;
      args = -c192.0.42.1 -buf=$buf1 -len=$len1 $tcpTrace1 -thresh=$thresh1 
             -megTime=$megTime1 -delay=$delay1 -dbStr=delay:$delay1
             -db=$db -timeInc=10000 -ethdTraceOut -ethdTraceIn $dbStr1 $dbStr2 
             $dbStr3 $dbStr4;
 
    host h1n42;
      protocols = megtest $tcpMegR, $tcpMegR $ip;
      args = -s -len=$len1 -buf=$buf1 -timeInc=10000 -ethdTraceIn 
             -ethdTraceOut;
\end{verbatim}

The protocol stacks for both hosts are straightforward; note, however,
that {\var h1n40} will use TCP Vegas as its transport protocol, while
{\var h1n42} will run Big Window TCP.

Let's look at the more significant arguments for the hosts in detail.
For explanations of arguments not discussed, see Section~\ref{hostApp}.

\begin{itemize}

\item The {\var -c192.0.42.1} parameter indicates that {\var h1n40} will
initiate the bulk transfer to the specified address.  If you look at
the entry for network 192.0.42.0, you will see that this is the
address of {\var h1n42}, which is given the {\var -s} argument.

\item {\var -buf} sets the maximum size of the TCP send buffer to 400KB
for both hosts. 

\item {\var -len} specifies that 1024KB of data should be transferred.

\item {\var -thresh} sets the initial threshold for TCP Slow Start at
200KB.  

\item {\var -delay} specifies that the host should wait 10 seconds
before starting the transfer, and {\var megTime} says that the transfer
should be cut off at 12 seconds.  Thus, the transfer is given 2
seconds to run.

\item {\var -ethdTraceOut} and {\var -ethdTraceIn} trace the average
number of bytes sent and received, respectively, by the host.

\item {\var -tcpTrace} will trace significant TCP events, such as the
transmission of a segment, the size of the send window, and the firing
of a timeout.

\end{itemize}

The other kind of host on our internetwork generates traffic
connections.  Each of these hosts is both a client and a server.

\begin{verbatim}
    set ip = ip, ip vnet, vnet eth arp, arp eth, eth ethd, ethd sim;
    set tcpTraff = btcp;
    set begcit = 25;
    set begdur = 7 -begint=2;
    set cit2 = 90;
    set db = db/junk -tcpTimerAuto ;
    set ed = 0;
    set time = 15;
    set ethdTrace = ;
    set dbStr1 = -dbStr=rbufs:$rbufs;
    set dbStr2 = -dbStr=group:$group;
    set dbStr3 = -dbStr=qmin:$qmin;

    host h1n1;
      protocols = traffic $tcpTraff, $tcpTraff $ip;
      args = -hosts=3 192.0.5.1 192.0.9.1 192.0.13.1 -s1 -begcit=$begcit 
             -begdur=$begdur -begkeep=200 -cit=$cit2 -db=$db -trafficTime=$time 
             -ethdDelay=$ed $ethdTrace $dbStr1 $dbStr2 $dbStr3 -reset=10;
\end{verbatim}

There is nothing surprising about the protocol stack on this host.
Big Window TCP is used as the transport protocol.  Again, let's
examine the arguments.

\begin{itemize}

\item The {\var -hosts=3 192.0.5.1 192.0.9.1 192.0.13.1} argument
specifies that this traffic host will act as a client, and open
traffic connections with the three hosts specified.  Note that host
192.0.5.1 is near San Francisco, 192.0.9.1 is near Chicago, and
192.0.13.1 is near Cleveland; thus, packets sent to these hosts will
have to traverse the point-to-point links between routers.  The {\var
-s1} parameter tells the host to accept connections initiated by other
hosts.

\item The {\var -begcit}, {\var -begdur}, {\var -begint}, and {\var -cit}
arguments all have to do with how often $tcplib$ opens connections
during the simulation.  See section \ref{hostApp} for details.

\item {\var -trafficTime} specifies to stop the traffic host after it has
run for 15 seconds.

\item {\var -reset} says to reset the trace information after 10
seconds.  The bulk transfers start running then, and that's when the
traces will get interesting.

\end{itemize}