How To Run PolyMix-3

1: Install FreeBSD

We recomend running Polygraph on FreeBSD. You can use another Unix operating system if you like.

Recommended minimum hardware:

Pentium III/500 MHz
256 MB RAM
4 GB IDE DISK
100baseTX Intel Etherexpress Pro NIC

To install FreeBSD, please see our Setting Up FreeBSD page.

2: Download and Compile Necessary Software

2.1: Install Polygraph

Get Polygraph version 2.5.4 from http://polygraph.ircache.net/sources/. Unpack and install it:

% cd /tmp
% wget http://polygraph.ircache.net/sources/tars/polygraph-2.5.4.tar.gz
% tar xzvf polygraph-2.5.4.tar.gz
% cd polygraph-2.5.4
% ./configure --prefix=/usr/local/polygraph-2.5.4
% make all
% sudo make install

Add /usr/local/polygraph-2.5.4/bin to your PATH, or perhaps make symbolic links in the /usr/local/bin directory.

Later versions of Polygraph may be used. If you are trying to reproduce others results, use the same version that others have used. Not all Polygraph versions are backward-compatable.

2.2: Install netperf

Get and install netperf from www.netperf.org or from our FTP site.

NOTE: netperf-2.1pl3 does not compile out-of-the-box on FreeBSD. Before running 'make' you need to edit makefile and add __FREEBSD__ to line 86:

CFLAGS = -O -D$(LOG_FILE) -DUSE_LOOPER -D__FREEBSD__

2.3: Install gnuplot

To use the automatic report generation programs, you'll need to install gnuplot with PNG support.

You can get gnuplot from ftp.gnuplot.vt.edu. You'll also need libpng-1.0.6.tar.gz and zlib-1.1.3.tar.gz, which you can find in the same FTP directory.

Installing gnuplot looks like this:

% ftp ftp.gnuplot.vt.edu
ftp> cd pub/gnuplot
ftp> get gnuplot-3.7.1.tar.gz
ftp> get libpng-1.0.6.tar.gz
ftp> get zlib-1.1.3.tar.gz
ftp> bye

% tar xzf zlib-1.1.3.tar.gz
% cd zlib-1.1.3
% ./configure && make
% sudo make install

% tar xzf libpng-1.0.6.tar.gz 
% cd libpng-1.0.6
% ln -s scripts/makefile.std Makefile
% make
% sudo make install

% tar xzf gnuplot-3.7.1.tar.gz 
% cd gnuplot-3.7.1
% ./configure --with-png
% make
% sudo make install

3: Set up the PolyMix-3 workload file.

When editing and understanding PolyMix-3 workload files, note that all PolyMix-3 input parameters are set as totals perceivied by the cache(s) under the test. If a device under test is comprised of several caching units, treat it a single ``big'' cache for the Polygraph configuration purposes.

You should use the exact same configuration files for all polyclt and polysrv processes. No manual adjustments for the number of polyclt processes is needed; all adjustments are done automatically in polymix-3-guts.pg file which is included from the polymix-3.pg file.

When in doubt or puzzled by a contradicting or insuffient documentation, do not try to guess the right setting; double check with us instead.

3.1: Edit workload files

Copy polymix-3.pg from the Polygraph source distribution ``workloads'' directory into a new working directory.

Edit polymix-3.pg and define the following variables:

ProxyCacheSize -- This is the total configured disk cache size plus the amount of RAM installed on the machine running a cache. For example, if your cache disk size is 50GB and you have 4GB of RAM, then you can write:
```
size ProxyCacheSize = 50GB + 4GB;
```
or just
```
size ProxyCacheSize = 54GB;
```
TheBench.peak_req_rate -- This is the offered request rate during the two ``peak'' phases. For example, if your caching proxy supports 1000 requests per second, you write:
```
TheBench.peak_req_rate = 1000/sec;
```
Also note that this peak rate value is used to deteremine which IP addresses to use for robot and server agents.
FillRate -- The request rate during the fill phase. It must be between 10% and 100% of your peak request rate. For example:
```
rate FillRate = 75% * TheBench.peak_req_rate;
```

3.2: Select Subnet

Select a subnet to use. Here we will use "10.X.0.0" and you can replace X with whatever you like. Your caching proxy's IP address should be 10.X.0.1. If you have more than one proxy (i.e., a cluster) you can use 10.X.0.2, etc.

We use the "10.X.0.100" address for the monitoring station so you may want to leave that address unused when prepareing for the cache-off.

In a routed network configuration, clients and servers will be on different subnets.

Tell Polygraph which subnet(s) to use by setting the TheBench.client_addr_mask and TheBench.server_addr_mask in polymix-3.pg. For example:

TheBench.client_addr_mask = '10.16.0.0';
TheBench.server_addr_mask = '10.16.0.0:80';

3.3: Configure and Install IP alias addresses

As of version 2.5.1, polygraph can internally calculate which IP addresses to use for robot and server agents. However, it can not yet automatically create the IP alias addresses.

You've told polygraph which subnet to use. Polygraph figures out which IP addresses to use based on the subnet and the TheBench.peak_req_rate value. For now, you have to manually install the aliases with the aka command. But first, you'll want to run pmix3-ips.pl to determine which address to install on which machines. The pmix3-ips.pl can be found in the ``polygraph-2.5.4/tools/'' directory.

When you run pmix3-ips.pl, it prints out the breakdown of addresses for a given subnet. For example:

% perl pmix3-ips.pl 16 1000
        bench:        16
        req.rate:   1000/sec (actual:  1003.20/sec)
        PCs:           3pairs
        robots:     2508 ( 836/machine)
        servers:     756 ( 252/machine)

        rbt_ips:      10.16.1-12.1-209
            1st machine:      10.16.1-4.1-209
            2nd machine:      10.16.5-8.1-209
            3rd machine:     10.16.9-12.1-209
        srv_ips:   10.16.129-134.1-126
            1st machine:  10.16.129-130.1-126
            2nd machine:  10.16.131-132.1-126
            3rd machine:  10.16.133-134.1-126

Now use the aka program to install the alias addresses on each machine. Continuing with the above example, on the first client machine you would run:

# aka fxp0 10.16.1-4.1-209

This must be done for each polygraph client and server machine, with the appropriate address range.

Check your work! Run

ifconfig fxp0

and you should see lots of IP addresses listed.

3.5: Configure Dummynet

On all polygraph clients, run:

# ipfw -f flush

On all polygraph servers, run

# ipfw -f flush
# ipfw pipe 1 config delay 40ms plr 0.0005
# ipfw pipe 2 config delay 40ms plr 0.0005 
# ipfw add pipe 1 ip from any to 10.X.0.0/16 in
# ipfw add pipe 2 ip from 10.X.0.0/16 to any out

Check your work! Ping a client from a server and you should see round trip times of about 80 msec.

4: Test Your Network

4.1: Make sure clients and servers can ping each other

% ping 10.X.1.1
% ping 10.X.129.1
...

4.2: Start the 'netserver' daemon on every polygraph machine

% /usr/local/bin/netserver

4.3: Run netperf between clients and servers.

For example:

# netperf -l 30 -H 10.X.1.1 -t TCP_STREAM

You should make sure that a client-server pair runs netperf in both directions at the same time. This guarantees that your network is operating well in full-duplex mode. If everything is good, netperf reports a throughput of about 80 MBit/s.

For a unidirectional netperf test, you should get about 92-95 MBit/s.

For longer tests, increase the -l <length> value.

5: Run a no-proxy test

The polygraph clients and servers should be able to sustain your peak request rate without a proxy cache involved. The proxy cache must not be connected to the network during this test.

On each polygraph client you would run:

% polyclt --config polymix-3.pg --verb_lvl 10 --ports 3000:30000

Similarly on the servers:

% polysrv --config polymix-3.pg --verb_lvl 10 --idle_tout 5min

You may want or need additional polyclt/polysrv options. For example, the location of the ``workloads/include'' directory (or its copy) needs to be specified using the ``--cfg_dirs'' option; logging may be enabled using the ``--log'' option; etc.

We recommend running the no-proxy test for 30-60 minutes at peak load. To create your custom no-proxy workload, follow these steps:

Run

% cd polygraph-2.5.4/workloads/
% cat polymix-3.pg include/polymix-3-guts.pg > /tmp/my.pg

Edit the resulting my.pg file to delete the ``#include "polymix-3-guts.pg"'' line.
Edit and use the my.pg file to suit your needs.

The platDur variable can be adjusted to make the test shorter. The phFRamp, phFill phases can be commented out so you don't have to wait very long to get to the peak rate.

The difference in response time among phases should be marginal in a no-proxy test. Response times should be about 2.8 seconds. If reply rate and response time look good for at least 30 minutes of peak load, you can stop the no-proxy test. If response time looks bad, re-examine your network setup or workload config.

6: Prepare your proxy for testing

6.1: Ping the proxy

Make sure your proxy has an address on the subnet and that all clients and servers can ping it.

6.2: Run msl_test

From a polygraph client or server machine, run the msl_test program against your proxy. This program uses some low-level IP packets to determine the MSL setting for your TCP stack.

Sample usage is:

# ./msl_test -i fxp0 -s 10.X.1.1 -d 10.X.0.1 -p 8080

The final argument (port number) should be the port number where your proxy accepts requests. It can not be any random port.

During this test, you will not be able to send any other traffic from the source machine to the proxy.

When finished, the program reports the TIME_WAIT value that it found. This value is twice the MSL value. Cache-off rules require the TIME_WAIT value to be 60 seconds. If the msl_test program reports a number smaller than 60 seconds, you may be in violation of the rules. Violators will be disqualified.

For more information, read msl_test.html.

7: Test your Proxy

7.1: Copy all ``.pg'' files to every client and server

7.2: Start polysrv processes

% polysrv --config polymix-3.pg --verb_lvl 10 --log srv.log

7.3:Start polyclt processes

% polyclt --config polymix-3.pg --verb_lvl 10 --log clt.log

NOTE: you may want to use additional or different command line parameters. For example you may want to save the polygraph stdout/stderr to a file for later reference.

If you need to start polygraph on many machines, you may want to use the "bb.pl" script from the polygraph source distribution.

7.4: Wait

We usually monitor experiments using the 'polymon' program. In order to use 'polymon' you must must use the --notify option to polyclt and polysrv.

8: Run the downtime test

Setup a single polygraph client machine and a single polygraph server machine to use the downtime-2.pg workload file (found in the workloads directory of the source distribution).

During the run, turn off the power to all equipment in the "participant zone", which includes your proxy and your networking gear. Start a stopwatch or timer.

Wait five seconds.

Return power to proxy and networking gear.

Watch the polyclt console output. Note when the first cache miss is successfully received by the client.

Also note when the first cache hit is successfully recieved.

9: Analyze the results

Polygraph includes a set of scripts, called ReportGen that you can use to display the results.

9.1: Copy all log files to a single location

9.2: Label the logfiles

Use label_results to label logfiles with a single name.

% perl label_results mytest1 clt.*.log

NOTE: With Polygraph 2.5.3 and earlier, reportgen.cfg will need to be adjusted if non-default configure --prefix is used.

9.3: Generate a report

Use make_report to make graphs and an HTML page describing the results:

% perl make_report mytest1

9.4: View the results

Use netscape or another browser to view the report. You may need to copy the files to an HTTP server.

netscape /tmp/polyrep/mytest1/index.html

$Id: index.html,v 1.14 2001/05/25 22:54:44 wessels Exp $