Setting up FreeBSD-3.4 for use with Polygraph

This page describes how to install and setup a FreeBSD system that you can use for testing with Polygraph. We have some customized FreeBSD distributions with special tweaks applied. You can choose from one of three installation methods:

CDROM
FTP
rsync

CDROM

Download our ISO image file and burn it on to your own blank CD. If you don't have your own CDROM burner, we can make one for you and mail it to you.

Download FreeBSD-3.4.TMF.iso. You can also get it from our FTP site.

To request a CDROM via postal mail, write to info at this domain.

FTP

You can install FreeBSD over the Internet using our FTP server. This is a common way of installing FreeBSD. If you've never done it before, you should read Installing FreeBSD in the FreeBSD Handbook.

To start, you'll need two blank floppy disks. You also need to download the two floppy disk image files kern.flp and mfsroot.flp. After downloading, put these files on the floppies. If you're using Unix, you do it like this:

dd if=kern.flp of=/dev/fd0

If you're using DOS or Windows, then you need the fdimage program to run this command:

fdimage kern.flp a:

Do the same thing to put mfsroot.flp on the other floppy.

Insert the kern floppy in your computer and boot. When prompted, insert the mfsroot floppy and continue.

During the install process, you'll get a window titled ``Choose Installation Media''. From here, select FTP and enter the URL for our server:

ftp://ftp.measurement-factory.com/pub/FreeBSD/

NOTE: you can also use the standard FreeBSD installation floppy disks (i.e., the ones from the official site). If you use those, however, you need to go to the Options menu and set the release name to ``FreeBSD-3.4.TMF''.

install.cfg

If you want to automate a lot of the install procedure, you can write an install.cfg file. You can use ours as a template, but you will need to change some things. You will need to change the _ftpPath value. You will need to change netDev if you don't have the Intel fxp0 NIC. You will need to change occurances of wd0 if you don't have an IDE disk.

See the sysinstall man page for a description of the config file.

You need to put this file on the filesystem contained on the second boot disk. In order to do that you have to mount the floppy, uncompress the mfsroot.gz file, mount the mfsroot file as a filesystem, copy install.cfg, unmount mfroot, re-compress it, and put it back on the floppy. For example:

#!/bin/sh

mount -r /dev/fd0 /mnt
cp /mnt/mfsroot.gz /tmp
umount /mnt

gunzip /tmp/mfsroot.gz
vnconfig /dev/vn0c /tmp/mfsroot
mount /dev/vn0c /mnt

cp /tmp/install.cfg /mnt/stand/install.cfg
umount /mnt
vnconfig -u /dev/vn0c

gzip /tmp/mfsroot
mount /dev/fd0 /mnt
cp /tmp/mfsroot.gz /mnt

umount /mnt
sync

Note, the procedure requires a FreeBSD kernel with

pseudo-device  vn

RSYNC

You can use rsync to transfer the entire FreeBSD source tree to your machine. Then you can compile a new kernel and system binaries. This technique really only works if you already have a version of FreeBSD-3.x installed. You can use it to update your system, rather than installing everything from scratch again.

You'll need rsync installed on your system.

To sync, use these commands:

% su
# cd /usr
# rsync -a -v rsync.measurement-factory.com::FreeBSD-3.4.TMF-src src

If you are upgrading from an earlier FreeBSD version (such as 3.3), then its a good idea to recompile all the system binaries, libraries, etc, with this command:

# cd /usr/src
# make world

For more information about make world, please see Using make world in the FreeBSD Handbook.

When that finishes, you'll need to build a new kernel...

Kernel Config File

If you're not comfortable creating your own config file, you can use this sample Save it to /sys/i386/conf/CACHEOFF3, then run these commands:

% su
# cd /sys/i386/conf
# config CACHEOFF3
# cd ../../compile/CACHEOFF3
# make depend
# make
# make install
# reboot

You will probably want to recompile polygraph after your system reboots so that it uses the right number of file descriptors.

If you like to make your own kernel config file, you really need the following options in your kernel:

HZ=1000
This option provides better granularity for dummynet delays.
DUMMYNET
Support for packet queueing and loss simulation.
IPFIREWALL
Needed for dummynet
NMBCLUSTERS=32768
The default value is too small, this value is perhaps larger than necessary. Running out of mbuf (clusters) during a test degrades performance and can even crash the system.
MAXFILES=16384
Polygraph needs more than the default number of file descriptors per process.
IF_ADDR_HASH
To take advantage of our hash table for large numbers of IP aliases.

So what did we change?

Added some hashing code to support large numbers of IP addresses/aliases.
disabled TCP delayed acks
increased user port range to 30,000
increased TCP SOMAXCONN to 1024
decreased TCP MSL to 3 seconds.

Here's a diff against the standard FreeBSD-3.4 tree.

Check Your Work

After you reboot the new kernel, run these commands to make sure the settings are correct.

Are TCP delayed acks disabled?

% sysctl net.inet.tcp.delayed_ack
net.inet.tcp.delayed_ack: 0

Is the user port range high enough?

% sysctl net.inet.ip.portrange.last
net.inet.ip.portrange.last: 30000

Is SOMAXCONN high enough?

% sysctl kern.ipc.somaxconn
kern.ipc.somaxconn: 1024

Is the IP address hash patch enabled?

% dmesg | grep ip_addr_hash
ip_addr_hash_reset: starting (old: capacity: 0)
ip_addr_hash_reset: done (1 entries, 20 bytes)

Check HZ, MAXFILES, and NMBCLUSTERS

% gdb /kernel
...
(gdb) p hz
$1 = 1000
(gdb) p maxfiles
$2 = 16384
(gdb) p nmbclusters
$3 = 32768
(gdb)

$Id: index.html,v 1.15 2001/06/05 19:32:49 wessels Exp $