During our switch-off planning meeting, we discovered a few
controversial points. In particular, these were: (1) whether or
not it is fair to compare a L4 test that uses Direct Server Return
(DSR) to a test that does not use DSR, and (2) whether it is
reasonable to disable persistent connections in some polygraph
server configurations.
Another meta-issue is whether we should require all participants
to run both L4 and L7 workloads. Some people feel that this
benchmark can only be useful if there is a mandatory test that all
participants must take. Otherwise, the benchmark is useless as a
way to compare different products. However, some products work
only as a L7 device, and can not function in a L4 configuration.
TMF's goals are to (1) maximize participation from vendors who
compete in this space and (2) provide fair apples-to-apples
comparison(s) with realistic workloads. In some ways, these two
goals may be at odds. It is difficult to design a test that provides
a fair comparison of the wide range of products available. Below
is our current best plan for meeting the above goals. If you have
strong objections to our plan, now is the time to speak up.
We have two workloads, abbreviated here as "L4" and "L7."
Briefly, in the L4 workload, the device under test is expected to
distribute load among N identical servers. Each server contains
all the content and can serve any request. In the L7 test, the
content is partitioned among the servers. The device under test
is expected to use URL suffix matching to distribute load.
The L4 and L7 workloads are further split into ``Strict'' and
``Lenient'' sub-configurations. The difference between the Strict
and Lenient configurations are optional server-side changes only.
The Strict configuration is meant to simulate a customer with
specific and inflexible requirements. This customer will not be
convinced to change its server configuration to accommodate a
particular load-balancing solution. The Lenient configuration, on
the other hand, represents a customer who is willing to make certain
server-side changes to accommodate the vendor's optimal load-balancing
solution. In other words, the Strict configuration is identical
for all participants, and no server-side customizations are allowed.
For the Lenient configuration, we are willing to make a number of
server-side customizations, as outlined below.
So we have four total combinations:
L4 Strict
L4 Lenient
L7 Strict
L7 Lenient
For each workload, the Strict test is mandatory, but the Lenient
test is optional.
Note that here we are talking about a single Product. The vendor
is not allowed to bring one product for L4 tests and a different
product for L7 tests. The same product must take both tests.
The L4 test is mandatory for those products that support it. It
is difficult for TMF to know for sure whether or not a particular
product can support L4 traffic redirection. If the participant
tells us that their product does not support L4 redirection, or
can not complete the test, our report will say:
Product X does not support L4 traffic redirection.
Because L4 configurations are a significant piece of the load-balancing
market, we feel that this is a strong incentive for L4-capable
products to actually take the test.
Workload particulars:
L4 Strict
---------
Each Polygraph server uses a single unique IP address,
bound to the physical network interface. This prevents a
Direct Server Return configuration.
Polygraph clients and servers support and expect to
use HTTP persistent connections.
The load-balancing device is free to alter HTTP messages
such that connections become non-persistent.
L4 Lenient
----------
Upon a participant's request, we will:
- add one IP alias addresses per server, to either the
physical or loopback interface.
- alter the server's routing tables.
- disable persistent connections for Polygraph servers.
L7 Strict
---------
Each Polygraph server uses a single unique IP address,
bound to the physical network interface.
Polygraph clients and servers support and expect to
use HTTP persistent connections.
The load-balancing device is free to alter HTTP messages
such that connections become non-persistent.
L7 Lenient
----------
Each Polygraph server uses a single unique IP address,
bound to the physical network interface.
Upon a participant's request, we will:
- alter the server's routing tables.
- disable persistent connections for Polygraph servers.
We are open to being lenient in other ways than the points mentioned here.
If you have specific ideas, please share them with us.
Duane W.
This archive was generated by hypermail 2b29 : Mon Feb 06 2006 - 12:00:13 MST