This page gives specific information about the Co-Advisor project. The items below are based on the feedback of future core members. The project is currently in its first stage. Potential members are requested to make a firm commitment based on the material on this page.

Table of Contents

1. Co-Advisor Overview
2. Design Requirements
3. Membership Perks
4. Budget and Fees
5. Current Status
6. Timeline
7. What's Next?

Co-Advisor is a collection of tests and a driver program to execute the tests. The tool works with Web clients, servers, and intermediaries. Co-Advisor checks for compatibility with existing Web products, compliance with protocol specs, agreement with good-practice standards, susceptibility to vendor-specific bugs, and other requirements. New tests are added based on user demand/interest; the test set is determined by actual user needs. Test groups can be formed (e.g., ``HTTP/1.1 compliance test suite'' or ``Netscape Navigator compatibility test suite''). Advisor's objective is to automate or simplify QA tasks, whatever they are.

The Measurement Factory owns all software developed for the project. However, third parties may develop and/or own test collections (just like third parties may own C programs without owning or licensing a C compiler, and just like Qwest owns its Yellow Pages without owning phone numbers or company names).

We intend to use the best practices of our Web Polygraph project while keeping the two projects independent.

The Co-Advisor project is seeded by membership fees. Project members get certain privileges. Non-paying users get restricted free access to the tool.

The following requirements are in the foundation of the Co-Advisor design.

1. Many meaningful tests can be added without modifying core software and without breaking existing tests or interfaces.

2. Simple user interface allowing for batch jobs and unmanned tests.

3. Support for sophisticated tests, possibly requiring client-server cooperation, run-time decision making, and/or transaction synchronization.

4. Focus on non-performance tests.

5. Collection of detailed logs of every transaction, documenting actions taken and explaining detected errors.

6. Test purpose and testing method documentation for every test case.

7. Documentation and logs in domain-specific language understandable for a professional not intimately familiar with Co-Advisor.

First tests will target HTTP intermediaries. These features may be added in the foreseeable future:

1. Log anonymization allowing for sharing test results without explicitly exposing the device under test.

2. Signatures to authenticate the results.

3. Testing of HTTP end clients and origin servers.

4. Testing of non-HTTP agents.

Project members are entitled to the following privileges.

1. opportunity to influence the tool design
2. exclusive access to early, private releases
3. vote on features priority
4. vote on public release dates and public test schedules
5. deep discounts on and priority handling of the development and test work
6. publicity and association with a for-the-common-good project

We expect that about 80% of the budget will go towards tool design, development, and support costs. The rest will be spent on publicity, legal support, and various minor overheads.

Membership fees are the basis for project sustainability. We want, however, to keep them minimal and instead charge for services actually used. The fee structure is designed to make membership very attractive for those who need more than a couple of for-fee Co-Advisor services per year.

All fees are approximate. Cryptic terms are explained below the table.

Service Fees ($1,000 US) members public project membership 15 (per year) n/a access to the current (private) tool distribution free n/a access to an older (public) tool distribution free free best-effort support free (higher priority) free (lower priority) simple development projects free (higher priority) free (lower priority) complex development projects 1-5 (per project) 3-15 (per project) training on customer site 1 (per day) 3 (per day) public test-offs 2 (per-entry) 5 (per-entry)

As membership and/or demand for services increase, membership fees are likely to go down and may eventually be eliminated. TMF may also offer discounts to veteran members to offset their costs of the initial contribution.

There will be two supported distributions of the software: a current (private) distribution and an older (public) distribution. Project members will determine when the transition of new tests and features to the public distribution takes place.

TMF develops new tests for project members and general public users. We distinguish two primary development categories: simple and complex. Simple development is when new tests require no modifications of the core software (i.e., the tests that are using existing software features). The other end of the spectrum is complex tests requiring extensive core modifications. Members always receive higher priority in case of work scheduling conflicts.

Project members always receive higher priority in case of support or development scheduling conflicts.

Given sufficient demand, TMF will host public test-offs using a subset of Co-Advisor tests. Test-off rules and test subset are determined by participants.

At the time of writing, the following companies expressed their interest: Akamai, Cisco, F5, InterX, Lucent, LodBroker, Mirror Image, and Novell. We have also discussed possible cooperation with the folks from AT&T and HP research labs. W3C is likely to form a QA activity around April 2001; TMF may participate in that activity.

TMF is currently evaluating a suite of compliance tests provided to us by one of the caching vendors. The tool currently contains 256 test cases and is designed to work with caching proxies. While more work is needed to make the tool publicly available, we believe that it can work as a prototype and can help to bootstrap the development.

Here is the ``totals'' statistics of the prototype application to Squid caching proxy:

	Passed Tests    : 93
	Failed Tests    : 163
	Aborted Tests   : 0
	Total Tests     : 256

Note that many of the failed tests are due to compatibility problems and other bugs. As we said, the tests are not ready for production use yet.

We live by the following schedule.

January - February

A core support group is built. Overall tool design is discussed and agreed upon. Membership fees get collected. A working tool prototype is made available to the members.

February

First software releases are made available.

March - April

Stable software version is released.

April

First round of non-competitive tests can be held if there is sufficient interest.

If you are interested in becoming a member, please contact us, preferably by e-mail. If you are concerned about details not covered in this document (e.g., exact payment procedures or software licensing terms), please let us know, and we will add missing information.

To go ahead with the project, we will need 5-6 members to make firm commitments in the form of purchase orders.