(PM) Re: [jack.rickard@boardwatch.com: Re: Benchmarks

Karl Denninger (karl@Mcs.Net)
Sat, 21 Feb 1998 21:51:21 -0600

On Sat, Feb 21, 1998 at 10:38:08PM -0500, Doug Hass wrote:
> Why in the world has Karl turned this into a crusade to act out a personal
> vendetta against Jack Rickard and Boardwatch?

Its a crusade against false reporting and unethical behavior, and is not
confined to Boardwatch - I call 'em as I see 'em, and always have.

> No one has seen the full report, and some people (Tim had some valid
> points among others) have made for a good discussion.

In fact, Jack has *ADMITTED* in a message posted this evening that *HE DOES
NOT KNOW WHAT THE CENTRAL SITE HARDWARE HE DIALED INTO WAS*.

That makes the entire test invalid as a test of a *PROTOCOL*. It may or
may not make it invalid as a test of a specific piece of hardware, but
there simply is no way to *KNOW* this! None at all!

There is no statistical validity left without that information.

> Tim does make one valid point--can we do some research on USR's possibly
> altered connection rates? I've had personal experience with Total Control
> equipment and did some limited testing that bore out the fact that USR DID
> falsify two things to us:

That's nothing new. USR has always connected at rates which it cannot
sustain, and then immediately downshifted. The PM3s catch that insanity,
in that their STOP records in the RADIUS accounting show the ENDING
connect rates - which is all that counts.

I've seen this behavior for more than two years. It makes for great
advertising copy ("ie: the best connect rates of any modem") but its
misleading as hell.

Further, any test such as this which consists of just dialing somewhere and
seeing what speed "pops up" is invalid *ON ITS FACE*. The reason is that
many K56Flex modems will connect at a *deliberately* low rate and then
UPSHIFT.

This is, in fact, precisely the behavior you want. The reason is that a
renegotiation takes almost zero time to complete. DETECTING that you need
to do one when you need to downshift means you've taken LAPM block errors
which have impaired performance of the connection. If you get too
aggressive, you end up with a full retrain, which is a LOT worse (ie: 6-10
seconds of no data at all!)

An upshift, on the other hand, takes no such impairment and is virtually
invisible to the end-user.

Thus, what you MUST measure to get accurate results is the data rate after a
few minutes of actual transmission. That's likely a stable data link rate,
and is all that *really* counts in the real world.

This, unfortunately, means that you must SIGN IN to the ISPs you're testing
against.

It also means that the ISPs who have reported their statistics here are the
ones with REAL and VALID data, not "dial and burn" testing methodology in
which we don't even know what the hardware was on the remote end!

Again, the problem I have with "tests" like this is that if the methodology
is screwed up, the entire test is invalid and its results are worthless.

Well, other than to make a lot of noise and raise hell with, that is.

Finally, Rickard has now admitted that he has, in fact, blackballed Lucent.

So in fact I was right on both counts.

--
-- 
Karl Denninger (karl@MCS.Net)| MCSNet - Serving Chicagoland and Wisconsin
http://www.mcs.net/          | T1's from $600 monthly to FULL DS-3 Service
			     | NEW! K56Flex support on ALL modems
Voice: [+1 312 803-MCS1 x219]| EXCLUSIVE NEW FEATURE ON ALL PERSONAL ACCOUNTS
Fax:   [+1 312 803-4929]     | *SPAMBLOCK* Technology now included at no cost
-
To unsubscribe, email 'majordomo@livingston.com' with
'unsubscribe portmaster-users' in the body of the message.
Searchable list archive: <URL:http://www.livingston.com/Tech/archive/>