Re: (PM) Re: [jack.rickard@boardwatch.com: Re: Benchmarks

M. David Leonard (mdl@equinox.shaysnet.com)
Sat, 21 Feb 1998 20:57:06 -0500 (EST)

Jack Rickard sez:
> > How it affects YOU and whether you go with Ascend, Livingston, 3COM, etc.
> > are not only of no concern to me, but probably none of my business. If we
> > know something or have such information and DON'T publish it, I feel we'd
> > be remiss. It has been an eye opener to see the VEHEMENT reaction to a
> > press release. With supposedly knowledgeable individuals commenting
> > caustically, nay hysterically, about SEVERE METHODOLOGY FLAWS in a test
> > whose methodology they have absolutely no information about whatsoever.

Jack,
IMHO, you've hit the problem on the head. While it may be common
for magazines to publish test results without disclosing their
methodology, this should in no way be construed as validating such an
approach. In any peer-reviewed journal you would have to disclose your
"materials and methods" to the readers or risk being considered a
crackpot with nothing worth anyone's time to bother reading. As far as
I'm concerned, the BoardWatch study mentioned above will be filed under
alt.folklore.computers until such time as you see fit to provide complete
documentation of your study. I understand that others are concerned
about the influence you wield in terms of your readership and their
general lack of discrimination in evaluating such technical matters.
Since V.90 is soon to replace both X2 and K56Flex I personally think this
is already moot. But your position would be a great deal stronger if you
did describe the details of your methodology, and would be stronger yet
had you submitted the entire study to serious per review _before_ you
went public with your conclusions. In the absence of such reasonable and
logical steps, one is left wondering if you operate on the Barnum
Principle ("There's a sucker born every minute.").

David

-
To unsubscribe, email 'majordomo@livingston.com' with
'unsubscribe portmaster-users' in the body of the message.
Searchable list archive: <URL:http://www.livingston.com/Tech/archive/>