CR Methodology

["Al Powell" <apowell@agcom.tamu.edu>]

Jon said:

> I glance at, then ignore Consumer Reports when it comes to car 
> purchases.  If I paid attention I would never have owned any of my 
> four VW's, all of which were great cars, nor would I have 
> purchased my current Audi Coupe.  

True also of my 1977 Chevy Monza Spyder and of my 1981 Citation Sport 
model, both of which were great cars - and I still own the Monza (but 
must sell it - hint, hint...)
> 
> I was always confused by their methods of obtaining information for 
> their reliability surveys.  I assume they only poll their subscribers 
> because I have never once received anything from them even after 
> having purchased a new car.

There was another post saying that there are seldom criticisms of 
CR's methodology.  Here is one observation....

CR does in fact depend entirely on self-reports from their 
subscribers.  They send out annual surveys in which their subscribers 
are asked to report on cars, appliances, and various other things.  
Now, I think that this is probably the MOST feasible and economical 
method for them to obtain data.  If I were in their place, I would 
probably do the same.

BUT - I do have some problems with the data is is POSSIBLE to 
generate from their surveys.  Since I don't have one in front of me,k 
bear with me....

Memory says they ONLY take reports on appliances which have been 
purchased in about the last year.  (Maybe it's two, so sue me....)  I 
believe that their cutoff on cars is about three years.  (Again, give 
or take a year, who cares.) I want to make two points about this:

1)  It reveals NOTHING about long-term durability.  Audis (the 
interest of this group) shine on durability, although the point has 
been made that reliability and durability are two different things.

2)  I have suspected for years  that CR readers tend to UNDER-report 
repairs to cars which they *believe* are more reliable, and to report 
repairs more often when they are pre-disposed to believe they are not 
reliable.  I cannot prove this assertion, but it is an inherent 
weakness of self-reported data.  Perceptions of the self-reporter can 
easily bias the data.

3)  Self-reporting has an additional weakness in that it depends on 
the reporter's memory.  If one does not remember a visit to the shop, 
one does not report it.  And what about warranty repairs?  Should 
they be reported or not?  I am confident (this assertion also not 
provable, but logical) that self-reporting of warranty repairs is 
VERY uneven with CR's method.

This being said, I think that when reporting on products toward 
which the reporter has little or no emotional (affective) 
relationship, reports are probably more accurate.  Let's put 
toasters and vacuum cleaners in this column....

But when reporting on products like cars, about which some people 
have strong emotional agendas (involving social status and 
self-image, among others...) it is reasonable to think that reports 
are biased by the pre-conceptions of the reporter.  Example: if one 
thinks Audis are not reliable, one tends to report every hiccup.  
if one thinks Impalas are highly reliable, one tends to overlook 
small problems.

Them's my comments about CR's data-gathering methodology, Audi 
comment included!!

So much for Research Methods 201.......



*.......................................................*
Al Powell, Ph.D.                 Voice:  409/845-2807
107 Reed McDonald Bldg.          Fax:    409/862-1202
College Station, TX 77843      
Http://agcomwww.tamu.edu/agcom/satellit/alpage.htm
*.......................................................*