A Statistical Fallacy Using an Unselected Sample for an Average Result (Revised)
Sunday, 29. July 2007, 15:34:12
Normally, statisticians seek large enough samples properly selected so that data are representative of the particular group under investigation. When samples are "unselected," biases can be introduced in several ways--especially when the sample size is too small and the data presented are unrepresentative.
Consider the following argument purporting to indicate common factors of all persons suffering from back pain:
(Benno Isaacs and Jay Kobler, The Nickolaus Technique (New York: Viking, 1978), 36.)
The results cited were in 80% of the cases the back pain was due to muscular insufficiency or poor flexibility.
Is an inference from a class composed of a given sequence of 5000 patients seen at two university hospitals in New York over a specified time-interval in the 1970's applicable everywhere at any time to each and every person who experiences back pain? Of course not.
A few of the implicit biases can be highlighted by raising the following questions:
(1) Are men or women more likely to seek help for back pain?
(2) Are persons with health insurance more likely to seek help for back pain?
(3) Is age, religion, race, culture, language, and so forth a significant factor for a person to seek actively university hospital treatment in New York?
(4) Does a hospital's reputation affect the composition of patient admission?
(5) Are persons in New York in the 1970's during the specified interval seeking help for back pain at New York University and Columbia University representative of everyone?
(6) Are persons with some kinds of back pain more likely seek help than others with different kind of back pain?
(7) Are persons in a large city more likely to seek help than person living in the country or small cities?
(8) Are college students more likely to seek help at a university hospital and are these individuals representative of the general population of the world?
(9) Do the university hospitals in question have a reputation for treating specific kinds of back pain?
(10) Was facility of transportation at the time of the study in New York city representative of facility of transportation for any place in the world?
The fallacy committed here is termed the hasty generalization. This fallacy occurs when one argues from the evidence of certain carelessly selected cases and to a generalized conclusion based solely on that evidence. For example, simply because some sand dunes are on a beach are mostly yellow in color, it does not necessarily follow all sand dunes on every beach are mostly yellow in color.
Likewise, simply because most of the 5000 persons with back pain admitted to New York University and Columbia University Medical Centers had muscular insufficiency and inadequate flexibility, this does not imply that anyone, everywhere, at any time, with back pain has muscular insufficiency and inadequate flexibility.
Note that the conclusion of the argument presented by the authors cited above may well be true. The fact that the argument is fallacious means that the conclusion does not follow logically from the evidence presented.
Consider the following argument purporting to indicate common factors of all persons suffering from back pain:
A combined medical group from New York University and Columbia University conducted a study covering 5000 consecutive patients with back pain. Because the study included every back-pain patient seen at the two universities until the total of 5000 was reached, it represented an unselected sample. Its results apply to everyone with back pain, rather than to a special segment of back-pain patients.
(Benno Isaacs and Jay Kobler, The Nickolaus Technique (New York: Viking, 1978), 36.)
The results cited were in 80% of the cases the back pain was due to muscular insufficiency or poor flexibility.
Is an inference from a class composed of a given sequence of 5000 patients seen at two university hospitals in New York over a specified time-interval in the 1970's applicable everywhere at any time to each and every person who experiences back pain? Of course not.
A few of the implicit biases can be highlighted by raising the following questions:
(1) Are men or women more likely to seek help for back pain?
(2) Are persons with health insurance more likely to seek help for back pain?
(3) Is age, religion, race, culture, language, and so forth a significant factor for a person to seek actively university hospital treatment in New York?
(4) Does a hospital's reputation affect the composition of patient admission?
(5) Are persons in New York in the 1970's during the specified interval seeking help for back pain at New York University and Columbia University representative of everyone?
(6) Are persons with some kinds of back pain more likely seek help than others with different kind of back pain?
(7) Are persons in a large city more likely to seek help than person living in the country or small cities?
(8) Are college students more likely to seek help at a university hospital and are these individuals representative of the general population of the world?
(9) Do the university hospitals in question have a reputation for treating specific kinds of back pain?
(10) Was facility of transportation at the time of the study in New York city representative of facility of transportation for any place in the world?
The fallacy committed here is termed the hasty generalization. This fallacy occurs when one argues from the evidence of certain carelessly selected cases and to a generalized conclusion based solely on that evidence. For example, simply because some sand dunes are on a beach are mostly yellow in color, it does not necessarily follow all sand dunes on every beach are mostly yellow in color.
Likewise, simply because most of the 5000 persons with back pain admitted to New York University and Columbia University Medical Centers had muscular insufficiency and inadequate flexibility, this does not imply that anyone, everywhere, at any time, with back pain has muscular insufficiency and inadequate flexibility.
Note that the conclusion of the argument presented by the authors cited above may well be true. The fact that the argument is fallacious means that the conclusion does not follow logically from the evidence presented.
WIDGETIZE
etoo # 11. September 2007, 17:42
Again, nice case, though. I just found you from onegoodmove.org and look forward to reading more.
Brad
philhelp # 29. September 2007, 16:22
The language before the post was revised was "The fallacy committed here is termed the fallacy of division. This fallacy occurs when one argues that a property of a class as a whole must apply, as well, to the individual parts of the class distributively. For example, simply because a beach is yellow in color, it does not necessarily follow that each individual grain of sand on that beach is yellow in color."
Brad notes above, "It doesn't seem to be an example of the fallacy of division. This fallacy is something like, MOST members of group X have Y, therefore most members of a larger group of which group X is a subset have Y," and he is entirely correct.
I am completely at a loss as to what I was thinking I wrote the original post.
Thanks Brad for your valuable correction.