Regression toward the mean

[Bungee Jumper]

No, I don't feel I'm the only one who's right. The people who wrote the articles to which I linked obviously understand the phenomenon a lot better than you do.

859927[/snapback]

 

Okay...so you and hyperstats are right, and everyone else is wrong.

[Orton's Arm]

Okay...so you and hyperstats are right, and everyone else is wrong. 

859930[/snapback]

Whether intentionally or otherwise, you are throwing Wraith into this discussion by implying he's expressed disagreement with the phenomenon I've been describing.

[Bungee Jumper]

Whether intentionally or otherwise, you are throwing Wraith into this discussion by implying he's expressed disagreement with the phenomenon I've been describing.

859939[/snapback]

 

Did you even read his posts?

[Orton's Arm]

Did you even read his posts? 

859946[/snapback]

Once again, you're trying to throw him into this debate. If Wraith wants to dispute what I've been saying, let him do it. Don't put words into his mouth.

[Bungee Jumper]

If Wraith wants to dispute what I've been saying, let him do it. Don't put words into his mouth.

859950[/snapback]

 

Uhhhh...he did.

[Orton's Arm]

Uhhhh...he did. 

859955[/snapback]

I have defended HA against people who have said that:

 

- This specific behavior does not happen

 

- This specific behavior does not need measurement error (with a probability distribution) for it to occur.

[Bungee Jumper]

I have defended HA against people who have said that:

 

- This specific behavior does not happen

 

- This specific behavior does not need measurement error (with a probability distribution) for it to occur.

859959[/snapback]

 

Read the WHOLE post, Poindexter.

[justnzane]

ok i missed a lot of fun today, but holy crap. First HA is attributing performance on IQ tests and SAT's to luck more than ability. That said he changed the context of my post from IQ to SAT Then he has made an arse out of himself some more by digging himself deeper in the hole, especially by getting into another childish argument w/ Tom/BJ

 

HA scoring a certain score on a test is a measure of ability, especially on the SAT, since you lose points for taking unreasonable guesses. So, Luck has very little to do with the SAT. Now, with an IQ test, you will not score high if you use a lot of uneducated guess. therefore luck has relative no freaking value of impact on a test. Thus, the scoring is based on your ability and will vary very little because your ability is contingent on what you know and are capable of. So, the expected outcome for either of these tests on a retest should be what the test taker scored before.

 

Now if the test taker add more education to their memory banks before retesting, the expected outcome is to jump up slightly. So, suck on those apples

[Wraith]

ok i missed a lot of fun today, but holy crap. First HA is attributing performance on IQ tests and SAT's to luck more than ability. That said he changed the context of my post from IQ to SAT  Then he has made an arse out of himself some more by digging himself deeper in the hole, especially by getting into another childish argument w/ Tom/BJ

 

HA scoring a certain score on a test is a measure of ability, especially on the SAT, since you lose points for taking unreasonable guesses. So, Luck has very little to do with the SAT. Now, with an IQ test, you will not score high if you use a lot of uneducated guess. therefore luck has relative no freaking value of impact on a test. Thus, the scoring is based on your ability and will vary very little because your ability is contingent on what you know and are capable of. So, the expected outcome for either of these tests on a retest should be what the test taker scored before.

 

Now if the test taker add more education to their memory banks before retesting, the expected outcome is to jump up slightly. So, suck on those apples

860048[/snapback]

 

Let's stop calling it "luck" and start calling it "variation."

 

If you've got a test or measurement system that shows no variation, you have either an incredibly good test, or an incredibly bad one. It either means the test procedure has minimized the variation to such a minimal level that the measurement system is incapable of discerning the variation, or the measurement system is so bad that it is incapable of discerning the variation when it is not at a minimal level. These are really the same thing, because we get to define how minimal is minimal enough. I could measure the length of something that is nominally supposed to be 36 inches long with a set of micrometers and with a yard stick. If the lengths are varying on the scale of .001 of inch, the micrometer will probably show variation and the yard stick certainly won't. Which measurement system is more capable? Disregard the fact that you'd need a very strange set of mics to measure something 36" long with....

 

Everything in the real world shows variation. That includes human intelligence. There are plenty of outside factors that affect your ability to think, such as the amount of rest you've had, caffeine in the blood stream, distracting factors, etc. How you think, solve problems, and answer questions, varies to some extent over time. If a test of human intelligence shows no variation, it simply means it is not capable of detecting the variation. Sometimes, that's acceptable, sometimes it's not.

[Bungee Jumper]

Let's stop calling it "luck" and start calling it "variation."

 

If you've got a test or measurement system that shows no variation, you have either an incredibly good test, or an incredibly bad one. It either means the test procedure has minimized the variation to such a minimal level that the measurement system is incapable of discerning the variation, or the measurement system is so bad that it is incapable of discerning the variation when it is not at a minimal level. These are really the same thing, because we get to define how minimal is minimal enough. I could measure the length of something that is nominally supposed to be 36 inches long with a set of micrometers and with a yard stick. If the lengths are varying on the scale of .001 of inch, the micrometer will probably show variation and the yard stick certainly won't. Which measurement system is more capable? Disregard the fact that you'd need a very strange set of mics to measure something 36" long with....

 

Everything in the real world shows variation. That includes human intelligence. There are plenty of outside factors that affect your ability to think, such as the amount of rest you've had, caffeine in the blood stream, distracting factors, etc. How you think, solve problems, and answer questions, varies to some extent over time. If a test of human intelligence shows no variation, it simply means it is not capable of detecting the variation. Sometimes, that's acceptable, sometimes it's not.

860133[/snapback]

 

Let me hasten to add, before HA drools all over your post, that the variation in accuracy of your measurement of a single item is completely different than the natural variation in a normally distributed population of multiple items. You measure your 36" item multiple times with your very strange set of micrometers, and the average of your measurements will be 36". If, on the other hand, you have a set of items with normally distributed lengths and an average length of 36", and measure all of them with your micrometers, you will not find that the inaccuracy of your micrometers causes everything to be measured as 36" over time.

 

I know you know this, Wraith; I'm adding the clarification (such as it is, I'm afraid the above was unclear) for HA's benefit, as I'm pretty damned sure he'll fail to comprehend your post and insist it proves that regression in the error causes regression in the bulk population...

[Orton's Arm]

Let me hasten to add, before HA drools all over your post, that the variation in accuracy of your measurement of a single item is completely different than the natural variation in a normally distributed population of multiple items.  You measure your 36" item multiple times with your very strange set of micrometers, and the average of your measurements will be 36".  If, on the other hand, you have a set of items with normally distributed lengths and an average length of 36", and measure all of them with your micrometers, you will not find that the inaccuracy of your micrometers causes everything to be measured as 36" over time.

 

I know you know this, Wraith; I'm adding the clarification (such as it is, I'm afraid the above was unclear) for HA's benefit, as I'm pretty damned sure he'll fail to comprehend your post and insist it proves that regression in the error causes regression in the bulk population...

860136[/snapback]

I'm not sure why you felt the need to go through all that. All Wraith was saying is that a person's I.Q. scores ought to vary from one test to the next, because that person's underlying ability to think varies based on amount of rest, the time of day, etc. That particular post wasn't intended to address the heart of what we've been arguing about here.

[Taro T]

I'm not sure why you felt the need to go through all that. All Wraith was saying is that a person's I.Q. scores ought to vary from one test to the next, because that person's underlying ability to think varies based on amount of rest, the time of day, etc. That particular post wasn't intended to address the heart of what we've been arguing about here.

860147[/snapback]

That is true, and even with the person not having something affecting their "underlying ability to think" their score would likely vary from test to test. But that isn't what most people would consider "measurement error". That variability is what it is, the variability that is naturally inherent in a system and within the results that an individual member of a population within that system will experience. (Although there CAN be examples that DO exhibit measurement error and using what appears to be the definition of measurement error that you are using (which is FAR broader than most would use), this variability would fall within that definition.)

 

As CTM/BJ and Wraith both stated, if a particular individual is given the test often enough and it is a well designed, fair test that doesn't have an inherent bias, you will be able to discern what that person's "true" IQ is.

[justnzane]

That is true, and even with the person not having something affecting their "underlying ability to think" their score would likely vary from test to test.  But that isn't what most people would consider "measurement error".  That variability is what it is, the variability that is naturally inherent in a system and within the results that an individual member of a population within that system will experience.  (Although there CAN be examples that DO exhibit measurement error and using what appears to be the definition of measurement error that you are using (which is FAR broader than most would use), this variability would fall within that definition.)

 

As CTM/BJ and Wraith both stated, if a particular individual is given the test often enough and it is a well designed, fair test that doesn't have an inherent bias, you will be able to discern what that person's "true" IQ is.

860170[/snapback]

Pretty well stated; and in regards to that last paragraph, I agree. I was actually part of a grad students psych thesis on the topic, where i was tested five times for IQ and scored an average of 135 with a high of 138 and a low of 133. so, yes the whole true IQ concept would say that mine is 135, provided that i continued to score in that range.

[Bungee Jumper]

That particular post wasn't intended to address the heart of what we've been arguing about here.

860147[/snapback]

 

 

Actually, it is. That's one of the examples of the variability you're mistakenly claiming causes regression toward the mean. Which is why I felt the need to futilely add the caveat for you.

[Ramius]

 

Actually, it is.  That's one of the examples of the variability you're mistakenly claiming causes regression toward the mean.  Which is why I felt the need to futilely add the caveat for you.

860293[/snapback]

 

some people just dont know when to quit.

[Bungee Jumper]

some people just dont know when to quit.

860297[/snapback]

 

I don't know if you mean me, or HA.

 

Or both, honestly.

[Wraith]

Let me hasten to add....

860136[/snapback]

 

You’re right, my initial response was only addressing whether an IQ test will show variation. Taken in the larger context of this debate, I can definitely see how it could confuse the issue. I did not do a very good job of making a distinction between measurement variation and population variation.

 

I guess what I should have said was that in every test there are two broadly defined sources of variation:

 

A) The thing BEING tested

 

B) The thing DOING the testing

 

A very basic definition of a “capable” measurement system is that the variation from measurement system (source B above) is sufficiently small that variation from the process (source A above) can be seen. In other words, the noise does not overwhelm the signal.

 

There is an extreme opposite case where an extremely incapable measurement system can demonstrate zero variation but be totally inaccurate. This is often the case when the measurement system or test lacks sufficient resolution (called gage resolution). In that case, the test will probably show no variation but may show bias (the error PDF standard deviation = 0 and mean != 0). This is what I was getting at with my yardstick and micrometer example.

 

In a lot of cases, the two sources of variation are very distinct. But in my example above, regarding the IQ test, the difference between process variation and measurement variation is not well defined. Is the amount of adrenaline in your blood stream a controllable factor (in which case, any variation it causes in the test score would be measurement variation) or not (in which case, any variation it causes would be process variation). This becomes even more ambiguous when you consider that adrenaline levels can be PARTIALLY regulated (removing any outside stimulus from the testing environment) but you cannot keep the test subject from daydreaming….

 

Of course, I know next to nothing about IQ tests, so this particular example may be totally useless. I have no idea if adrenaline affects how well you perform on an IQ test. But the points remain that: everything shows variation; just because a test exhibits no variation does not make it an adequate test; there are two broadly defined categories of variation in a test, and occasionally it gets difficult trying to figure out which category various factors belong to.

 

It’s an interesting subject. One I wrestle with a occasionally in my day-to-day work.

[Wraith]

some people just dont know when to quit.

860297[/snapback]

 

Isn't that what makes debate fun? You always know the other guy is coming back for more...

[Ramius]

Isn't that what makes debate fun? You always know the other guy is coming back for more... 

860318[/snapback]

 

Very true. One of the biggest problems with the whole IQ thingy, is that you are trying to assign a number to this basically abstract concept called "intelligence". It cant be realiably or realistically quantified, no matter how good the test.

 

You can make your plastic parts all day, and they arent going to change just because you got into a fight with your wife the night before, or you drank too much, or you are excited about the weekend.

 

With an "IQ test", lets say i score a 130. The next time i take the test, maybe i score a 110. But what the score doesnt tell you is that i was up all night right before the test. So is that error in the test? No, that variance in score is attributed to being a human and having roughly 8,000,000 affecting your performance at any given time. I didnt "regress toward the mean" because there was error. I !@#$ed around all night, got no sleep, and then did piss poor on the test. The test was fine. it was ME that wasnt.

 

You cant reasonably expect to assign a number to something like "intelligence" when there are a ton of factors and forces that effect it.

[Bungee Jumper]

Very true. One of the biggest problems with the whole IQ thingy, is that you are trying to assign a number to this basically abstract concept called "intelligence". It cant be realiably or realistically quantified, no matter how good the test.

 

You can make your plastic parts all day, and they arent going to change just because you got into a fight with your wife the night before, or you drank too much, or you are excited about the weekend.

 

With an "IQ test", lets say i score a 130. The next time i take the test, maybe i score a 110. But what the score doesnt tell you is that i was up all night right before the test. So is that error in the test? No, that variance in score is attributed to being a human and having roughly 8,000,000 affecting your performance at any given time. I didnt "regress toward the mean" because there was error. I !@#$ed around all night, got no sleep, and then did piss poor on the test. The test was fine. it was ME that wasnt.

 

You cant reasonably expect to assign a number to something like "intelligence" when there are a ton of factors and forces that effect it.

860370[/snapback]

 

 

And again - in another attempt to futilely point out the blisteringly obvious for HA's benefit - that's variation in a single individual's scores, NOT variation in test scores throughout a normally distributed population.

 

Not that he'll understand it this time, any more than he has the other fifty times I've said it...