Experimental Confound.

Posted: December 13th, 2022

Experimental Confound.

Dealing With Confounds Imagine an experiment in which research participants are asked to recognize letter strings briefly presented on a computer screen—let’s say for 30 milliseconds—-followed by a mask. In the first 50 trials, the letter strings are random sequences (“okbo,” “pmla,” and so on). In the next 50 trials, the letter strings are all common four-letter words (“book,” “lamp,” “tree,” and so on). Let’s say that the participants are able, on average, to identify 30% of the random sequences and 65% of the words. This is a large difference; what should we conclude from it? Experimental Confound. In fact, we can conclude nothing from this (fictional) experiment, because the procedure just described is flawed.

ORDER NOW.                    

The data tell us that participants did much better with the words, but why is this? One possibility is that words are, in fact, easier to recognize than non-words. Experimental Confound.A different possibility, however, is that we are instead seeing an effect of practice: Maybe the participants did better with the word trials not because words are special, but simply because the words came later in the experiment, after the participants had gained some experience with the procedure. Conversely, perhaps the participants did worse with the non-words not because they were hard to recognize, but because they were presented before any practice or warm-up. To put this in technical terms, the experiment just described is invalid—that is, it does not measure what it is intended to measure—namely, the difference between words and non-words. Experimental Confound. The experiment is invalid because a confound is present—an extra variable that could have caused the observed data pattern. Experimental Confound.The confound in this particular case is the sequence, and the confound makes the data ambiguous: Maybe words were better recognized because they’re words, or maybe the words were better recognized simply because they came second. With no way in these data to choose between these interpretations, we cannot say which is the correct interpretation, and hence we can draw no conclusions from the experiment. How should this experiment have been designed? One possibility is to counterbalance the sequence of trials: For half of the participants, we would show the words first, then the random letters. For the other half of the participants, we would use the reverse order—random letters, then words. This setup doesn’t eliminate the effect of practice, but it ensures that practice has the same impact on both conditions. Specifically, with this setup, practice would favor one condition half the time and the other condition half the time. Thus, the contribution of practice would be the same for both conditions, and so it could not be the cause of a difference between the conditions. Experimental Confound. If this point isn’t perfectly clear, consider an analogy: Imagine a championship football game between the Rockets and the Bulldogs. As it turns out, there’s a strong wind blowing across the field, and the wind is coming from behind the Rockets. The wind helps the Rockets throw and kick the ball farther, giving them an unfair advantage. The referees have no way to eliminate the wind. What they can do, though, is have the teams take turns in which direction they’re moving. For one quarter of the game, the Rockets have their backs to the wind; then, in the next quarter, the direction of play is reversed, so it’s the Bulldogs who have their backs to the wind, and so on. (This is, of course, how football games operate.) That way, the wind doesn’t favor one team over the other, and so, when the Rockets win, we can’t say it was because of the wind; in other words, the wind could not have caused the difference between the teams. Returning to our word/non-word experiment, we know how it would turn out when properly done: Words are, in fact, easier to recognize. Our point here, though, lies in what it takes for the experiment to be “properly done.” In this and in all experiments, we need to remove confounds so that we can be sure what lies beneath the data pattern. Several techniques are available for dealing with confounds; we’ve mentioned just one of them (counterbalancing) here. The key, however, is that the confounds must be removed; only then can we legitimately draw conclusions from the experiment. Discussion Question What is an experimental confound, and how might it render an experiment invalid? How would it apply to the experiment above? Experimental Confound.

Experimental confound

An experimental confound is a variable that occurs in an experiment, is not of interest to the experiment but affects the results through moderating the effects of the independent variable on the dependent variable. In fact, it is an extraneous influence that has the potential for ruining the experiment and causing it to produce useless results through suggesting that there is a correlation between the dependent and independent variables when in fact there is no such correlation between the two. Considered as the third variable in any experiment (with the independent and dependent variables acting as the first two variables), confounding variables are factors that are not accounted for in the experiment design by still end up affecting the result, ruining the experiment and giving out useless results through acting as extra independent variables that are not accounted for but have a hidden effect on the dependent variable (Harrington, 2011). Besides suggesting correlations that do not exist through increasing variance, confounding variables can also introduce bias in the experiment through underestimating or overestimating parameters. With regards to bias, confounding variables can have a negative of positive effect. The negative effect occurs when the correlation between the dependent and independent variables in the experiment shift towards the null so that the effect is underestimated. The positive effect occurs when the correlation between the dependent and independent variables in the experiment shift away from the null so that the effect is overestimated (Harrington, 2011). Overall, confounding variables affect the experiment design and must be identified and controlled in order to produce accurate (true correlations) and valid (non-biased) experiment results. Experimental Confound.

The effects of confounding variables in experiments can be minimized through a stepwise approach that ensures that the experiment measures what it is intended to measure. In the discussed experiment, it was noted that the differences reported from the identification of word and non-words could have been affected by sequence as a confounding variable. In this case, the participants first come into contact with non-words before words thus creating the possibility that the results could be affected by the fact that words are more familiar, and the participants were less familiar when they first started the experiment with non-words and they became more familiar in the next round when using words (Bordens & Horowitz, 2013). In applying the stepwise approach, the first step is to identify the confounding variable, identified as sequence. The second step is to apply a methodology that would minimize the confounding variable. The first methodology is to reduce bias through random sampling and assignment (counterbalancing). This entails randomly dividing the participants into two groups with equal numbers of participants so that the first group begins with the non-words followed by the words while the second group begins with the words followed by the non-words. The average of the two groups would be considered as the results. The second methodology is to introduce a control variable that restricts the participants’ demographics using a narrow range of inclusion and exclusion criteria (Bordens & Horowitz, 2013). For instance, introducing age ranges as a controlling variable that restricts the valid participants to persons between 30 and 40 years of age since age could also have an effect on the experiment and identified controlling variable. Applying the discussed methods reduces experimental confound. Experimental Confound.