Garbage In, Garbage Out

You probably already know (even if you forgot it for a while) that it is impossible to reach valid conclusions if you begin with unreliable information. So, how can you discern what information to trust, particularly in the areas of research-based sources, religious persuasion, political rhetoric, and advertising information?
Let’s begin with research. If you get information that is based in research, it must be true, right? Well, maybe if it’s scientific. Or maybe its truth depends on whether you understand how research-based conclusions work. For example, how can you be sure that the apparent conclusions actually measure what they say they measure? Furthermore, statistical data from scientific research can be used in a wide variety of ways and the same data can even seem to support opposite conclusions.
Let me give you an example of how confusing research-based statistical data can be. I was the chaplain in a state prison for a number of years from the mid 1970’s to the mid 1980’s. At least a part of my job was to provide “rehabilitative” opportunities to the inmates. A common measure for those who addressed the viability of prison and its rehabilitative programs was called the “recidivism rate.” Recidivism measures the tendency of released inmates to re-offend. Depending on the source of the information, you might discover that the recidivism rate was about 67%, or maybe 33%. If you were in favor of rehabilitative programs, you would tend to believe the lower number, but many who were not in favor of rehab programs typically used the higher number.
One of these numbers must be wrong, right?
(NOTE: these numbers are 30 years old, are probably no longer accurate, and are used only to illustrate a point about statistical data.) Well, actually both results were correct. Let me explain.
Let’s begin with these two questions: how can these very different numbers possibly come from the same raw data, and what then might they mean? The difference in the numbers results from two different ways of measuring recidivism. If you had polled the new inmates as they arrived at prison, you would have discover that about 2/3 of them had been in prison before, but if you had followed the people released from prison over the following few years, you would have discovered that only about 1/3 of them would offend again.
So, is one measure of the recidivism rate more correct than the other? Not really, as long as you know what the given study was actually measuring. But even you know what is being measured, there remain many more layers of complexity that might need to be considered. For example, the recidivism rate only measures the rate of repeat prison sentences. It does not consider the relative severity of the crimes. What if most of the people who returned to prison had been newly convicted of a lesser crime than the time before? Or what if most were convicted of a more serious crime this time? Without any observable variation in the Recidivism Rate, you still might make an argument for or against the effectiveness of rehabilitation, but how reliable is your data. I could identify several additional factors that might affect the validity of the Recidivism Rate, but hope you get the point from this illustration: if you want to think clearly, research-based data must be used with great care.
In my non-scientific observation of how research-based data is used, particularly in the worlds of politics and advertising, it is seldom intended to offer reliable information that helps voters or consumers make a more informed choice among alternatives. Typically, (in my experience) the point of advertising and political discourse seems to be to direct the choice of the recipient in a predetermined direction. In short, the point of such communication is manipulation.
You might wonder if I am using this blog to manipulate your thinking and your choices. Well, considering that I am a product of this culture, probably so. But I don’t want to do that. I invite you, therefore, to call me on any perceived (though largely unintended) manipulations. My intentional objective, however, is to stimulate your thinking, not manipulate you toward specific conclusions.
In the spirit of honest disclosure, I do believe that manipulation is minimized when one participates in the give and take of information and perspective that is characteristic of healthy community. In community, we help keep one another honest, or at least more aware. So if I want any particular result from my blog, it is that you will grow in your understanding and appreciation of healthy community.
This is how I see it. What do you see from your perspective?
Wayne Gustafson
The Promised Land is within and among us.”
0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *