Tuesday, 11 March 2014

On Bad Science and False Premises

Many scientists do bad science by hiding behind empiricism. 

Empiricism is when you do things using the scientific method i.e you observe something, investigate it by building a hypothesis based on what you know, look at the existing literature, create a hypothesis, conduct a study, experiment, collect evidence and build a theory. 

Here's the thing. Your hypothesis has to be based on evidence. You cannot simply make stuff up to explain what you see, and then justify your assumptions by claiming that you're testing your hypothesis. You can't do that. 

If you create a hypothesis based on baseless assumptions, and then test it and find data that supports your hypothesis, it might only be correlational, or coincidental, or due to confounding variables. The data itself would not indicate cause and effect, or the truth of your hypothesis. This is because your original theory was made up to begin with. A false premise. 

The data collected in support of a baseless hypothesis might be good, but since it has been collected based on a preconceived notion of its role in the hypothesis, it is meaningless.

"I think men like playing first person shooter video games because of their hunter gatherer tendencies. Let me conduct an experiment to see if I can indicate this to be true. Yup, there's an association between the two variables. Men play more first person shooter video games than other types of games, and they play these more than women do. These games simulate hunting, and in the past men hunted more than women. Therefore, my hypothesis is supported."

Of course, the only reason this supports your hypothesis is because you made up the conclusion to begin with. The data might be sound, and the association might be real (and due to any number of other factors), but the inference is false, because you're merely fitting the data into a preconceived role or conclusion in your mind, and in your silly hypothesis. You've decided your conclusion in advance, and then you're letting your results justify your conclusion, even though they might indicate any number of other things that haven't been hypothesised yet. 

"If A exists, then so must B. I found A, therefore B exists." 

Unfortunately, the former is an assumption. Therefore, the implication in the latter is not always true. Bad researchers don't get this.

You cannot use the fact that you're testing your hypothesis to excuse the fact that your hypothesis is based on no originating data, and merely something you dreamed up. Your starting point must always be evidence. Without evidence, you have no starting point, and no hypothesis. You cannot indulge in guesswork as a replacement for evidence. You cannot assume that a complex facet of human behaviour exhibited by people today has an evolutionary basis originating in a behavioural trend exhibited by people 500K years ago.

This is where evolutionary psychology gets it wrong. Researchers make a guess, a 'just so' story that feeds into our existing status quo of how we imagine the world should work (it sounds true therefore it is probably true). When confronted with the fact that they have no evidence to support their theory, they defer to helplessness and empiricism - 

"No one can prove evolution. No one was around to record behaviour 500K years ago, so my hypothesis could be wrong, but at least I can test it. So my approach is scientific. At least I'm better than a religious person. At least I'm willing to test my theory by creating and testing hypotheses and admit I could be wrong." 

What. A. Crock.

It is extremely important to be consciously aware of your underlying assumptions and implicit biases when formulating a theory. Do researchers do this? Nope. They allow their biases and assumptions to dictate their research questions, their hypotheses. Whether their hypotheses are true or not is irrelevant. Once a hypothesis is created and tested, it becomes part of the scientific narrative. Your students, peers and other researchers will spend the rest of their lives creating hypotheses that match your theory, hoping to reject null hypotheses in favour of ones that support your theory, all founded on nothing, everyone wasting their time and energy.

If someone thought their desk was a dragon, we'd call them crazy. However, if they spent their lives trying to test the animatedness of their desk, believing it were alive, would you still take them seriously? Of course not. But people around the world everyday do the same with bad hypotheses. Hypotheses with no basis. Hypotheses created based on someone's assumptions. Hypotheses created because they 'feel right'. Hypotheses created because their creator lets them reflect his or her own biases about how the world functions.

Your ability to create and falsify hypotheses does not justify your creation of theories with no underlying bases i.e making bad assumptions. It is important not to confuse science with empiricism. You might be a brilliant empiricist, but you'd still be a bad scientist if the assumptions inherent in your empiricism had no basis. It's easy to hide poor assumptions and reasoning behind good empiricism. Don't do this.


No comments:

Post a Comment