Sometimes it appears that scientists worry about all sorts of problems before actually doing an experiment, but there’s a reason for this. One of the things scientists often want to do is to make comparisons between two things, such as comparing which of two drugs is better for treating a particular disease. It is important that the experiment to test this is designed to allow a “fair comparison”. If you want to compare which of two children can run the fastest, you could set up a race, shout “Go” and see who crosses the finish line first. But if one child had a 20 meters head-start, it’s obvious that wouldn’t be a fair comparison.
Example from everyday life.
Consider a teacher who wants to test if a new way of teaching multiplication is better than the old way. The teacher could ask the class to split into two groups, teach one group the old way and the other group the new way. Afterwards, the teacher could give the students a test and see which group did better in the test.
That sounds reasonable, right?
But there’s some problems with this. What would happen if the students stayed with their friends when they divided into groups? Perhaps the students who are good at maths tend to be friends with one another, so when they split into groups, one group had more students who were good at maths than the other group. So now, even if both methods for teaching multiplication were equally good, one group might do better in the test simply because that group had better maths students. That could lead us to incorrectly conclude that one method of teaching multiplication was better than the other.
The solution to this problem is to “randomise” students to each of the groups – students would be assigned to each of the groups at random. And then it would be sensible to check that the “maths ability” of each group was the same. We could do this by looking at how well the students in each group did in their last maths test. We could work out the average results for both groups in the last maths test and if they were the same it would be reasonable to use these groups to test the new method for teaching multiplication.
By doing this, we are setting up a fair comparison.
We would have to worry about other things as well to make sure the comparison was fair. For example, the two groups should be taught by the same teacher, do the exam at the same time of day, in rooms with comparable amounts of noise and so on.
An example from Medicine.
Let’s return to our problem of comparing two medicines to treat a disease – is a new medicine better than the existing medicine? To do this, we could split patients at random into groups, give one group the old medicine and one group the new medicine and measure the effect of the drugs on the disease in the two groups.
We need to randomise the patients to each of the two groups. Some patients may be more sick than others, they may be different ages, sexes, body weights, races and so on. All of these factors could affect how well the drug works. So we would need to check if any factor which might affect how the drug works is equally represented in the two groups.
Is this really worth worrying about?
Well, imagine the patients were not randomly distributed between the two groups. Perhaps one group had patients who were more sick, and this group received the new drug. Those patients might not respond to the drug as well as the other group, simply because they were more sick in the first place. We might conclude that the new drug is less effective, even though in reality it might be even better than the old drug. As a result, we might not use a valuable new drug, preventing patients from being treated as well as possible,and wasting the millions of pounds that were spent on developing the drug.
So it’s important that when we make comparisons, we set them up as fairly as possible so the results tell us what we are really trying to find out. Another way of making comparisons fair involves a concept called “Ratios and normalisation” and we will discuss that on another page.
