Biology IX RO

Chapter 1 · Method and the microscopeLesson 1.5CS

Advanced scientific investigation: errors, risks and arguing for results

What would it take to convince someone who does not believe you that a well’s water has too much nitrate in it, or that hand-washing saves lives?

Size
from 1 mm to 1 nm
Level
cell and cell structures
Time
one lesson in the lab

Test tomorrowten questions from chapter 1

Before you read: what do you think?

Researchers in Bucharest expected the wells in the two communes (rural districts) closest to the capital to have the least nitrate. The limit for drinking water is 50 mg/l. What did they find in 10 wells in Brănești, in Ilfov county?

See the answer

In five wells nitrate was between 272 and 493 mg/l, more than five times the limit, and in four it could not be detected. The starting hypothesis was disproved for Brănești; the authors point out that a non-ferrous metallurgy plant used to operate in the commune.

In 1846, at the Vienna General Hospital, women giving birth in the First Clinic died five times as often as those in the Second Clinic. The physician Ignaz Semmelweis looked for the difference between the two and imposed a simple rule: washing hands in a solution of chlorinated lime. Yearly mortality in the First Clinic fell from 11.4% in 1846 to 1.3% in 1848, but many colleagues did not believe him.

From a real problem to a hypothesis

An advanced investigation starts from a problem in a real context, in health, the environment or biotechnology, and narrows it down until it becomes a question that can be answered. Semmelweis started from a problem: puerperal fever, an infection after childbirth, was killing many women. His question: why do more women die in the First Clinic than in the Second?

The two clinics had one difference: the first was staffed by doctors and students, who also performed autopsies, and the second by midwives, who did not. When a colleague, Jakob Kolletschka, was accidentally pricked by a student’s knife during an autopsy and died with the same symptoms as the sick women, Semmelweis formulated his hypothesis: doctors’ hands carry disease-causing particles from the corpses to the patients. The independent variable became hand-washing with chlorinated lime, and the dependent variable, mortality.

Random errors and systematic errors

Random error makes repeated measurements vary, sometimes up, sometimes down: a hurried reading, a drop slightly too big or too small, a stopwatch started sometimes too early, sometimes too late. It is reduced by repeating measurements and calculating the mean. Natural differences between individuals, between two seeds for instance, also make results vary, although they are not measurement errors; that is why several individuals are used.

Systematic error shifts all the results in the same direction: a balance that always reads 2 g too high, an out-of-date reagent, a method that underestimates. Repeating does not reduce it, because every measurement is wrong in the same way. It is uncovered by calibrating the instrument, by samples of known value or by a second method, independent of the first.

Two terms come from this distinction. Precision shows how close repeated measurements are to one another. Accuracy shows how close they are to the true value. An uncalibrated instrument can be very precise and still inaccurate.

precise, accurate small errors precise, inaccurate systematic error imprecise, accurate mean random error imprecise, inaccurate both errors
Figure 1The centre of the target is the true value. Repeats and the mean correct the scatter but do not move a shifted cluster: systematic error is uncovered by calibration or by a second method.

Risks and the strategy of the investigation

Before starting, the risks are identified: to safety (chemicals, glass, cultures of microorganisms, which at school are never grown from unknown samples), to ethics (people, animals) and to the result (sources of error, lost samples). For each one, you note how it is prevented.

Then the strategy is chosen. An experiment changes a variable on purpose. For a reliable comparison, a control group and random assignment are used. When that is not possible, for the wells of a village for example, an observational study is carried out, comparing groups that already exist. The strategy also sets how many samples are taken, where, when and how they are coded, so that the person measuring does not know which sample is in front of them.

Arguing and improving

You argue for a result in three steps: the claim (what you assert), the evidence (the data that support it) and the reasoning (why the evidence leads to the claim). A complete argument also discusses alternative explanations and says why they are less likely.

Semmelweis had strong evidence, but he did not convince people. He had no mechanism his colleagues would accept, and his main work did not appear until 1861. His explanation was accepted only decades after his death, when Louis Pasteur, Joseph Lister and Robert Koch demonstrated the role of microorganisms in disease.

Every investigation ends with its limitations and with proposals: more samples, other seasons, other places, a more accurate method, a variable that was left uncontrolled.

0 10 20 30 40 50 60 70 1 52.08 2 50.03 3 4 66.58 5 6 7 8 49.78 published mean limit: 50 mg/l well nitrate (mg/l)
Figure 2The published mean of the eight wells, 49.78 mg/l, is below the 50 mg/l limit, but three wells exceed it. The mean alone would hide exactly the wells with a problem.

Worked example

Argue for Semmelweis’s result

The claim: hand-washing with chlorinated lime reduced mortality in the First Clinic.

The evidence: yearly mortality fell from 11.4% in 1846 to 1.3% in 1848. The difference is 11.4 − 1.3 = 10.1 percentage points; the ratio 11.4 ÷ 1.3 ≈ 8.8 shows that the risk of death fell almost ninefold.

The reasoning: beforehand, the only known difference between the clinics was the hands that came from autopsies; once the hands were washed, the first clinic’s mortality came down towards that of the midwives’ clinic.

Alternative explanations to discuss: did something else change in the hospital at the same time? Did fewer sick patients arrive? Semmelweis changed a variable on purpose, but the women were not assigned at random, and mortality is compared before and after the change, so these possibilities have to be ruled out with data. An improvement: a month-by-month comparison with the Second Clinic, where the rule did not change.

In Romania

Nitrate in the wells of four villages

Maria Pele, Gina Vasile and Mihaela Artimon, of the University of Agronomic Sciences and Veterinary Medicine of Bucharest, published their analyses of well water from four places in 2010: Matca (Galați county, vegetables in greenhouses and polytunnels), Săhăteni (Buzău county, vineyards), Brănești (Ilfov county, a former non-ferrous metallurgy plant) and Clinceni (Ilfov county, no intensive farming). They took samples from 7–10 wells in each village, at depths of 5–60 m, and kept the samples at 4 °C.

Nitrate was measured by two independent methods, by spectrophotometry and with a nitrate-sensitive electrode; the results were similar, and the authors used their mean. All seven wells in Matca and seven of the eight in Săhăteni were over the limit of 50 mg/l, with values up to 540.8 mg/l.

At Clinceni, the published mean of the eight wells was 49.78 mg/l, below the limit, but three wells exceeded it: 52.08, 50.03 and 66.58 mg/l. Excess nitrate is dangerous above all for babies, in whom it can cause methaemoglobinaemia; the study mentions 11 cases in Buzău county in 2007.

In short

  1. An advanced investigation starts from a real problem in health, the environment or biotechnology, turned into a question that can be answered.
  2. Random error varies in both directions and is reduced by repeats; systematic error shifts all the results in the same direction and is uncovered by calibration or by a second method.
  3. Precision shows how close repeated measurements are to one another; accuracy shows how close they are to the true value.
  4. The strategy can be an experiment, which changes a variable on purpose, or an observational study, which compares groups that already exist.
  5. An argument has a claim, evidence and reasoning, and discusses alternative explanations.
  6. The investigation ends with its limitations and with proposals for improvement.
WorksheetMy notebook

Common mistakes

  • WrongIf I repeat the measurement many times, I get rid of every error.

    RightRepeating reduces random error. A systematic error, such as an uncalibrated instrument, repeats itself in the same way in every measurement.

  • WrongThe mean for the village wells is below the limit, so the water from every well is fit to drink.

    RightThe mean hides the individual values. At Clinceni the published mean was 49.78 mg/l, but three wells out of eight were over the limit.

  • WrongIf the data are clear, the conclusion convinces on its own.

    RightThe conclusion has to be argued for: evidence, reasoning, alternative explanations ruled out. Semmelweis had good data and did not convince people without an accepted mechanism.

Check yourself

Seven questions. For the first six you see the explanation at once, with the paragraph of the lesson that supports it; for the written ones you compare with a good answer.

  1. A balance reads 2 g more than the true mass at every weighing. What kind of error is this, and how do you uncover it?

    Answer

    b) Systematic error; you uncover it by weighing an object of known mass. All the results are shifted in the same direction, so the error is systematic. Repeating gives the same extra every time; a standard weight brings it to light.

  2. Why did the researchers measure the nitrate in the wells by two different methods?

    Answer

    d) To check that a single method was not shifting the results systematically. Two independent methods that give similar results make a systematic error in either of them unlikely.

  3. What strategy did the study of the wells in the four villages use?

    Answer

    a) An observational study, comparing groups that already exist. The researchers changed nothing in the wells; they compared villages that already existed, with different activities.

  4. Four pupils measure the same 50 mm leaf with a ruler whose end is broken off and get 53, 53, 53 and 53 mm. What are their results like?

    Answer

    c) Precise but inaccurate. The values are identical to one another (precise), but all of them are far from the true value (inaccurate), because of a systematic error.

  5. By how many percentage points did yearly mortality in the First Clinic fall between 1846 (11.4%) and 1848 (1.3%)?

    percentage points
    Answer

    10.1 percentage points. The difference between two percentages is expressed in percentage points: 11.4 − 1.3 = 10.1.

  6. Sort the sources of error: random or systematic?

    Answer

    Random error: A hurried reading of the scale, sometimes too high, sometimes too low; A stopwatch started sometimes too early, sometimes too late.

    Systematic error: A thermometer that always reads 1 °C too low; An out-of-date reagent that gives values too low for every sample; A pipette that delivers slightly more than it should every time it is used.

    Systematic errors shift all the results in the same direction; random errors vary in both directions.

  7. A village council wants to find out whether the wells next to fertilised fields have more nitrate than those in the village centre. Propose the strategy of the investigation, two sources of error and one improvement.

    A good answer

    It is an observational study: several wells are chosen in each area, say eight each, sampled on the same day, in clean bottles, kept cold and coded so that the laboratory does not know the area. Sources of error: an uncalibrated instrument (systematic error), checked with a solution of known concentration, and repeated readings of the same sample that differ slightly from one another (random error), reduced by repeating the measurement and calculating the mean; the natural differences between wells and between days call, in turn, for more samples. Improvement: sampling again in another season, after the fields have been fertilised, and recording the depth of each well.

    A complete answer chooses the right strategy (observational), asks for several samples and coding, names one systematic and one random error with the way each is controlled, and proposes a concrete improvement.

Practical work for this lesson

Terms

random error
An error that makes repeated measurements vary in both directions; it is reduced by repeats.
systematic error
An error that shifts all the results in the same direction; it is uncovered by calibration or by a second method.
precision
How close repeated measurements of the same quantity are to one another.
accuracy
How close a measurement is to the true value.
observational study
An investigation that compares groups that already exist, without the researcher changing a variable on purpose.
argumentation
Supporting a claim with evidence and with the reasoning that links the evidence to the claim.
Sources and link to the programme

From the programme: “Stages of the advanced scientific investigation/method based on observations and experiments: defining a problem in a real health/environment/biotechnology context, formulating the hypothesis, setting the variables, identifying risks/sources of error, deciding on the experimental strategy, analysing the data obtained, drawing up and presenting the conclusions, arguing for the results, proposing new ideas to improve the investigation” (specialism, Natural Sciences). Specific competences addressed: 1.2, 2.2, 4.1, 4.2.