Challenge Accepted: Science in the Northern Gateway Pipeline Process

By Sarah Boon

Earth & Environmental Science editor

A couple of weeks ago, our science policy editors Karine Morin and Pascal Lapointe discussed the Northern Gateway Pipeline (NGP) decision, and challenged bloggers to address the science behind the decision itself. Several Canadian science bloggers stepped up to the plate – David Ng and Meredith Hanel – though there were no responses in the Earth & Environmental Science category.

So this week your Earth & Environmental Science editor is taking on the science behind (or lack thereof) the NGP decision, specifically the concerns outlined in the letter that 300 scientists sent to the Prime Minister opposing the review process, which was completed by a three-person Joint Review Panel (JRP) mandated by the National Energy Board (NEB) and the Minister of the Environment.

Why is this an environmental science topic? Because the key risks of the NGP are primarily environmental: impacts on species, risks of oil spills, effects on climate change…the list is longer than you might think.

Note that the scientists didn’t oppose the approval itself, but the analysis the JRP used to support that decision. You may be asking: why do scientists care? They care because the government insisted that the approval process was science-based – and who better to assess that than scientists themselves? They know that science is a key component of public policy, along with social and economic considerations. If even one leg of this three-legged stool is out of whack, you end up with a very shaky chair to sit on.

So how does science work? Well, imagine you’re a scientist who’s just finished a huge project. You write a paper that describes what you did and why you think it’s important, and you send it off to be published in a prestigious journal in your research field. They send it to some reviewers to get an external opinion on just how good your research is. Those reviewers consider many aspects of your study to determine whether or not it’s good science, and whether or not it should be published.

You could consider the JRP’s analysis to be just like a scientific study – trouble is, it would never get published because it wouldn’t hold up to scientific scrutiny.

The 300 scientists concerns centre on questions of scientific method, which are key considerations that scientists take into account when reviewing research in their field and determining whether or not it’s good science. Some of the questions scientists ask that are applicable here include:

1. How well was the study structured?
2. Did the researchers use valid data and was it analyzed appropriately?
3. Do their results fit the data (and results from other peer-reviewed science)?
4. Are the study conclusions valid based on the evidence presented?

So having read the letter and had a bit of trouble figuring out exactly what was going on, you might ask what are the flaws in the JRP’s work? Well we’re here to translate it for you.

1. Study design:

The JRP only looked at a small set of risks associated with the pipeline. They didn’t take all the risks into account because many of them are uncertain (for example, how diluted bitumen will act if spilled in salt water, or the likelihood of increased greenhouse gas emissions from enhanced oil sands development). Instead of thinking about how they could realistically assess those risks, the JRP classified them as unimportant. On the flip side, they looked at a lot of benefits of the project – things like jobs, trade ties with Asia, and effects on the Canadian economy. All of these benefits were classified as important.

This introduces bias into the project because the risks are underrepresented, while the benefits are overrepresented.

2. Data used in the study:

The JRP’s study used information from the company building the project to do their study, but didn’t balance that with info from research done by other groups (scientists, public interest groups, governments, etc.). They took the company’s data as the truth, without testing its validity by comparing it to other studies. This leaves out a lot of data that would be very useful in completing this project, which means the results aren’t based on the most accurate and reliable information.

3. Do the results fit the data?:

Not only did the data used in JRP’s analysis come from the company building the pipeline, but their results contradicted evidence from the government’s own scientists. So the results they got based on their biased data don’t match with results that would have been derived using other data (see #2).

4. Are the study conclusions valid?:

By now we can fairly easily say no. If the study design was flawed (looking only at a few risks but many benefits), the data used was biased (and data from the literature were missing), and the results didn’t match results found in the existing literature – then they can’t draw any reliable conclusions from the study.

In science, this means the research wouldn’t be published without some serious revisions, or it would be abandoned in favour of a new approach. In this case, however, the study was used to define public policy. This is what troubles scientists the most: that a flawed analysis is being used to make such a major decision.

So next time someone says that the NGP decision was science-based, ask them to think about it a little more carefully, and you might come to the same conclusion as those 300 scientists.

————–

Some recent highlights from the Environmental and Earth Science feed on Science Borealis:

• Highly relevant to this conversation is Graham Young’s post at Ancient Shore, which tackles why the public might not connect with the geosciences, and the balance between environmental issues and resource extraction.
• Graphic arts meets earth science in Elisabeth Kosters’ interesting post exploring M.C. Escher’s connection with geology.
• Dirk Steinke at DNA Barcoding highlights diversity within the Fungi kingdom and the identification of a `new’ species of mushroom found in a grocery store package.
• At Watershed Moments, Sarah Boon explains the complex interplay of environmental processes in Fish, Forests, and Snow, and also shares her thoughts about disciplined writing.
• The Saskatchewan Research Council provides an overview of why it’s important to test drinking water within the home in its blog From the Lab to the Loading Dock.
• Finally, the Nature Conservancy of Canada is running a Wildlife World Cup where readers have been voting on their favourite Canadian species. The semi-finals pit the Burrowing Owl against the Bobcat, and Wood turtle versus the Karner Blue butterfly. There is still time to vote this week!