The China Study Toppled – A Tale of the Confirmation Bias

by Darrin on July 9, 2010

china study toppledSome Great Weekend Reading for Ya…

If you are a longtime LMVM reader, you doubtless know that I stress the importance of nutrition first and foremost when it comes to men’s common goals of fat loss and muscle building. And unlike the U.S. Government, I believe that an optimal human diet consists of massive quantities of both meat and veg, while grains, dairy, and legumes should be consumed judiciously (if at all).

Although this isn’t a “Paleo blog” or a “low carb blog” in the strictest sense – my main goal is to help young, busy men build strong, attractive, and healthy bodies quickly and efficiently – you can certainly see where my sympathies lie. You get the highest leverage on your health, fitness, and wellness by including meat in your diet.

The Myth of the Plant-Based Diet

“Eat food. Not too much. Mostly plants.”
-Michael Pollan

Having a lifelong interest in all things health-related, I naturally stumbled into vegetarianism when I was younger. While I was only a STRICT vegetarian for less than a year, I leaned that way for a long, long time, eating very little meat for years.

After all, everyone KNOWS that meat causes cancer, heart disease, (insert your favorite Disease of Civilization here)… right?

I’d heard all the rationalizations of why vegetarianism is the natural human diet (the structure of our teeth, the length of our intestines, etc.) and was researching this further when I landed on the Beyond Vegetarianism site one day. Changed the way I see things forever.

Humans have never thrived on an all-plant diet. There’s always been at least SOME meat thrown into the mix. Our grunting, spear-toting forebears certainly ate plenty. And the healthy hunter-gatherer societies of today include nary a veg*n.

But still the myth persists: MEAT KILLS.

T. Colin Campbell and the China Study

A scientific justification for the superiority of plant-based diets relative to animal-based ones has been the holy grail of veg*ns forever. What started out as a personal choice based on ethical rather than health priorities has shifted over the years to be the overruling dogma.

Perhaps it started with Ancel Keys, who infamously cooked up data in The Seven Countries Study to justify the importance of low-fat diets, leading to The World’s Biggest Fad Diet, which still persists to this day, decades later.

Or maybe it was modern day gurus such as Dr. Dean Ornish and Dr. Andrew Weil, who sold millions of copies of their veg-leaning diet books. (Although the latter seems to be coming to his senses.)

But in an ocean of half-assed studies and inconclusive data stood one monumental achievement: The China Study.

The China Study is longtime veg*n Dr. T. Colin Campbell’s exhaustive and authoritative work detailing his findings studying the impact of animal protein and cholesterol on disease and mortality. Upon its release, it was heralded as The Definitive Proof That Meat Kills.

It has been critiqued many times by people skeptical of his interpretations (see Dr. Loren Cordain’s and Chris Masterjohn’s takes), but these people have largely been drowned out by the status quo who still believe it is indisputable that whole grains are a vastly superior food choice than a juicy steak.

That all just changed.

The China Study Debunked

As some of you have already heard, Denise Minger of Raw Food SOS has been poring over the raw data of The China Study and noticed something rotten in the state of Denmark. Namely, T. Colin Campbell’s conclusions do not logically follow from the data recorded.

I will steal NONE of Denise’s thunder and instead urge you to head on over to her site and give it a read yourself. Despite the glut of statistics and graphs, it is truly an engaging read. (Testament to Denise’s writing skillz, which I am justifiably jealous of.)

Here’s the link: China Study: Fact or Fallacy?

The Confirmation Bias Strikes Again

So now that The Death Star that is The China Study has been toppled, what are we to do with the ruins?

I’ll save the vitriol for others. Dr. Campbell was merely doing what we all do, albeit on an epic scale: seeing only what he wanted to see despite all evidence to the contrary. The confirmation bias is a well-established cognitive “blind spot” inherent to humans that makes it possible to make sense out of a messy, chaotic world. But at the expense of comprehending phenomena that threaten our egos.

The confirmation bias is the reason that it will be a long time until people finally realize that calories in minus calories out is NOT the primary cause of fat accumulation. Or that isolation exercises and body part splits are a waste of time when it comes to building muscle. But these “hacks” are still there for all the busy guys who want to take advantage of them.

If you are a vegetarian for personal / spiritual / ethical reasons, I support your choice. What I can’t get behind is the idea that this is the healthiest choice. I sincerely hope that this is just one more nail in the coffin of the low fat / low cholesterol / low meat / low calorie behemoth and a sign that people are more interested in basing their dietary decisions on what humans have thrived on for many generations rather than half-baked studies stigmatizing certain macronutrients and food groups.

That would be swell.

{ 4 trackbacks }

Canibais e Reis » Blog Archive » Artigos do RAW FOOD SOS sobre o China Study, o estudo que serve de suporte científico à dieta vegetariana
July 10, 2010 at 1:53 am
The China Study Toppled – A Tale of the Confirmation Bias | SEO Updates
July 10, 2010 at 2:13 am
The China Study Smackdown Roundup | Free The Animal
July 10, 2010 at 1:48 pm
Tweets that mention The China Study Toppled - A Tale of the Confirmation Bias -- Topsy.com
July 10, 2010 at 6:47 pm

{ 10 comments }

Raymond - ZenMyFitness July 9, 2010 at 7:58 pm

Yeah cool topic! many people become fanatical and lose sight of why eating less meat is good or bad for them.
I say try it for yourself i.e eat more/less/ no meat and see what happens .. After years of reading contradictory studies I’ve given up on unbiased info.
I think some people need more meat due to lack of mineral & vitamins.
I choose not to eat due to animal cruelty .. but I know if I did eat meat, my results in the Gym would increase considerably
Its great your post is like a Meat MythBuster!

Darrin July 10, 2010 at 10:29 am

Raymond,
I know that you aren’t a big meat guy, but I also know that it is primarily due to spiritual reasons and have always appreciated your “non-fanatical” approach to it. And jeez guy, by looking at your pics, you seem to be in ridiculously good shape! So you are obviously doing something right! As I’ve always said, QUALITY of food is the most important factor. A shift to vegetarianism often implies a shift from Real Food to Food Products. And there’s the rub.

john July 10, 2010 at 5:50 pm

Don’t be fooled people, Denise has misinterpreted raw data, just as many inexperienced “researchers” do. Denise is not qualified to read such data correctly.
Please refer to the use and misuse on pp. 54-82 of the China Project monograph.

The following is Dr Campbell’s rebuttal. The rest can be found http://www.vegsource.com/articles2/campbell_china_response.htm

” China Project results are no exception to these limitations of single experiments. It was very large, unique and comprehensive but it was observational (i.e., not interventional), simply observing things as they were at a single point in time. It provided an exceptionally large number of hypothetical associations (shown as statistically assessed correlations) that may indicate but does not prove cause and effect relationships. These unanalyzed correlations are considered raw or crude. It is highly unusual to find such ‘raw’ data in a scientific report because, in part, untrained observers may misunderstand such raw data.

For the monograph, we were somewhat uncertain whether to publish such raw data but decided to do so for two principle reasons. First, we wanted to make these data available to other researchers, while hoping that data misuse would not be a significant problem. Second, because these data were collected in rural China at a time when data reliability might have been questioned, we chose to be as transparent as possible. We discussed data use and misuse on pp. 54-82 of the China Project monograph that curiously was overlooked by Masterjohn and Jay’Y’.

Darrin July 10, 2010 at 7:41 pm

John,

Thanks for stopping by. As passionate as I am about my own views, I am always interested in hearing the other perspective. It’s this cognitive dissonance that truly leads to discovery and progress.

That being said, the defense that “Denise is not qualified to read such data correctly” is absurd. She seemed to anticipate this kind of response when she wrote in her conclusion:

In rebuttals to previous criticism on “The China Study,” Campbell seems to use his curriculum vitae as reason his word should be trusted above that of his critics. His education and experience is no doubt impressive, but the “Trust me, I’m a scientist” argument is a profoundly weak one. It doesn’t require a PhD to be a critical thinker, nor does a laundry list of credentials prevent a person from falling victim to biased thinking.

But if the quantity of formal education truly is the marker of how qualified a person is to re-analyze and re-interpret raw data, then you might like to know that Dr. Stephen Guyenet has called this “the most thorough review of the data I’ve seen so far. She raises some points about Campbell’s interpretation of the data that are frankly disturbing. As I like to say, the problem is usually not in the data– it’s in the interpretation.” Drs. William Davis and Kurt Harris have also voiced their satisfaction with Denise’s analysis.

The China Study was an epidemiological study, one that is suited more to the approach of “lets take all this data and see what trends we can find” rather than “I think such-and-such is correct, let me take this mountain of data and see how I can make it fit my hypothesis.” Denise took the former approach. Dr. Campbell, the latter. For more on how you can take these types of studies and arrive at whichever conclusion you desire, check out Dr. Harris’s excellent post: Polish a Turd and Find a Diamond?

To once again quote Denise:

“The China Study” is a compelling collection of carefully chosen data. Unfortunately for both health seekers and the scientific community, Campbell appears to exclude relevant information when it indicts plant foods as causative of disease, or when it shows potential benefits for animal products. This presents readers with a strongly misleading interpretation of the original China Study data, as well as a slanted perspective of nutritional research from other arenas (including some that Campbell himself conducted).

I have no doubt that Dr. Campbell has the best of intentions. But as I mentioned in my post, we are all constant victims of the confirmation bias in our lives. We only see what we want to see. But it’s important to know that half of what we know is wrong. The hard part is figuring out what half that is. And that’s why we need skeptics like Denise.

Dave - Fitness Training Tips July 12, 2010 at 6:46 pm

Darrin,
Very interesting post. I’m a big meat eater so any research that supports eating meat sounds good to me!
Dave

Darrin July 12, 2010 at 9:47 pm

@Dave:
No doubt. More reason to fire up the grill!

freelee July 13, 2010 at 5:09 am

Denise’s analysis of The China Study is heavily flawed and therefore invalid. Debunked by a cancer epidemiologist…

Here it is…
http://www.30bananasaday.com/group/debunkingthechinastudycritics/forum/topics/a-cancer-epidemiologist

The proper testing procedure as stated by an expert on analysing stats…
http://www.30bananasaday.com/group/debunkingthechinastudycritics/forum/topics/my-response-to-denises

Campbells response to Denise..
http://www.30bananasaday.com/group/debunkingthechinastudycritics/forum/topics/campbells-response-to-denises

Darrin July 13, 2010 at 6:47 am

Freelee,

Thanks for “stopping by.” I put that in quotes because, due to the cookie-cutter nature of your comment and complete lack of context in the discussion we are having here, I suspect that you, just like “John,” are spamming all the posts breaking Denise’s work. Nevertheless, I appreciate the links and have plenty to add later. But I need to run to work right now.

Darrin July 13, 2010 at 11:17 am

Alright, I’ll keep this brief as I have little expectation you’ll show back up here again. I’m responding primarily for my readers.

Had you read my post and my reply to John – perhaps even Denise’s post – you would have realized the redundancy of yours. (Although I appreciate the links. I’m a big fan of cognitive dissonance and the last thing I want is a site where all my readers agree with me.)

Epidemiological studies such as this can be used to prove almost any theory – as long as you know the right things to tweak. While they are extremely valuable in discovering new variables to test in the future, they are wholly inappropriate for making blanket conclusions, which is what Campbell did. (i.e. meat kills). Cambell is also guilty of taking the correlations he found and turning them into causations. (A big no no.)

I could go on and on but you’ll need to prove to me you’re actually going to engage in the conversation rather than spamming me. To all the 30 Bananas a Day crew: I am more than happy to debate this with you, here, in the comments section of this post. But unless you can prove that you are not spamming (i.e. relate to my post and the other comments here) I will no longer be accepting your comments.

Darrin July 17, 2010 at 4:04 pm

Quickie update:

Denise has posted her response to Dr. Campbell’s criticisms here: http://rawfoodsos.com/2010/07/16/the-china-study-my-response-to-campbell/

C’mon 30BAD peeps! You have the time to spam the hell out of this post (Thank you Akismet!), can’t you drop one thoughtful and relevant comment down while you are over here?

Comments on this entry are closed.

Previous post:

Next post: