Monday, May 16, 2011

Meta-Research

...or the research of research, as I see it.  I recently read an article published in PLoS about how most published research is actually false (link). 

This very idea makes me cringe.  If that's true, why do we even bother?  Why spend enormous amounts of money to support something that's false? 

The paper doesn't really address either of those points, but rather it talks about how it can make such a claim.  Statistically, it is difficult to confirm things without absolutely enormous amounts of data.  Of course, getting data sets that are large enough for arbitrary experiments can range from difficult to impossible, with infeasibility being common.  This can put the statisticians at odds with the scientists.  The author's argument drives at this, pointing out that the data sets usually used are not large enough to be able to get a statistically valid answer. 

There is another problem.  People are people.  We are inherently biased.  It has been said that data is objective, and I used to believe that was true, at least in theory.  But then the question was posed to me: why don't scientists measure _everything measurable_ regarding an experiment?  Of course, this would mean an enormous amount of data, of which most of it is probably irrelevant.  But do we really know it's irrelevant?  The answer is no.  Our bias isn't shown so much in what we measure, but rather in what we choose not to measure - those things we think are irrelevant. 

Research costs money.  This usually means getting a grant.  Getting a grant usually means convincing someone that your research is going to do some good, be it cure a disease or (more commonly) make money.  With that in mind, why would anyone pay any amount of money for an application that reads "We want to do X.  We drew it out of a hat.  We have no idea what it does, and we have no idea what could come of this."  That is mostly unbiased (who put the ideas in the hat?).  It is also the least convincing argument I have ever heard for giving someone money. 

Now try "We want to research X, because it seems to have an effect on weight retention.  If this is true, we could develop an effective drug for weight loss."  Now we have something profitable.  But here's the problem: everyone involved wants it to be true.  More than likely, even someone working within ethical bounds is going to act differently when the desired outcome is known ahead of time.  I've been told repeatedly that one should never do data analysis until all the data is in.  However, we do not do this.  I have watched people stare at long running experiments that appear to deviate from expectations.  Frequently the target is personified, "Why are you putting a band there?  You're supposed to put it over here!"  I do this type of thing myself.  We already know what the experiment is "going" to do; we just need the formality of it actually doing it.

All in all, I think the paper is particularly interesting.  It gives a feel for how heterogeneous science really is, and it illustrates the ever-present (though shunned) human element.

Research shows that most research is wrong.

Monday, May 9, 2011

Vaccinations and Autism

And now for something completely different.  I read the original paper that supposedly linked autism to the measles/mumps/rubella (MMR) vaccine (link).  I know that this can get to be a heated topic, but for the moment I'm going to try to focus on the paper itself.  (Of course I'm going to be biased, but I'll try not to be!)

The paper suggests that a new disorder has been discovered.  Characteristic of this disorder is a combination of inflammatory bowel disease (IBF)-like symptoms, combined with autism-like symptoms.  The most notable feature of this disorder is that sufferers consistently presented with it between 24 hours and 18 months of receiving the MMR vaccine.  Most sufferers presented with symptoms within two weeks.  Such a disorder would be quite interesting, as the gastrointestinal tract and brain are two very different areas.  The author's original data in support of this disorder was a case study of 12 people.  Shortly after the paper was published with the original 12, an additional 40 patients were observed, of whom 39 were found to have this new syndrome.

Those are the facts, as presented by the authors.  Without going beyond the paper, this is not very convincing data of a new disorder.  Within the paper itself, there is only complete data presented for the original 12.  Of these 12, there is still considerable variability between patients.  Additionally, there is no control group; these 12 were hand picked by the authors.  The authors openly acknowledge this.  This was published as an "Early Report", and was more or less intended to be a springboard from which further research could be conducted.  To directly quote the paper, "We did not prove an association between measles, mumps, and rubella vaccine and the syndrome described."  Though the evidence suggests an association, there is simply not enough data to be able to make a scientifically valid determination.  Even if there is sufficient data to back an association, then one must determine if the relationship is causative or merely correlation.  (For example, when hot cocoa drinking is up, the crime rate goes down.  The reason is that it's typically cold when people drink hot cocoa, and the crime rate is known to drop in cold weather.)  Medical case studies need hundreds if not thousands of patients to be able to draw any hard and fast conclusions, and 12 patients is not enough to make such a claim.

Now I'll go beyond the paper.  For one, the main author (Dr. Wakefield) was covertly being paid by a law firm that was intending to sue the MMR vaccine manufacturers.  This is a conflict of interest.  Generally, conflicts of interest are rare in published research.  If they exist at all, they should be openly acknowledged.  (Here is a link to a paper with an open acknowledgment of a conflict of interest.)  This is a red flag.  Science is supposed to be as objective as possible, but with a conflict of interest it can be disadvantageous to be objective.

The more troubling problem is that most of the data itself is just plain not true.  Although 10/12 patients were listed as having something classifiable as autism (9/12 if you ignore data with question marks next to it), it was revealed that 3 of them never had a formal diagnosis.  Only a professional can make such a diagnosis.  Many of the symptoms of autism appear in other disorders, and only someone skilled in seeing all these disorders can actually make this judgment.  (I'm sorry, you cannot diagnose yourself as having a complex disorder just by reading a few pages on Wikipedia.)  As such, this is fraud. 

Another point is that earlier drafts of the paper used lengthier values for the time between exposure to MMR and first signs of symptoms.  As it came closer to the final draft, these time intervals shrank dramatically.

A third point is that much of the data was acquired not directly by doctors at the time of visit, but rather by parents at other times.  In the case of one of the children, such data was not acquired until 2 1/2 years until after symptoms first appeared.  For something as complex as autism, nonspecific data acquisition is not sufficient.  There are particular things that professionals look for, preferably directly as opposed to through a medical file.

I could go on and on about the different kinds of fraud and deception that occur in this paper.  A complete description of all these things can be found in here.  Note that this is from BMJ, which is a peer-reviewed source of legitimate medical information.  This is not some random website that some anonymous person made.  I must make that point clear, as there is a lot of misinformation on the Internet regarding this situation.

There have been a substantial number of follow-up peer-reviewed publications that have shown no link between autism and vaccinations, including this one.  However, the damage has already been done.  Many members of the general public think that there is a link because of this paper.  It has left a bad taste in people's mouths, with big bad science coming along to give our children autism.  This blog post is just going to be part of the fodder in this battle, which will likely continue without merit for years to come. 

People who still think there is a link will likely associate me with some evil corporate machine, and dismiss me.  Fine.  It would not be the first time someone has written me off that easily.  Let's assume there is a link, that this paper was correct, that it should never have been retracted, and this is all part of some conspiracy to cover the truth.  So if that's all true, then why does no one relate vaccinations to inflammatory bowel disease?  The bulk of the data of the paper is in support of IBF, not autism.  Dr. Wakefield is neither a psychologist nor a pediatrician, though he does specialize in the gastrointestinal tract.  The paper is not suggesting a link between autism and MMR - it is suggesting a link between autism, MMR and IBF.  It brands the combination of these three under a new disorder.  Removing one element means something else entirely, something the authors were not discussing.  In other words, if one believes what this paper is claiming, then it is self-contradictory to say that there is a link between autism and MMR without IBF involved.  As to how it happened to be autism and not IBF that was picked up by the media I'll never know.

Monday, May 2, 2011

Recycling: It Can Save Your Life

...assuming you're a lung cell.  I recently read an article published in the Public Library of Science (link) about how Pseudomonas aeruginosa infects people.  The bacterium can infect the lungs of people with other preexisting lung conditions, including pneumonia and chronic obstructive pulmonary disease (COPD).

Pseudomonas aeruginosa is an interesting infection, mostly because it requires a bit of sophistication on the part of the bacterium.  In the lungs, there is a protective mucous membrane that coats the outer layer of cells.  This outer layer of cells is known as the epithelium.  This mucous prevents most everything that is foreign to the lungs from directly contacting the epithelium, which can prevent many kinds of damage and infection.  Pseudomonas aeruginosa can't break through this layer, so it devises a strategy: send specially manufactured vesicles that can.  These vesicles have proteins on the surface that allow them to bind and fuse with cells in the epithelium, and they contain proteins that cause cellular change.  They are somewhat analogous to so called "bunker buster" bombs, which are able to penetrate a formidable outer shell and deliver a payload to the inside of the structure.  Only these are released with little guidance.


As for the payload, Pseudomonas aeruginosa causes a slight but severe change in infected cells.  In healthy cells there is a protein, namely CFTR, that regulates the amount of mucous there is in the lungs.  The protein must reside on the surface of cells to have any effect.  As part of normal cellular activities, this protein is occasionally ubiquitinated, meaning a ubiquitin group is bound to it.  This triggers a pipeline of events to occur.  The ubiquitinated protein is first sequestered from the cell membrane.  It then can follow one of two paths.  In one path, the ubiquitin group is removed, and the protein returns to the cell membrane.  In the other path, the ubiquitin group remains bound, and the protein is eventually degraded.  In healthy cells, these two paths run in tandem.  This is necessary to remove CFTR proteins from the membrane that no longer function, and are essentially just wasting space on the membrane.

What was previously known is that Pseudomonas aeruginosa infection somehow selectively shuts down the path that causes CFTR to return to the cell membrane.  As such, all the sequestered CFTR ends up being degraded.  The cell ends up degrading more CFTR than it can spare, and proper function is lost.  Without CFTR to regulate mucous properly, mucous builds up.  This is beneficial to Pseudomonas aeruginosa, as the once protective mucous ends up being its home, but this is at the detriment of its victim.  This mucous buildup is how people can literally drown in their own lung fluids, not to mention that it makes for a friendly environment for other opportunistic pathogens.

This paper investigated exactly how Pseudomonas aeruginosa is able to shut down the recycling pathway, forcing all ubiquitinated CFTR to be degraded.  The authors found that the vesicle payload contains a protein called Cif.  Although they were unable to determine exactly how, they found that Cif prevents the enzyme that deubiquitinates CFTR from functioning properly.  The reason why is somewhat complicated.  There is another protein, namely G3BP1 that normally inhibits the deubiquitination enzyme from function.  This protein is naturally occurring in lung cells, and it is presumably necessary for normal function.  G3BP1 can bind to the deubiquitination enzyme, temporarily preventing it from functioning.  In healthy cells, G3BP1 does not bind with very high affinity, presumably without other naturally occurring factors to help it along, so the net effect on the deubiquitination enzyme is minimal.

This is where Cif comes in for infected lung cells.  Cif stabilizes the interaction between G3BP1 and the deubiquitination enzyme, preventing the enzyme from functioning for much longer than with G3BP1 alone.  The effect is that the overwhelming majority of ubiquitinated CFTR never ends up getting deubiquitinated, as the deubiquitination enzyme has been inhibited by the interaction between G3BP1 and Cif.

I have a few questions regarding this mechanism, which could make for good future work.  For one, I suspect that some people are naturally immune to Pseudomonas aeruginosa infection, simply because they have mutations in either G3BP1 or the deubiquitination enzyme that prevent Cif from binding well.  It should be possible to conduct a clinical study on people with preexisting lung disorders, looking for those who for some reason never develop Pseudomonas aeruginosa infections, despite the significantly likelihood.

I also think that knowledge of this mechanism could lead to a novel drug treatment that prevents Cif from working properly.  Such a drug would somehow induce a conformational change in Cif that would prevent its proper binding to G3BP1.

The overall infection mechanism could be exploited for other purposes, as well.  Classically, specific drug delivery is a problem.  However, Pseudomonas aeruginosa is able to release vesicles that seem specific to lung tissues and contain specific payloads for said tissues.  With genetic engineering, it should be possible to change the payload to be whatever is necessary at the moment.  The result would be a targeted drug delivery system, injecting a specific drug into a specific tissue at a (below) microscopic level.  Perhaps we could even deliver an anti-Cif drug via the same mechanism used to inject Cif in the first place, of all ironic things.