Tag Archives: Research

How studies are misleading

Posted on by

1

Cover of "Bad Science"

Cover of Bad Science

Academia

How many times do you hear the words, “studies show….”.  It’s amazing how the minute these magic words are uttered, we lie supine to whatever statistic or generalization follows.   Studies are vital to gaining wisdom because, assuming they are done properly, they provide us with valid and reliable generalizations upon which we can base our individual choices and public policy.  Without generalizations, all you have are platitudes.  Scientific generalizations however, require more than just validity and  reliability.  The extent to which any particular study is useful to a society is largely dependent upon its audience.  Studies must be scrutinized by a very logical and unbiased mind.  It takes a mind that is open to the premise yet maintains a bit of healthy skepticism.  But how do we create such minds?

I remember being taught the difference between fact and opinion in elementary school.  It was a simple concept to digest for my gradeschool

mind.  According to my 8-year-old daughter, “A fact you can prove and opinion is just something someone thinks.”  I listen to him with pride yet am terrified to know that if his formal education goes anything like mine did, that is the extent to which he will be taught about how to find truth in the world.  From here on out, he will be subjected to various studies and lead to take them at face value.  Sure college professors talk about challenging what you think you know to be true but that with which they wish you to challenge your  common sense is most likely the newest study, praised for it’s unlikely and counter-intuitive conclusions.  We are intoxicated with the esteem these studies seem to bring to their authors and seek to push the envelope and defy our own innate wisdom in search for novelty.

Clinical Practice

Imagine you are a member of some steering committee for a big-time HMO.  You are on a panel of fellow MDs discussing the screening criteria for STDs in pediatrics.  The factors to consider are as numerous as any clinical initiative can be; the confidentiality of the patient, the cost for screening, the emotional aspect of both the patient and their parents, the method of reporting, etc.  As you check your watch to see that lunch is 15 minutes away, your stomach rumbles.  You glance back up across the table and see that a colleague has this 8 page study in hand.  She makes the following claim as she slams it down on the table, “If we don’t screen everybody, we could miss 10% of our patients due to inaccurate self reporting on sexual behavior.”  “Wow! 10% is pretty substantial”, another doctor says.  You pick up the study and skim through it to read in the conclusion;

“More than 10% of young adults with a laboratory-confirmed positiveSTD result reported abstaining from sexual intercourse in the12 months before assessment and STD testing.”
You break for lunch and take a copy of the study to review for yourself.  The first thing you want to know is the total number of cases.  You look and find that the study was done with 14,012 pediatric patients.  Next you check to see that the researchers have controlled for age, race, socio-economics, etc.  They seem to have all their bases covered.  Then something occurs to you.  The claim stated that 10% of young adults with a laboratory-confirmed positive STD result reported no sexual intercourse in the 12 months before the study.  So what is the percentage of laboratory-confirmed positives to the complete sample?

N=14,012

946 tested positive for STDs.

946/14,012=0.07=    07% of the subjects tested positive

Now, of those who tested positive, the study finds that 10% falsely reported their sexual history.  Since the entire reason for reviewing this study is to see how many patients will be missed if your organization does not establish a policy of screening ALL pediatric patients for STDs you perform the following calculation:

964 tested positive

10% of 964 = 118

118/14,012 = 0.8%

Lunch is over and you go back in to the meeting and ask the budget committee if you are willing to take the resources from other clinical operations to screen all patients for STDs so that you will not miss 0.8% of the patients due to inaccurate self reporting.

The responsibility of the reader

There is a great burden of responsibility placed upon the reader of academic research.  One must realize that research is not free and is funded by any number of entities and interests.  Those who write the studies are usually paid professionals who have a masterful talent for reporting data to lead the reader to a desired conclusion.  There is no way of knowing if any of this was at work with the study offered in this example however if the reader hadn’t asked those other important questions regarding the true implications of the study, the hospital may have taken resources from a worthwhile cause to chase down 0.8% of the pediatric population.  This may or may not be the right decision but it must be made on the basis of the correct figures.

To conclude, I’ll quote one of my favorite thinkers:

“Studies either confirm common sense, or they’re wrong”   – Dennis Prager

Related articles

Sleep….then Move

Posted on by

0

Coffe break

Image by Francisco Javier Argel via Flickr

There are three major ways in which we can affect our bodies to a degree that is noticeable immediately; exercise, food and sleep. The first two are fairly obvious. If we run a marathon, we become tired. If we overindulge, we become lethargic. If we don’t get enough sleep, we drink a coffee and everything is all good…at least until 2pm. We are a people who don’t generally place enough value on getting a normal amount of sleep.
Let us define normal sleep. Some experts have prescribed a sliding scale based on age. I personally subscribe to the Ayurvedic definition of normal sleep. Sleep should take place between the hours of 10pm to 6am. During these hours our bodies slow down so that healing and regeneration can take place. From 10pm to 2am, the physical body is metabolizing, replenishing and repairing cells and tissues. From 2am to 6am the mind is digesting the days thoughts and emotions while restoring and preparing for the next day. This is why when you stay up too late, feel lethargic during the next day when you don’t sleep well in the early morning you wake up foggy in the brain.
The reasons we take sleep for granted are as subjective as shoe size however it is my opinion that one common factor comes in to play when we decide to sty up and make a dent in that DVR library at 10pm. The consequences of abnormal sleep are more subtle than the consequences of over-eating or over-exerting oneself. We are opportunistic beings from our brains all the way down to our mitochondria. We will prioritize our actions by weighing the pros and cons. If we can stay up a little later and finish that report that we could have started earlier, it may mean a promotion and a nicer car in due time. A mentor of mine, Dr. Suhas Kshirsagar often says, “We first spend our health on our wealth and then spend our wealth on our health”. Eight years in the pharmacy biz has made this abundantly clear to me.
When we don’t get enough sleep, our bodies make sacrifices in other areas. This is the noticeable change in the way we feel that I was talking about in the first few lines of this post. Consider this; nothing in our bodies is not constantly breaking down and being replenished. We “use up” a certain number of cells everyday in order to carry out the most basic functions. The body can not heal if it does not rest properly. Normal sleep is vitally important and necessary just to maintain the status quo. If anybody wanted to make any sort of improvement or change to their bodies, they would have to at least get normal sleep on a regular basis.

Related articles