Frequently Asked Questions

Building an experiment (4)

Day to day, symptoms can vary widely depending on a spectrum of inputs — food, sunlight, exercise, socializing, sleep — that have nothing to do with a given treatment. Extending the duration of each treatment block, boosting the number of treatment blocks, or increasing the frequency of reporting means experimenters gather more data. This can negate the effects of non-treatment effects, and boost an experiment’s statistical validity. Every experiment, though, depends on the experimenter’s tolerance for continuing, so most experiment designers compromise on a duration that seems to the person being tested.

Comment on this FAQ

Your email address will not be published. Required fields are marked *

Placebo effects can be powerful, so blinding can contribute to an experiment’s success. A compounding pharmacist can create treatments and placebos that look identical, or a trusted friend might help disguise nonpharmaceutical treatments and placebos.

Some single person experiments includes “blinding” ⁠— making each treatment appear the same to reduce a psychological biases for or against each version. (Knowing that a treatment is newer or a stronger dosage can boost its perceived effectiveness by an average of 9% versus a control.)

Doctors can instruct a compounding pharmacist to create a double blind by putting different treatments or strengths (or a placebo) into identical pills or liquids. The bottles can be sequentially labelled (for example 1, 2, 3, 4, 5, 6) so that neither you or the doctor know which treatment you’re taking in a given time period.

Once the experiment is over, the pharmacist can provide a key that reveals which code corresponded to which treatment. (This might reveal, for example that you’d taken six successive weeks of A, B, B, A, A, and finally B.)

Particularly for nonpharmaceuticals, some people ask a friend to help with blinding by distributing different treatments into identical containers. This is simple for oils⁠—for $6.99, you can buy 25 plastic dropper bottles or six glass dropper bottles. Funnels are available too. The same goes for ointments and creams⁠—five jars cost $10 on Amazon.

Funnels for blinding in a personal experiment for CBD.
Funnels for $6.49.

The friend might label each as A, B, C, D, E, F (one for each treatment period using the schedule you pick) being careful to separately note which treatment went into each container, and then pass the containers over to you.

(Note that, historically, not all experiments include blinding, particularly when it’s impossible to create seemingly identical versions of each treatment, for example when a treatment involves obvious physical alterations.)

Comment on this FAQ

Your email address will not be published. Required fields are marked *

Just like everybody is different, every treatment is different. Here are some links to trials using a variety of treatment block durations.

Melatonin: 2 weeks treatment blocks (sleep in Parkinsons); 4 weeks (sleep in alcohol dependence); duration of hospital stay (delirium in elderly patients); 30 days (inflammation in athletes); 21 days (sleep). More melatonin studies.

Caffeine: 2 and 4 week treatment blocks (insulin resistance);

Cannibidiol: 6 week treatment blocks; 12 weeks; single dose x 3; 4 weeks; 4 weeks; single dose; 10 days; 8 weeks. (More CBD studies.)

Comment on this FAQ

Your email address will not be published. Required fields are marked *

Including a placebo in an experiment helps tease out the often powerful psychological effects of being treated.

But often placebos are difficult or impossible to implement, particularly if the treatment has physical manifestations (tingling, scars) that lead to unblinding or there’s no compounding pharmacist is available.

In some cases, unblinded personal experiments without placebos have yielded the same results as large, expensive gold-standard trials with blinding.

Finally, its important to note that placebo effects can be both positive or negative.

  • I am glad for commenting to let you understand what a impressive experience my friend’s child gained studying the blog. She figured out a lot of pieces, most notably how it is like to have an awesome giving character to make other folks quite simply have an understanding of selected grueling things. You really surpassed people’s expected results. Thanks for supplying those beneficial, dependable, explanatory and as well as cool thoughts on that topic to Gloria.

  • Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    History (1)

    In the spring of 2019, we had been talking for nearly a year about building a software toolkit to help automate what many people already do informally⁠—track symptoms and statistically evaluate the effects of various treatments. As the idea started to jell, we started casting about for names for the toolkit.

    Though there are a variety of scientific names for the methods that the WWB toolkit borrows from⁠— single person trial, n-of-1 trial⁠— none of them inspired any enthusiasm.

    One day, headed to the car for a brief vacation, I realized I’d forgotten my blood thinner medications. (I had a pulmonary embolism last fall.) Walking out of my home holding two bottles of best selling blood thinners — Xarelto and Elequis — one in each hand, I was thinking about the wooziness that sometimes seemed to occur while I was on the blood thinners.

    I weighed both bottles and asked myself out loud, “which works best?”

    I knew, from reading and conversations, that this exact question is incredibly common for people struggling to find the right solution⁠—behavior, medicine, supplements⁠—for chronic health conditions. Luckily, the domain name was available!

    WhichWorksBest is a creation of Pressflex LLC, which has been around since 1998. The name WhichWorksBest? ties into the fairly literal, “just the facts” naming conventions that Pressflex has used for other ventures.

    For example, Pressflex was founded to help newspapers and magazines get online flexibly and affordably. Blogads, which we launched in 2002, was an automated service to help bloggers sell ads. Pullquote, launched in 2011, helps people store and share interesting quotes. AdBiblio, launched in 2014, helps book publishers advertise online. Racery, also launched in 2014, helps companies and charities build and host virtual races.

    One remaining puzzle. Very few companies — or zero? — have a question mark in their logos. If WhichWorksBest? ever ends up in the headlines, this may confuse readers. But the question mark also conveys the confusion and questioning that is at the heart of WhichWorksBest? WhichWorksBest? is a question you ask, not an answer we give. So, for now at least, the question mark remains.

    Category: History
  • I enjoy you because of your own labor on this blog. Ellie delights in managing investigation and it’s easy to see why. A number of us hear all about the compelling tactic you produce priceless guides through this blog and even recommend contribution from some others on the situation then our favorite daughter is discovering a lot. Take advantage of the rest of the year. You are always conducting a remarkable job.

  • I together with my pals have been following the nice information found on your web site while then I had an awful suspicion I never expressed respect to the web site owner for them. All of the young men became happy to study all of them and already have certainly been enjoying these things. Appreciation for genuinely simply considerate and for picking out this sort of amazing subjects most people are really desirous to learn about. My personal honest apologies for not expressing appreciation to you sooner.

  • Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    Theory (3)

    A seven point scale affords users both a neutral choice (4) and more granularity than a five point scale.

    There’s lots of discussion among researchers about which scale best allows people to report on symptoms or opinions.

    The consensus: whereas an even numbered scale forces people to err on the side or a positive or negative opinion (in a 1-6 scale, 3 is slightly negative and 4 is slightly positive), an odd-numbered scale allows for a “neutral” choice. So in a seven point scale 4 means neutral, neither bad or good.

    Five and seven point scales are commonly used in research because too many rating options can create confusion. It’s tough enough evaluating symptoms and making comparisons about symptom severity from day to day! We opted for a seven point because it allows for 50% more nuance and granularity than a five point scale.

    Category: Theory

    Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    Treatments⁠—both pharmaceutical and nonpharmaceutical⁠—are classically evaluated in random control trials (RTCs). Large, carefully selected groups of participants and their doctors are blinded to which patients are getting which treatment or control.

    In contrast, an n-of-1 trial looks only at the effects of different treatments on a single person. The trial periods and target outcomes are defined ahead of time. With the assistance of a compounding pharmacists, blinding and placebos are possible.

    The two types of trials have radically different purposes⁠—the RTC is to determine broad efficacy and average treatment effects, and the n-of-1 determines the effect on a single individual.

    “All RCTs do is show that what you’re dealing with is not snake oil,” according to William R. Shadish, PhD, a professor of psychological science at the University of California at Merced. “They don’t tell you the critical information you need, which is which patients are going to benefit from the treatment.”

    According to Dr. Sunita Vohra and Salima Punja, BSc:

    results from randomized controlled trials (RCTs), often considered the ‘gold standard’ of research evidence, are often not well suited to the realities of clinical practice given patient heterogeneity, comorbidities, and the use of multiple concurrent therapies. In fact, RCTs may exclude the majority of patients seen in routine clinical practice. In addition, evidence is lacking on the long-term effectiveness, comparative effectiveness, and additive effectiveness of many therapies for chronic conditions. This lack of relevant evidence can limit a clinician’s ability to make evidence-based decisions.

    Category: Theory

    Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    Short answer: done right, n-of-1 experiments for many treatments of chronic conditions are more scientifically valid for an individual than the generalized results of a large scale randomized control trial.

    It’s easy to see why many people, even some doctors, believe n-of-1 studies are “not statistically valid.”

    Many doctors weren’t exposed to the voluminous literature on N-of-1 studies in medical school or, if they were, quickly forgot the concept. Once they moved into clinical practice, they’ve been inundated, day after day, by drug reps justifying their drug’s superiority over others based on large parallel group random control trials (RCTs).

    “This study of 3,500 patients shows that our drug X results in a 17% improvement in symptom Y,” says the drug rep, pointing to the study’s title. Parallel group RCTs are often perceived as the “gold standard” for proving a drug’s efficacy and safety.

    In fact, statisticians have long argued that the results of RCTs are not generalizable for any given individual.

    First, the RCT’s participants often have nothing in common with a given patient, whether because of age, sex, fitness levels or other variables. Historically, women and minority groups in particular have been underrepresented in studies.

    Second, even if a study’s participants do appear to match the patient, research indicates that drug efficacy varies strongly from patient to patient, whether because of genetics, microbiome or behavior. Behind the headlines of all large studies reporting “average” benefits is compelling evidence that most drugs don’t work equally well for all people. This is called “heterogeneous treatment effects,” or HTC.

    “All RCTs do is show that what you’re dealing with is not snake oil,” according to William R. Shadish, PhD, a professor of psychological science at the University of California at Merced. “They don’t tell you the critical information you need, which is which patients are going to benefit from the treatment.”

    This means that, for an individual, a drug can’t be proven to be effective, much less optimal, unless it’s been systematically tested for that individual, ideally with crossovers, placebos and blinding. In short, an n-of-1 trial.

    In fact, it’s the trial-and-error drug experiments (formally called “trials of therapy”) used by most doctors to find the best drug for an individual patient that lack scientific rigor. “These so-called trials are unblinded, have no control, and involve no formal assessment of effectiveness, making them vulnerable to invalid conclusions about treatment response,” observe Dr. Sunita Vohra and Salima Punja, BSc.

    >> More on the difference between n-of-1 trials and large group RTCs.

    Category: Theory
  • Thank you so much for giving everyone remarkably memorable chance to read from this blog. It’s usually so amazing plus stuffed with fun for me personally and my office acquaintances to search your blog at a minimum 3 times every week to read through the new guidance you have. And indeed, I am just certainly astounded concerning the very good information you give. Certain 1 tips in this post are truly the simplest I’ve ever had.

  • Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    Validity (1)

    First, it’s important to remember that what works for one person frequently does not work for another person.

    Also realize that Facebook makes it possible to check on, if not prove, how legit someone is. Do you know people in common? Do they regularly post things that seem real? Do real people like those posts? How long have they been a member of Facebook groups you belong to?

    Category: Validity
  • I wanted to draft you this very little note just to thank you yet again for these pleasant thoughts you have discussed at this time. It was so generous with you to supply freely all that numerous people might have made available for an ebook to get some profit for their own end, even more so since you might well have tried it if you desired. These concepts as well worked to provide a good way to realize that some people have the identical passion really like my personal own to know the truth way more with respect to this issue. I believe there are thousands of more pleasant times ahead for individuals who looked over your website.

  • I happen to be commenting to let you understand of the beneficial experience my friend’s child experienced reading through your web site. She figured out several pieces, most notably what it’s like to have an amazing giving style to let others effortlessly understand selected complicated issues. You truly surpassed visitors’ expected results. I appreciate you for providing the great, healthy, informative and even cool tips about this topic to Mary.

  • Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    Load More

    Hi! Thinking about a personal experiment?

    I’m GuideBot, here to help you 24/7.

    Pick a Topic!

    Guidebot Image