Frequently Asked Questions

Building an experiment (4)

Day to day, symptoms can vary widely depending on a spectrum of inputs — food, sunlight, exercise, socializing, sleep — that have nothing to do with a given treatment. Extending the duration of each treatment block, boosting the number of treatment blocks, or increasing the frequency of reporting means experimenters gather more data. This can negate the effects of non-treatment effects, and boost an experiment’s statistical validity. Every experiment, though, depends on the experimenter’s tolerance for continuing, so most experiment designers compromise on a duration that seems to the person being tested.

Comment on this FAQ

Your email address will not be published. Required fields are marked *

Placebo effects can be powerful, so blinding can contribute to an experiment’s success. A compounding pharmacist can create treatments and placebos that look identical, or a trusted friend might help disguise nonpharmaceutical treatments and placebos.

Some single person experiments includes “blinding” ⁠— making each treatment appear the same to reduce a psychological biases for or against each version. (Knowing that a treatment is newer or a stronger dosage can boost its perceived effectiveness by an average of 9% versus a control.)

Doctors can instruct a compounding pharmacist to create a double blind by putting different treatments or strengths (or a placebo) into identical pills or liquids. The bottles can be sequentially labelled (for example 1, 2, 3, 4, 5, 6) so that neither you or the doctor know which treatment you’re taking in a given time period.

Once the experiment is over, the pharmacist can provide a key that reveals which code corresponded to which treatment. (This might reveal, for example that you’d taken six successive weeks of A, B, B, A, A, and finally B.)

Particularly for nonpharmaceuticals, some people ask a friend to help with blinding by distributing different treatments into identical containers. This is simple for oils⁠—for $6.99, you can buy 25 plastic dropper bottles or six glass dropper bottles. Funnels are available too. The same goes for ointments and creams⁠—five jars cost $10 on Amazon.

Funnels for blinding in a personal experiment for CBD.
Funnels for $6.49.

The friend might label each as A, B, C, D, E, F (one for each treatment period using the schedule you pick) being careful to separately note which treatment went into each container, and then pass the containers over to you.

(Note that, historically, not all experiments include blinding, particularly when it’s impossible to create seemingly identical versions of each treatment, for example when a treatment involves obvious physical alterations.)

Comment on this FAQ

Your email address will not be published. Required fields are marked *

Just like everybody is different, every treatment is different. Here are some links to trials using a variety of treatment block durations.

Melatonin: 2 weeks treatment blocks (sleep in Parkinsons); 4 weeks (sleep in alcohol dependence); duration of hospital stay (delirium in elderly patients); 30 days (inflammation in athletes); 21 days (sleep). More melatonin studies.

Caffeine: 2 and 4 week treatment blocks (insulin resistance);

Cannibidiol: 6 week treatment blocks; 12 weeks; single dose x 3; 4 weeks; 4 weeks; single dose; 10 days; 8 weeks. (More CBD studies.)

Comment on this FAQ

Your email address will not be published. Required fields are marked *

Including a placebo in an experiment helps tease out the often powerful psychological effects of being treated.

But often placebos are difficult or impossible to implement, particularly if the treatment has physical manifestations (tingling, scars) that lead to unblinding or there’s no compounding pharmacist is available.

In some cases, unblinded personal experiments without placebos have yielded the same results as large, expensive gold-standard trials with blinding.

Finally, its important to note that placebo effects can be both positive or negative.

  • I actually wanted to make a small comment so as to say thanks to you for those unique advice you are writing on this site. My particularly long internet look up has at the end of the day been compensated with reputable concept to write about with my classmates and friends. I ‘d suppose that many of us visitors actually are very fortunate to live in a wonderful website with very many perfect individuals with beneficial techniques. I feel pretty privileged to have used the website page and look forward to so many more thrilling minutes reading here. Thanks once again for a lot of things.

  • I would like to show my respect for your generosity supporting men and women who absolutely need help on this one concept. Your personal dedication to getting the message all over ended up being amazingly interesting and has frequently permitted some individuals like me to achieve their desired goals. Your own invaluable help and advice means a lot a person like me and somewhat more to my office workers. Thank you; from all of us.

  • My husband and i have been very happy Edward could conclude his analysis via the precious recommendations he was given using your web page. It’s not at all simplistic to simply continually be giving away strategies which usually the others might have been trying to sell. Therefore we understand we’ve got the website owner to appreciate because of that. Those illustrations you’ve made, the easy web site navigation, the friendships you can help to engender – it’s most astonishing, and it’s really assisting our son in addition to us do think this content is satisfying, and that’s really fundamental. Thanks for the whole lot!

  • Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    History (1)

    In the spring of 2019, we had been talking for nearly a year about building a software toolkit to help automate what many people already do informally⁠—track symptoms and statistically evaluate the effects of various treatments. As the idea started to jell, we started casting about for names for the toolkit.

    Though there are a variety of scientific names for the methods that the WWB toolkit borrows from⁠— single person trial, n-of-1 trial⁠— none of them inspired any enthusiasm.

    One day, headed to the car for a brief vacation, I realized I’d forgotten my blood thinner medications. (I had a pulmonary embolism last fall.) Walking out of my home holding two bottles of best selling blood thinners — Xarelto and Elequis — one in each hand, I was thinking about the wooziness that sometimes seemed to occur while I was on the blood thinners.

    I weighed both bottles and asked myself out loud, “which works best?”

    I knew, from reading and conversations, that this exact question is incredibly common for people struggling to find the right solution⁠—behavior, medicine, supplements⁠—for chronic health conditions. Luckily, the domain name was available!

    WhichWorksBest is a creation of Pressflex LLC, which has been around since 1998. The name WhichWorksBest? ties into the fairly literal, “just the facts” naming conventions that Pressflex has used for other ventures.

    For example, Pressflex was founded to help newspapers and magazines get online flexibly and affordably. Blogads, which we launched in 2002, was an automated service to help bloggers sell ads. Pullquote, launched in 2011, helps people store and share interesting quotes. AdBiblio, launched in 2014, helps book publishers advertise online. Racery, also launched in 2014, helps companies and charities build and host virtual races.

    One remaining puzzle. Very few companies — or zero? — have a question mark in their logos. If WhichWorksBest? ever ends up in the headlines, this may confuse readers. But the question mark also conveys the confusion and questioning that is at the heart of WhichWorksBest? WhichWorksBest? is a question you ask, not an answer we give. So, for now at least, the question mark remains.

    Category: History
  • I am glad for writing to make you understand what a useful encounter my cousin’s daughter had reading through your blog. She came to understand numerous details, most notably how it is like to have a great giving style to make the rest just fully grasp a number of specialized subject matter. You actually did more than people’s desires. Thank you for rendering the essential, dependable, educational not to mention fun tips on this topic to Julie.

  • Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    Theory (3)

    A seven point scale affords users both a neutral choice (4) and more granularity than a five point scale.

    There’s lots of discussion among researchers about which scale best allows people to report on symptoms or opinions.

    The consensus: whereas an even numbered scale forces people to err on the side or a positive or negative opinion (in a 1-6 scale, 3 is slightly negative and 4 is slightly positive), an odd-numbered scale allows for a “neutral” choice. So in a seven point scale 4 means neutral, neither bad or good.

    Five and seven point scales are commonly used in research because too many rating options can create confusion. It’s tough enough evaluating symptoms and making comparisons about symptom severity from day to day! We opted for a seven point because it allows for 50% more nuance and granularity than a five point scale.

    Category: Theory

    Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    Treatments⁠—both pharmaceutical and nonpharmaceutical⁠—are classically evaluated in random control trials (RTCs). Large, carefully selected groups of participants and their doctors are blinded to which patients are getting which treatment or control.

    In contrast, an n-of-1 trial looks only at the effects of different treatments on a single person. The trial periods and target outcomes are defined ahead of time. With the assistance of a compounding pharmacists, blinding and placebos are possible.

    The two types of trials have radically different purposes⁠—the RTC is to determine broad efficacy and average treatment effects, and the n-of-1 determines the effect on a single individual.

    “All RCTs do is show that what you’re dealing with is not snake oil,” according to William R. Shadish, PhD, a professor of psychological science at the University of California at Merced. “They don’t tell you the critical information you need, which is which patients are going to benefit from the treatment.”

    According to Dr. Sunita Vohra and Salima Punja, BSc:

    results from randomized controlled trials (RCTs), often considered the ‘gold standard’ of research evidence, are often not well suited to the realities of clinical practice given patient heterogeneity, comorbidities, and the use of multiple concurrent therapies. In fact, RCTs may exclude the majority of patients seen in routine clinical practice. In addition, evidence is lacking on the long-term effectiveness, comparative effectiveness, and additive effectiveness of many therapies for chronic conditions. This lack of relevant evidence can limit a clinician’s ability to make evidence-based decisions.

    Category: Theory

    Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    Short answer: done right, n-of-1 experiments for many treatments of chronic conditions are more scientifically valid for an individual than the generalized results of a large scale randomized control trial.

    It’s easy to see why many people, even some doctors, believe n-of-1 studies are “not statistically valid.”

    Many doctors weren’t exposed to the voluminous literature on N-of-1 studies in medical school or, if they were, quickly forgot the concept. Once they moved into clinical practice, they’ve been inundated, day after day, by drug reps justifying their drug’s superiority over others based on large parallel group random control trials (RCTs).

    “This study of 3,500 patients shows that our drug X results in a 17% improvement in symptom Y,” says the drug rep, pointing to the study’s title. Parallel group RCTs are often perceived as the “gold standard” for proving a drug’s efficacy and safety.

    In fact, statisticians have long argued that the results of RCTs are not generalizable for any given individual.

    First, the RCT’s participants often have nothing in common with a given patient, whether because of age, sex, fitness levels or other variables. Historically, women and minority groups in particular have been underrepresented in studies.

    Second, even if a study’s participants do appear to match the patient, research indicates that drug efficacy varies strongly from patient to patient, whether because of genetics, microbiome or behavior. Behind the headlines of all large studies reporting “average” benefits is compelling evidence that most drugs don’t work equally well for all people. This is called “heterogeneous treatment effects,” or HTC.

    “All RCTs do is show that what you’re dealing with is not snake oil,” according to William R. Shadish, PhD, a professor of psychological science at the University of California at Merced. “They don’t tell you the critical information you need, which is which patients are going to benefit from the treatment.”

    This means that, for an individual, a drug can’t be proven to be effective, much less optimal, unless it’s been systematically tested for that individual, ideally with crossovers, placebos and blinding. In short, an n-of-1 trial.

    In fact, it’s the trial-and-error drug experiments (formally called “trials of therapy”) used by most doctors to find the best drug for an individual patient that lack scientific rigor. “These so-called trials are unblinded, have no control, and involve no formal assessment of effectiveness, making them vulnerable to invalid conclusions about treatment response,” observe Dr. Sunita Vohra and Salima Punja, BSc.

    >> More on the difference between n-of-1 trials and large group RTCs.

    Category: Theory
  • A lot of thanks for every one of your work on this web site. My daughter delights in carrying out research and it’s easy to understand why. My partner and i hear all relating to the dynamic method you make very helpful tips via your web blog and as well foster contribution from website visitors about this point so my princess is certainly discovering a great deal. Take advantage of the rest of the new year. You are always performing a really good job.

  • Thank you for all of the work on this blog. My niece take interest in going through research and it’s really easy to see why. We learn all of the dynamic tactic you render both interesting and useful items by means of the website and as well as attract contribution from the others on the point so my simple princess is truly discovering a great deal. Have fun with the rest of the year. You’re conducting a stunning job.

  • Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    Validity (1)

    First, it’s important to remember that what works for one person frequently does not work for another person.

    Also realize that Facebook makes it possible to check on, if not prove, how legit someone is. Do you know people in common? Do they regularly post things that seem real? Do real people like those posts? How long have they been a member of Facebook groups you belong to?

    Category: Validity
  • I simply needed to thank you so much once more. I am not sure the things that I might have undertaken in the absence of these thoughts revealed by you regarding that theme. It truly was a very traumatic circumstance in my view, but coming across your well-written approach you treated it made me to weep for contentment. Extremely grateful for your work and thus wish you know what a powerful job that you are getting into instructing people today with the aid of your web blog. I’m certain you’ve never encountered all of us.

  • I want to show my respect for your kind-heartedness in support of persons that absolutely need help on in this content. Your personal commitment to getting the solution all around had become pretty good and have all the time allowed women much like me to attain their dreams. Your personal valuable information means a great deal a person like me and a whole lot more to my office workers. Best wishes; from everyone of us.

  • I wanted to create you a little bit of note to be able to give many thanks once again for your superb techniques you have provided on this site. It was simply pretty open-handed of people like you to deliver without restraint exactly what a few people could possibly have offered for sale for an e-book to help make some money on their own, certainly seeing that you could possibly have tried it if you ever decided. Those solutions also acted to be the great way to fully grasp other people have the same dream really like my own to understand more and more concerning this issue. I think there are many more pleasant situations up front for individuals who find out your blog post.

  • My husband and i have been absolutely ecstatic Ervin could round up his investigation because of the ideas he acquired while using the site. It is now and again perplexing to just find yourself giving away concepts many people might have been selling. And we all fully understand we have you to give thanks to for that. The type of illustrations you made, the simple website menu, the friendships you help to engender – it is mostly unbelievable, and it is facilitating our son in addition to us understand that article is pleasurable, and that’s particularly fundamental. Thanks for all the pieces!

  • I am only commenting to let you know of the exceptional experience my friend’s girl experienced going through your web site. She learned lots of pieces, which include how it is like to have a wonderful teaching nature to make many people without problems know precisely a variety of impossible issues. You truly surpassed her desires. Many thanks for offering the powerful, safe, edifying and also unique thoughts on the topic to Sandra.

  • Comment on this FAQ

    Your email address will not be published. Required fields are marked *

    Load More

    Hi! Thinking about a personal experiment?

    I’m GuideBot, here to help you 24/7.

    Pick a Topic!

    Guidebot Image