
Show HN: A Platform For Self-Experimentation - chrisrxth
http://doubleblinded.com/
======
dangrossman
It's hard to do a test like this when there's a significant chance you're
buying nothing but powdered rice regardless of what the bottle says.

[http://well.blogs.nytimes.com/2015/02/03/sidebar-whats-in-
th...](http://well.blogs.nytimes.com/2015/02/03/sidebar-whats-in-those-
supplements/)

[http://www.nytimes.com/2013/11/05/science/herbal-
supplements...](http://www.nytimes.com/2013/11/05/science/herbal-supplements-
are-often-not-what-they-seem.html)

Want to do a double-blind study on whether St. John's Wort improves your mood?
None of the bottles tested from Walmart, Target, GNC or Walgreens actually
contained any St. John's Wort in 2015. That's despite all the reports years
earlier about the same problem. There's little reason to believe things have
improved or will improve so long as supplements remain largely unregulated.

~~~
afarrell
I kinda wish the FDA had some certification to the effect that "this contains
what it says it contains and its contents are no more likely to kill you than
a peanut butter and jelly sandwich."

~~~
david_ar
Peanut butter probably isn't the best example...

~~~
afarrell
Why not? It is widely consumed, but there is a portion of the population that
has a severe reaction to it.

------
Uptrenda
Isn't there a huge potential for crowd sourced research platforms to take off
and help advance the state of the art?

So far it seems like most studies are done by scientists / academics in a very
closed off kind of way, but the true nature of science is open to everyone and
with the right platform we might be able to use normal people to gather enough
data to confirm or deny our hypotheses in a way that's much more open than the
standard study.

Essentially, anyone would be able to test their hypotheses and either help
advance scientific knowledge or at least provide extremely interesting markers
that could be used as a basis to get funding for more rigorous / controlled
studies. This approach obviously won't work well for every kind of question
but just as the OP has highlighted: it's particularly well-suited for making
self-experimentation more rigorous and I imagine it would work well for the
social sciences.

Imagine if we gave independent researchers like Gwern the tools they needed to
help answer more questions. If it was designed right: it would also be an
incredible learning tool for the scientific method (since you could
participate in the experiments + propose new ones.)

Are there any platforms like this in existence, I wonder? U-Uber for science?

~~~
chrisrxth
There are some in existence already:

\- quantifiedmind.com for self-testing your mind, which has a few group
experiments as well. I'm looking into integrating with them as part of
doubleblinded.com.

\- PACO (by Google) [https://www.pacoapp.com/](https://www.pacoapp.com/)
Pretty cool and flexible platform, although not the most user friendly or easy
to figure out. It is open source and has an app, which is cool.

\- A few sites like 23andme and ubiome have a lengthy list of survey questions
they ask you, which I only imagine they would pass through pattern recognition
algorthms to detect correlations between whatever information you give them
and your Genome (or gut genome in the case of 23andme).

So yeah, there is stuff out there. We want to do this for nootropics and
supplements (and eventually much more).

~~~
Uptrenda
The "eventually much more" part is what most interests me.

You guys might actually be some of the first researchers in the world to
experiment with this kind of model. I can see this being very big if its
marketed right.

------
alexbock
This makes me wonder if there is a simple procedure for doing a pseudo-double-
blind study on oneself (without a third party). That is, trying something that
can be prepared in pill form in such a way that you don't know during any
given day/week/month if it was a placebo, but that you can still determine
that later to collate the data. I know a lot of people that swear by
particular unregulated supplements or dietary choices and it would be really
interesting to have an easy way to see if something they're taking is having
the effect they think it does.

~~~
ryporter
I, too, have thought about conducting a pseudo-double-blind experiment on
myself. I think I could pull it off, but I'd also be willing to pay a third
party to help me. For example, DoubleBlinded could partner with a respect
supplement maker, and offer personal experiments in any supplement made by
their partner. The consumer wouldn't have to worry about setting up the
experiment correctly, and they would get their own results immediately upon
completion. Aggregate studies can wait until enough customers have self-
experimented on the same supplement, and DoubleBlinded can afford to wait that
long, because the partnership model keeps their costs low.

~~~
chrisrxth
This is definitely one potential path we are considering. There's a question
of whether we want to keep this specific to supplements/pills or try to
broaden it to other things like exercise or diet (which are not exactly
double-blindable)

------
aresant
A very cool idea @ the high level, but one fatal flaw:

"At the end of each day you will fill out a short survey with a few questions
evaluating the effects of the pills."

Self-reporting is an unsuitable mechanism to draw out scientific results.

There's an excellent detailed explanation available (1) but in TL;DR here are
four of the most compelling factors at work:

1 - Honesty/Image management

2 - Introspective ability

3 - Understanding / Question Interpretation

4 - Response bias

Take Image Management & Response Bias - participants know that they will be
able to see their results vs. the control group and it's not a leap to realize
how easily our ego and even subconscious need for validation could
dramatically skew the full study results.

(1) [http://www.sciencebrainwaves.com/the-dangers-of-self-
report/](http://www.sciencebrainwaves.com/the-dangers-of-self-report/)

~~~
j42
I'm not sure I see that as a fatal flaw... ultimately, the user is sitting in
front of a digital box that can guide and prompt them in all the same ways a
researcher can--the only limit is in performing tests that require a medical
professional to assess biomarkers. If the pace of medical device development
continues, it's even reasonable to think something like a Theranos-that-works
could commoditize the process while being intrinsically tamper-resistant.

Regardless, users can be prompted to perform any software action (knowingly or
unknowingly, to affect bias) and that action can be measured by the system. It
may so happen that every critical measurement occurs unbeknownst to the user,
before they self-report anything (if at all). As we are currently undergoing a
period of sensor-proliferation (fitness/health devices, wearables, internet of
things, etc...) it's not unrealistic to think we will soon be able to
instantly correlate data from a smartphone camera, blood/tissue, and the
cloud.

Now there's always the problem of intentional fraud/deception, but I think the
aggregate nature solves that problem. A small percentage will try to "break"
the system, and that small percentage will never surpass a critical threshold
with enough volume. In terms of ML/SVM's, we're now very good about filtering
outliers or "misrepresented data"... while the responsibility is on you to
develop a reliable classifier (for data-consistency more than arbitrary
measurement), I imagine at scale you could infer trends with the same relative
accuracy of traditional academia and research.

It's a really fascinating new direction--even if only an adjunct to
traditional research--and I'll definitely be keeping an eye on the project.

------
rootedbox
I would not mess with Saint Johns wart. It will make you feel happy and
cheery; but side effects are brutal. And if you participate in other drugs it
might have harsh side effects.

~~~
ak1394
And it's been banned in EU in 2011.

~~~
gabemart
It's still available for sale from the biggest herbal supplement retailer in
the UK [1]. That fact alone makes this claim seem unlikely.

[1] [http://www.hollandandbarrett.com/shop/product/holland-
barret...](http://www.hollandandbarrett.com/shop/product/holland-barrett-
maximum-strength-st-johns-wort-capsules-60097281)

------
Mz
I have done a lot of alternative med stuff. I know good protocols for how to
test the waters, introduce new supplements, figure out what they are doing,
etc. I would not use the service you are offering. It sounds to me like you
are asking me to be a guinea pig.

To make this useful, I think you need to do the following things:

1) Educate people on good practices generally. Your current site looks to me
to be an invitation to be someone's guinea pig and pay for the privilege
instead of being paid.

2) Offer doubleblinded kits for a wide variety of supplements that people can
choose from. The one supplement you are currently offering is one I never
heard of. So I don't care. But I might care if you offered me the chance to
pick and choose from a list of supplements.

I am no longer taking supplements, but I did take a lot of supplements at one
time. I can see a service like this having a use for people, but not in its
current format.

Also, I would remove the phrase "self experimentation." That sounds incredibly
Dr. Jeckyl and Mr. Hyde to me. "Self testing" might be okay, but that's a
terrible phrase and should be stricken from company vocabulary.

~~~
chrisrxth
Hey, thanks for the feedback. I totally agree about educating people on good
practices. This is a big part of what I want to do over the next few years - I
think educating people on the scientific method, especially in regards to self
experimentation is important. We want to work to create best practices and
protocols in the self-experimentation/quantified self world. We are planning
on putting together some content (videos and such) on the topic and
incorporating that educational aspect in our platform.

As far as the supplements go, I agree with that as well. We're planning on
adding more once the L-Theanine trial is sorted out. This is just our practice
round to get the protocol right. What other supplements would you be curious
to see?

------
weebro
Did you come up with this idea, or why was doubleblinded registered just after
this Reddit post?

[https://www.reddit.com/r/Nootropics/comments/3p0y2x/crowdfun...](https://www.reddit.com/r/Nootropics/comments/3p0y2x/crowdfund_crowdsource_trials_for_nootropics/)

[https://www.123-reg.co.uk/order/domain?X-CSRF-
Token=45b50019...](https://www.123-reg.co.uk/order/domain?X-CSRF-
Token=45b5001971906ca700fadd142a6f975d24dab8e6&domain=doubleblinded)

That doesn't seem very trustworthy

------
synapticrelease
This sounds like it could very easily be abused as a way to peddle supplements
while skimping on the experimentation aspect. It's a good idea though. Do you
have plans to include something other than supplements? For instance, common
foods (to be consumed daily for long periods of time) or simple
activities/exercises? You could still derive revenue by selling the
equipment/ingredients through your platform but it would allow you to vary the
product while maintaining the consumer base. Also, doing it with something
other than supplements could be cheaper and more newsworthy if you do discover
something. Very interesting!

~~~
volkhavaar
Thank you for your informative and critical feedback - skepticism is at the
heart of science! Very true, the system could potentially be used to promote a
specific agenda. However, we very much want to avoid a system that allows
something such as this as much as possible. Ideally through implementing open-
source / 3rd party / community-input based protocols. If you have suggestions
in this regard we'd love to hear them!

And yes, we plan on including food at a later date!

------
kingsley3
I just reported results from experimenting on myself at a bio-tech conference
called "Basel Life Science Week 2015".

Cocoa seems to have made me two years younger.

The poster I presented is at

[http://morse.kiwi.nz/kingsley/lib/exe/fetch.php?cache=&media...](http://morse.kiwi.nz/kingsley/lib/exe/fetch.php?cache=&media=science:screening_anti_aging_interventions_for_fun_and_profit.png)

------
chejazi
I wonder why they picked L-Theanine for their first experiment - the effects
are commonly known (caffeinated tea vs caffeinated coffee). Maybe this
knowledge was used deliberately so people "feel" a drug the first time around,
and want to keep going.

~~~
chrisrxth
We picked L-Theanine because of its safety profile and mild psychactive
properties, meaning you can tell if you've taking it or not but it's not an
overbearing feeling. We wanted the first experiment to be fairly predictable
while we work through all the kinks of launching this.

------
greggman
Is there motivation to cheat? If I want to sell this stuff what's to stop me
from signing up a bunch of people, having the pills tested, then reporting
what I want you to find so I that you'll tell the public my product is
effective?

~~~
chrisrxth
This won't be an issue because we are supplying the supplements which are
going to come from an established and reputable supplier. This is not a
platform where anyone can sell their own supplements.

~~~
greggman
I think you missed my point. I sign 100-1000 people up for your study. You
send the supplements to them. I quickly test the supplements to find out which
is which. I then send you a report that makes the days my people took the
supplements look positive for efficacy. You then report the supplements are
effective. I can now point to your report to sell my supplements. It doesn't
matter they are not your supplier. I still benefit from people thinking a
supplement is effective.

------
subliminalzen
If this can somehow preempt experimentation on animals, I'm all for it.

