#32 - An Imperfect Match
Thursday, July 31, 2014 - 04:38 PM
This week, dating site OK Cupid put up a blog post describing
experiments it conducted on its users. In one experiment, the site told
users who were bad matches for one another that they were actually good
matches, and vice versa. Alex and PJ talk to OK Cupid President and
co-founder Christian Rudder about the ubiquity of online user
experimentation and his defense of potentially sending OK Cupid's users
on bad dates.
Thanks for listening. If you like our show, please subscribe to us on iTunes. Or you can follow PJ and Alex and TLDR on Twitter.
More in:
-
PJ Vogt
PJ Vogt is on Twitter here. If you'd like to subscribe to TLDR's short weekly podcast, please go here.
-
Alex Goldman
Alex Goldman is a producer for On the Media and the co-host/creator of TLDR. He is on twitter here. Subscribe to the TLDR podcast here.
- Leave a comment
-
RSS Feed for Comments
Comments [2]
Full disclosure: I'm the happiest
possible OkCupid customer since they introduced me to my wife. My bias
acknowledged, I'm completely on their side on this.
First: Yes, every pixel of every website is an experiment, whether it's a controlled and thoughtful one or an unacknowledged and uncontrolled crapshoot. As Rudder points out, it's the sites that know when they're experimenting which are being more responsible. Despite Rudder's repeated challenges, you guys never offered an alternative to the scientific method.
Second: The context and consequences of the experiment make all the difference. OkCupid is a dating site, not a dispenser of chemotherapy. The results it delivers are inevitably riddled with false positives and false negatives. Every user should behave accordingly, viewing potential matches with a proper mix of skepticism and frivolity. The full responsibility for any bad dates or broken hearts lies with people who entrust their love lives to silly algorithms. OkCupid adding a little noise into the mix doesn't change that a bit.
First: Yes, every pixel of every website is an experiment, whether it's a controlled and thoughtful one or an unacknowledged and uncontrolled crapshoot. As Rudder points out, it's the sites that know when they're experimenting which are being more responsible. Despite Rudder's repeated challenges, you guys never offered an alternative to the scientific method.
Second: The context and consequences of the experiment make all the difference. OkCupid is a dating site, not a dispenser of chemotherapy. The results it delivers are inevitably riddled with false positives and false negatives. Every user should behave accordingly, viewing potential matches with a proper mix of skepticism and frivolity. The full responsibility for any bad dates or broken hearts lies with people who entrust their love lives to silly algorithms. OkCupid adding a little noise into the mix doesn't change that a bit.
Aug. 01 2014 03:13 PM
Score: 0/0
Score: 0/0
I think PJ`s 2 cents make
absolutely no sense at all. There is no difference between changing the
algorithm a little bit and changing the algorithm a whole lot. If every
company changing their SaaS(Software as a Service)-Product would ask
for permission/notify people every time they change their product, all
users would go mad. You have to look more abstract at this issue:
okcupid changed their product - if that change is only for some users,
or only temporary is completely irrelevant - and they have the right to
do that freely and without notification. That really is the end of the
story right there.
Notifying all affected users(which are not always actually predictable) about every product change is not only absolutely unfeasible and unreasonable, it also does not help the users in the slightest. The only thing that would do is annoy them with constant spam(some Websites change multiple times _daily_). This is an A/B test like any other that happens anywhere anytime, on almost any website, and actually almost any other product in this world, there is absolutely nothing wrong with it, it in fact is a very good thing that they are doing this test, in this specific case, because it actually is simple quality assurance (which is not the case with most A/B tests, but with this one).
Aug. 01 2014 07:08 AM
Notifying all affected users(which are not always actually predictable) about every product change is not only absolutely unfeasible and unreasonable, it also does not help the users in the slightest. The only thing that would do is annoy them with constant spam(some Websites change multiple times _daily_). This is an A/B test like any other that happens anywhere anytime, on almost any website, and actually almost any other product in this world, there is absolutely nothing wrong with it, it in fact is a very good thing that they are doing this test, in this specific case, because it actually is simple quality assurance (which is not the case with most A/B tests, but with this one).
Score: 1/1