Chris Coyne on changes to the OKCupid matching algorithm
I messaged Chris Coyne (one of the OKCupid founders) to voice my displeasure with the new matching system and concern about the direction of the site. Below is his response, which he gave me permission to publish:
hey ____ - the match calculation is still the same, but about a year ago we changed how much we adjust down the match percentages. It was:
score = score - 1/sqrt(q's in common)
score = score - 1 / (q's in common)
This change was announced in two places:
1. on our homepage 2. on the FAAAQ page (where we updated the explanation)
As you point out, this has 2 effects: 1. users who stop at say 30 or 40 questions can still show up in people's match results, because they aren't downgraded into oblivion 2. we're accepting a little more tolerance of error
This tradeoff wasn't something we took lightly. We spent months studying the effects of both ways of calculating, and it turns out the new way solved a huge problem: message diversity. in the old calculation, a small minority of users on the site were getting most of the messages, not because they were actually better matches, but because they were the ones who chose to answer hundreds of questions. everyone else got burned. the over-aggressive confidence adjustment wasn't simply making match results more accurate, it was making the site ineffective for casual users.
Interestingly, this decision benefited everyone *including* the serious users who'd answered lots of messages. Even the users with 500+ q's answered ended up with more long term conversations.
Sorry this change isn't working for you, but we're very happy with it, and the site has done a lot better since then.
-Chris p.s. there is actually a completely new way of calculating matches which we believe does an even better job, and we're experimenting with it now. We believe it's much more accurate and addresses your concerns above, without screwing with diversity. But it's computationally much more intensive. If we switch to it there will be a big announcement.