Invitation design

Over the past couple of weeks our invite flow has drawn some attention. This blog post is intended to explain what happened and what we’ve done to correct it.

Original Design

First, some history on our invitation experience.  We released the following design in May of 2007, allowing users to invite their friends from their address books.

 
This page appears when a member invites contacts from an address book (Yahoo, Gmail, Hotmail, AOL).  The page has three sections:

• Top-left section shows friends already on Shelfari
• Bottom-left section shows friends which are not yet on Shelfari
• Top-right section allows the user to customize the invite message

Each of the three sections has a “Send invites” button, which triggers the action of sending invites to all of the users marked with checked checkboxes anywhere on the page

Right after we built this design we carefully monitored how our users were using it. At that point less than 1% of our users with large address books sent invites to their whole address books.  This seemed like a reasonable number and we concluded that our design was clear and didn’t have many people accidentally emailing their address books.  While we received occasional complaints we put re-designing this page at a lower priority because of our initial research. 

In September we began monitoring our email spam complaints.  Our spam complaint rate is actually very small – less than 1 out of every 200 of invitation messages is being marked as spam by recipients.

So, what changed?

Over the last few months our complaint rate has risen.  In the middle of last week I ran an analysis of how many people are sending to their entire address book and was surprised to learn the rate had risen dramatically.  It was 10x what it was in June when I measured it last.  You can call us stupid or evil if you’d like, but the reality was we have simply been extremely busy.  We figured that if the page had not substantially changed since June, that the user experience was much the same. 

The biggest change is that we have seen explosive growth.  Back in May we had tens of thousands of users. Now we have hundreds of thousands of users. While our growth has been fueled by people inviting their friends, we have not used tweaks to our invitation process to achieve our growth.

Since we have more users on Shelfari, the top-left section (with contacts already on Shelfari) is much longer for an average user than what it was in May.  As that section grew, it began pushing the list with users not yet to Shelfari lower and lower, finally pushing it off the screen for many users. If the user didn’t scroll the page and realize that there are more checked contacts on the page, he or she ended up unwittingly emailing their entire contacts list.  The screenshot below illustrates the problem (the darker section would most likely be off screen when the page first appears)

Because of this effect, we had a much larger percentage of users pressing the “Send Invites” button unaware that all of the checked contacts from the bottom section would also get email invitations to Shelfari.

One specific suggestion we had received in the past was simply to remove the extra “Send Invites” button in between the two sections.  In retrospect I should have done this a long time ago. It doesn’t take many development resources to remove a button. Still – we were a small team and I was relying on old data to justify maintaining an old design.  I sincerely apologize for our bad design misleading anyone with this decision to leave the extra button on the page.

What is the right thing to do?

Last week we had three new developers and one new program manager start! All of us who have been working hard on Shelfari are very excited to have extra engineering help.  Last week after realizing how poor our invitation experience was performing, we quickly released a new invitation flow.
Our new program manager did a quick comparison of invite flows from other sites: Facebook, MySpace, BeBo, LinkedIn and a couple more.   For reference we have included Facebook’s find friends flow:

Screen 1: Sign in

Screen 2: Friends on Facebook

Screen 3: Friends not on Facebook

Our Fix

Seeing how important it was to fix this problem, we decided to forego a complete redesign and to go with a design that resembles Facebook.  We split the invite process into two screens:

SCREEN 1: Allows our users to see contacts already on Shelfari and select which of them they want to add to their friends list.

SCREEN 2: Allows our users to clearly see contacts not yet on Shelfari, decide which contacts they want to invite, and send the invite.

This new updated design went live last Thursday night. We’ll most likely revise it again in the coming weeks, but we will make sure to be super clear about what emails are getting sent when new users join the site.  We will also update you when we make changes and talk about them in detail.

We want to deeply apologize to all of our users who mistakenly emailed unwanted contacts. Looking back at the screens from May, the placement of the middle “Send Invites” button was clearly a bad design with bad consequences. Our key goal is to build the world’s largest community of book lovers.  Unwanted email does not help us achieve that in any way.

My goal in writing this blog post is to remove any confusion about what we have done in the past, why we did it, what we do now and what we plan to do in the future.

Happy Reading,

Josh

Comments

[AJ]

Very good. I think this was overdue, but now I can back off from my anti-Shelfari stance a bit. Thanks for the fix.

[Deane]

No, you're not done -- do NOT check all the boxes by default. The odds of someone wanting to send an invite to everyone is slim to none.

Leave the boxes unchecked, and let the user check on the ones he/she wants to send.

[shelfari = spam]

Deane's comment seconded.

[Michael Buckbee]

I'd agree with the other two commenters that "checked by default" is a bad move as it is likely to lead to unintended emailings = spam.

[Josh]

Thanks for your comments. We do pre-checked boxes, as does Facebook, and most other quickly growing social networking sites.

We will continually monitor our spam complaints and also our numbers. If we find that too many people are making mistakes with the pre-checked boxes, we will consider changes.

[dikonstrukt]

In the biz, this is called opt-in and opt-out. Opt-in (where you have to check the box) always yields better quality users. No surprise, opt-out (user has to uncheck the box and your current design) yields much less quality (and frequently complaints of spam). While some sites out there still do opt-out, it is no longer the industry standard for marketing initiatives. It has a bit of a bad feel to it. It's your decision, of course, but having worked for internet companies for ten years, I can tell you that you will grow fast but have a largely inactive user base. While a large userbase number looks good on paper, when it gets compared to to ad/book sale revenue, it will be painfully obvious (to investors, for example) that your opt-out strategy has generated a low quality list.

[Guybrarian]

Thanks much for fixing this, you all. I'll make a fifth vote for the Opt-In model (thanks for your cogent comment, Josh), and agree with Deane that you're really only half there with the format change.

[Guybrarian]

Correction: thanks for your cogent comment 'Dikonstrukt,' (and as for your previous comment about me being a 'librarything fanboy,' I am familiar with a number of social cataloging sites, and take a strong interest in this area; it pains me to see unfair practices coming to the fore. Expressing my criticism of Shelfari hardly makes me some sort of librarthing agent - unlike the agents under shelfari's employ sent forth to market their site under false pretenses (http://www.librarything.com/thingology/2007/11/shelfari-astroturfing-evidence.php)

Yeh, I'll admit it: I tend to like the guys who play fair, and think there's still room for a little integrity in the book 'biz.' More fool I, I guess.

[MockTurtle]

Here's another vote for opt-in (boxes NOT pre-checked). This is one of the things I really dislike about Facebook, and would much rather see Shelfari break from the social-networking pack in this regard.

[Nancy White]

It is good to see the clarification. (I'm still dealing with pissed off and confused friends after sending to 900+ without understanding what I was doing.)

I will chime in that moving to unchecked boxes gives you a higher quality of participant and raises your own street cred.

Even though others are using the default checked box, that does not mean it is a good strategic decision that yields you better results over time. With more and more people using the checked box, suspicion will rise and use will probably decline.