Invitation design

Over the past couple of weeks our invite flow has drawn some attention. This blog post is intended to explain what happened and what we’ve done to correct it.

Original Design

First, some history on our invitation experience.  We released the following design in May of 2007, allowing users to invite their friends from their address books.

 
This page appears when a member invites contacts from an address book (Yahoo, Gmail, Hotmail, AOL).  The page has three sections:

• Top-left section shows friends already on Shelfari
• Bottom-left section shows friends which are not yet on Shelfari
• Top-right section allows the user to customize the invite message

Each of the three sections has a “Send invites” button, which triggers the action of sending invites to all of the users marked with checked checkboxes anywhere on the page

Right after we built this design we carefully monitored how our users were using it. At that point less than 1% of our users with large address books sent invites to their whole address books.  This seemed like a reasonable number and we concluded that our design was clear and didn’t have many people accidentally emailing their address books.  While we received occasional complaints we put re-designing this page at a lower priority because of our initial research. 

In September we began monitoring our email spam complaints.  Our spam complaint rate is actually very small – less than 1 out of every 200 of invitation messages is being marked as spam by recipients.

So, what changed?

Over the last few months our complaint rate has risen.  In the middle of last week I ran an analysis of how many people are sending to their entire address book and was surprised to learn the rate had risen dramatically.  It was 10x what it was in June when I measured it last.  You can call us stupid or evil if you’d like, but the reality was we have simply been extremely busy.  We figured that if the page had not substantially changed since June, that the user experience was much the same. 

The biggest change is that we have seen explosive growth.  Back in May we had tens of thousands of users. Now we have hundreds of thousands of users. While our growth has been fueled by people inviting their friends, we have not used tweaks to our invitation process to achieve our growth.

Since we have more users on Shelfari, the top-left section (with contacts already on Shelfari) is much longer for an average user than what it was in May.  As that section grew, it began pushing the list with users not yet to Shelfari lower and lower, finally pushing it off the screen for many users. If the user didn’t scroll the page and realize that there are more checked contacts on the page, he or she ended up unwittingly emailing their entire contacts list.  The screenshot below illustrates the problem (the darker section would most likely be off screen when the page first appears)

Because of this effect, we had a much larger percentage of users pressing the “Send Invites” button unaware that all of the checked contacts from the bottom section would also get email invitations to Shelfari.

One specific suggestion we had received in the past was simply to remove the extra “Send Invites” button in between the two sections.  In retrospect I should have done this a long time ago. It doesn’t take many development resources to remove a button. Still – we were a small team and I was relying on old data to justify maintaining an old design.  I sincerely apologize for our bad design misleading anyone with this decision to leave the extra button on the page.

What is the right thing to do?

Last week we had three new developers and one new program manager start! All of us who have been working hard on Shelfari are very excited to have extra engineering help.  Last week after realizing how poor our invitation experience was performing, we quickly released a new invitation flow.
Our new program manager did a quick comparison of invite flows from other sites: Facebook, MySpace, BeBo, LinkedIn and a couple more.   For reference we have included Facebook’s find friends flow:

Screen 1: Sign in

Screen 2: Friends on Facebook

Screen 3: Friends not on Facebook

Our Fix

Seeing how important it was to fix this problem, we decided to forego a complete redesign and to go with a design that resembles Facebook.  We split the invite process into two screens:

SCREEN 1: Allows our users to see contacts already on Shelfari and select which of them they want to add to their friends list.

SCREEN 2: Allows our users to clearly see contacts not yet on Shelfari, decide which contacts they want to invite, and send the invite.

This new updated design went live last Thursday night. We’ll most likely revise it again in the coming weeks, but we will make sure to be super clear about what emails are getting sent when new users join the site.  We will also update you when we make changes and talk about them in detail.

We want to deeply apologize to all of our users who mistakenly emailed unwanted contacts. Looking back at the screens from May, the placement of the middle “Send Invites” button was clearly a bad design with bad consequences. Our key goal is to build the world’s largest community of book lovers.  Unwanted email does not help us achieve that in any way.

My goal in writing this blog post is to remove any confusion about what we have done in the past, why we did it, what we do now and what we plan to do in the future.

Happy Reading,

Josh

Comments

[AJ]

Very good. I think this was overdue, but now I can back off from my anti-Shelfari stance a bit. Thanks for the fix.

[Deane]

No, you're not done -- do NOT check all the boxes by default. The odds of someone wanting to send an invite to everyone is slim to none.

Leave the boxes unchecked, and let the user check on the ones he/she wants to send.

[shelfari = spam]

Deane's comment seconded.

[Michael Buckbee]

I'd agree with the other two commenters that "checked by default" is a bad move as it is likely to lead to unintended emailings = spam.

[Josh]

Thanks for your comments. We do pre-checked boxes, as does Facebook, and most other quickly growing social networking sites.

We will continually monitor our spam complaints and also our numbers. If we find that too many people are making mistakes with the pre-checked boxes, we will consider changes.

[dikonstrukt]

In the biz, this is called opt-in and opt-out. Opt-in (where you have to check the box) always yields better quality users. No surprise, opt-out (user has to uncheck the box and your current design) yields much less quality (and frequently complaints of spam). While some sites out there still do opt-out, it is no longer the industry standard for marketing initiatives. It has a bit of a bad feel to it. It's your decision, of course, but having worked for internet companies for ten years, I can tell you that you will grow fast but have a largely inactive user base. While a large userbase number looks good on paper, when it gets compared to to ad/book sale revenue, it will be painfully obvious (to investors, for example) that your opt-out strategy has generated a low quality list.

[Guybrarian]

Thanks much for fixing this, you all. I'll make a fifth vote for the Opt-In model (thanks for your cogent comment, Josh), and agree with Deane that you're really only half there with the format change.

[Guybrarian]

Correction: thanks for your cogent comment 'Dikonstrukt,' (and as for your previous comment about me being a 'librarything fanboy,' I am familiar with a number of social cataloging sites, and take a strong interest in this area; it pains me to see unfair practices coming to the fore. Expressing my criticism of Shelfari hardly makes me some sort of librarthing agent - unlike the agents under shelfari's employ sent forth to market their site under false pretenses (http://www.librarything.com/thingology/2007/11/shelfari-astroturfing-evidence.php)

Yeh, I'll admit it: I tend to like the guys who play fair, and think there's still room for a little integrity in the book 'biz.' More fool I, I guess.

[MockTurtle]

Here's another vote for opt-in (boxes NOT pre-checked). This is one of the things I really dislike about Facebook, and would much rather see Shelfari break from the social-networking pack in this regard.

[Nancy White]

It is good to see the clarification. (I'm still dealing with pissed off and confused friends after sending to 900+ without understanding what I was doing.)

I will chime in that moving to unchecked boxes gives you a higher quality of participant and raises your own street cred.

Even though others are using the default checked box, that does not mean it is a good strategic decision that yields you better results over time. With more and more people using the checked box, suspicion will rise and use will probably decline.

[casey]

We're really supposed to believe that Shelfari did not know this was an issue? People have been complaining about this for months on the blogs. Shelfari has certainly found the time to astroturf every blog that shows up on a google search of "shelfari", but they didn't know about dozens of complaints? They only made the change after LibraryThing and GoodReads called them out on it, so maybe they really don't read their customers' blogs.

The bottom line is that sending out as many emails as possible *can* be good business -- the CEO of Shelfari says as much in this very thread. Spam exists because some people do respond to it. So what really happened here was that the cost of being labeled a spammer finally outweighed the benefits of Shelfari being a spammer.

Shelfari's traffic started to spike about the same time they started spamming and astroturfing. The suggestion that nobody at Shelfari noticed this (even as they've frequently crowed about their Alexa stats on various blogs, including this one) is ludicrous.

Why did Amazon give a million dollars to: (1) a company that doesn't know about email or privacy best practices; (2) a company that doesn't read its customers' blogs but does read its competitors'; (3) a company founded by a bunch of non-techies who use words like "leverage" and "liase" non-ironically, but can't even competently handle a simple PR problem; (4) a company where an intern spent months illicitly astroturfing without anyone apparently noticing; (5) a company with at least twice as many employees as their nearest competitor, but hasn't made any significant improvements to their bug-riddled site in several months?

[DaveA]

So, Josh-

Two points:

1) Who is this Dan guy? You should study the Whole Foods debacle: their CEO took tremendous heat for astroturfing, which even had the potential to derail their Wild Oats acquisition.

2) Do the right thing: Opt-in. Saying "FaceBook does it" doesn't excuse bad acting. It's like using MSFT as a model for how to do UI.

[tim]

don't be greedy... make it opt-in!
agree with josh above.

look at flickr's success... and all without that nasty webmail trick... they fostered that community by make continuous tweaks and improvements and posting comments, showing they care about photography, etc.

i'm trying this site now and so far so good, although the community isn't really there yet... hope i stick with it... book discussion outside an amazon reviews is something that's really needed...

[Jay Levitt]

@ Josh: We at AOL always "pre-opted" our users into everything too. It was much easier to use that way, our growth was phenomenal, and it didn't hurt our reputation at ALL - and I got to buy some really nice cars. Go for it!

P.S. But I wouldn't wait for the 2009 models to come out.

P.P.S. Are you hiring? A bunch of my friends are looking for work.

[Les]

I'm the latest victim and am dealing with irritated people/companies asking me to stop sending them invitations and reminders. Does delting your account stop this? I hope so.

[Janice Lopez]

I cannot get into Shelfari - my virus checker tells me there is a malware program and tells me to disconnect immediately.
Janice
New Mexico
March 24, 2009

[Shelfari]

@Janice--Thanks for letting us know you're seeing this alert. Avast has recently made changes which are incorrectly marking Shelfari's javascript (.js) as infected. Our javascript is not infected and the site is safe to use. Our developers are working with Avast to correct the false alert being generated by their product. The alert should
disappear soon.

Amanda

[Buy Viagra]

hi people, i prefer this king of blogs are very intresting and helpfull, i really like it, congratulations keep it going

[Fit Health Guide]

excellent information, you have done an excellent work with this article! congratulations!

Cheryl D. Washington

[Josi]

Thanks much for fixing this, you all. I'll make a fifth vote for the Opt-In model (thanks for your cogent comment, Josh), and agree with Deane that you're really only half there with the format change.

[Buy Cialis]

this is the best post that I find in long time.

[forex trading system]

don"t really post but cannot getting into Shelfari im finding to be difficult- my virus checker continues tell me there is a malware. any advice?

[weight loss pills]

Thanks for the tutorial

[Gajoob]

We're really supposed to believe that Shelfari did not know this was an issue? People have been complaining about this for months on the blogs. Shelfari has certainly found the time to astroturf every blog that shows up on a google search of "shelfari", but they didn't know about dozens of complaints?

[bobby]

great info. thnaks