On March 5, I pulled the trigger on soft launching a new product, jsonip Pro http://getjsonip.com. My core goal failed dramatically, though I had some success around the periphery. There’s a few lessons here, some technical and some ethical. If you just want the summary, just scroll to the bottom.
For some quick background, I have been running a free service called jsonip.com since November 2010. Its a utility service that returns a visitor’s IP address in a JSON object. Supports CORS and JSONP as well.
With literally 0 (zero) dollars spent beyond monthly hosting and just word of mouth via Twitter and Hacker News, jsonip has gone from 10 requests a week to over 10 million requests a day. Over the last couple of months and after many conversations of beer and whisky with friends, people kept saying “and that’s free?” Yeah, I replied. Its a simple utility and I’m happy that people find it useful. But ever since jsonip has crept over the 10 million a day mark, I’ve kept asking myself if there was anything more I could do. Is there any way that I could turn this into a product that people might pay for?
I was talking with a buddy in Seattle recently and telling him about it. He asked if I had considered adding geoip lookup, and that was the pebble that started the avalanche in my brain. I realized that there were different features I had been wanting to add to jsonip for over a year, and that I can bundle all of those together and more, and offer something that will have a lot more value than just basic IP lookup. Hence, jsonip Pro was born.
I’ve built enough products on my own and for various companies, along with studying enough startup literature, to know the value of the MVP. In my case, I already more or less have the MVP. Its http://jsonip.com. Its been running for years, has hundreds of users, and supports millions of requests a day. What I know is that offering a paid product is going to be something different. There are different things to take into account:
- How many features do I need complete?
- How much work is required to complete all the features?
- What are the unknown unknowns? (I’m a single developer. There’s stuff dealing with payments, scaling, etc that I know about. What are the things that I *don’t* know about?)
I gave these questions a lot of thought and came to the conclusion that its going to be a bit of work to get this going. Not insurmountable, but a heavy investment in time for me.
The MVP of MVP’s is interest. Are there going to be enough people interested in the product I’m planning to justify the time and effort to put into building it?
One popular way to handle this is what I term a “soft launch”, though of course that phrase has multiple meanings. To me, it means getting a signup page out that describes your product. Collect emails and comments. Track with Google Analytics. See what the reactions are on Twitter. Post to Hacker News and gauge the responses (if any).
I have my list of features. They’re posted on http://getjsonip.com. I have a good idea of the total time its going to take to complete everything.
Its the unknown unknowns that bite you in the ass. More on that in a second.
For this launch, I wanted to hit everything perfectly. I spent a few days working on the signup page. Its hosted with LaunchRock. They’re free and work pretty good. Their UI is pretty usable but needs some improvement. Honestly, they’re better than anyone else I found and I didn’t want to host this myself. That would have involved setting up signup capture, integrating email services, etc. Too much work for a 1-use signup page.
The rest of my launch strategy involved posting on Hacker News, sending tweets to people that were known jsonip.com users, and sending a few emails to people I know (under 10 people).
I got everything ready, did some research on the best time to post to HN (morning or afternoon), then pulled the trigger.
Some people will criticize me for what follows. I’m kicking myself for it too. Basically, the main community I know about and have been a part of for years that deals with launching products and startups is Hacker News. I have a fairly high karma level there and frequently interact via comments and submissions. I don’t have a huge Twitter following nor do I have an extensive email list of interested parties. Hacker News is basically the main target at this point for me.
Since my main goal at this stage was to gauge interest, I needed eyeballs. I wanted at least a few hundred people to take a look at getjsonip.com, signup if they were interested, leave a comment, or send me an email, or just go away. Any combination of those actions provides me with information I need about whether to proceed or not.
My strategy was executed like this:
- Post on Hacker News in the morning, http://news.ycombinator.com/item?id=5326491
- Go on Twitter and look for people that had talked about it at some point. Send them an @ reply of the form, “@<person> launching jsonip Pro. Take a look news.ycombinator.com/item?id=5326491”.
- Send an email to my tiny contact list with roughly the same message, again linking to the Hacker News post.
Since I have limited eyeball-getting-resources, the idea was to use multiple channels, but focus everyone to HN with the hope that if even a couple of folks thought it was an interesting idea, they might spare an upvote. I’m sure by now you’re starting to see the problem.
If anyone remembers back a few years to Digg’s heyday, they had a massive problem with voting rings. Groups of users, or more often groups of accounts controlled by bots, would all vote for the same story in order to push it to the front page. Digg had lots of traffic and making it to the homepage was valuable at the time.
Hacker News is a *very* popular site these days. We face similar issues. Of course, this didn’t even occur to me the morning of as a problem. Like I said earlier, unknown unknowns.
Over the course of 1.5 hours, I executed. HN Post, done. Tweet, tweet, tweet. Email.
As I made breakfast, I immediately start getting feedback on Twitter. All positive!
- @geuis Oh wow, nice!
- @geuis Cool. Now even handier!
- @geuis Smart idea. 🙂
- @geuis Nifty! Don’t have an immediate need for this, but I’ll definitely keep this in mind. 🙂
I got a couple of email replies, also positive.
I would occasionally check the Hacker News post. After a couple of hours, it was up to 8 votes. That’s often, but not always, enough to at least make it temporarily to the bottom of the front page which is all I needed anyway. But then a funny thing happened on the way to the Forum.
The more votes the post got, the further down the http://news.ycombinator.com/newest page it got. When it was 300+ and I realized that 300 new stories hadn’t been posted in 2 hours, I started realizing what was going on. After a couple of searches and getting a hint from a friend, it became obvious.
Since the votes were coming from visitors that had hit the link directly, their votes weren’t being counted. And while I can’t confirm it, I suspect each additional vote just accelerated the downward process in /newest.
I had inadvertently created a voting ring.
I *detest* spammy shit. One of my favorite scenes is in the book Daemon where the daemon geo locates the main email spammers in the world and then hires hit men to assassinate them all in the span of a few minutes. Global spam drops to near nothing overnight. I’m not sure how HN does it, but the way I would do it is that any votes coming from users on pages with no news.ycombinator.com referer get sent to the special level of hell reserved for voting rings. Yet here I had done one of the very things I hate the most. Merde.
If not a voting ring mechanism, then lots of users were flagging the post. That’s possible too, though I think unlikely.
Lets see. I screwed up my mini-launch.
Using what limited resources I have, I tried formulating a plan that would get the most feedback I could reasonably expect. I did get a lot of positive feedback in proportion to the sample size that actually saw http://getjsonip.com, but near 100% positive feedback isn’t actually that useful at this stage. Its a little like showing something to your friends and family and them all saying what a smart cookie you are.
What I was hoping to get was a scale of interest, or indeed no interest. I value the commentary of fellow HN users, but sometimes you even end up with lots of “awesome +1” comments from well-meaning people. That’s very encouraging, but what you really need to see are critiques in combination with the good stuff. In your analytics, you need to see that you had X overall visitors, they stayed for N time, Y percent signed up, etc.
I had less than 10 visitors, 8 upvotes from people I know or that use jsonip.com, 2 signups from unknown people, and 1 signup from a friend of mine. (Love ya buddy, but you don’t count!)
There’s a lot of encouraging signals in that data, but the sample size is too small to be really useful at this point.
You might have guessed by now, but there’s an ulterior motive for writing this post.
My friend with the most excellent hints was the one that initially alerted me to the possible direct-link-voting problem. He also said something that prompted me to write up this experience, “HN readers would like that blog post.”
What I’m hoping is that by drawing attention to how I screwed up my soft launch, I might still be able to garner enough attention to complete the study I started with. Its also to describe my experience, and hopefully provide a road sign for other people working on their own products.
- If you’re starting from scratch, build a minimal viable product. Often that’s just one feature. Other times, its just a signup page to measure interest. Don’t waste time on building lots of stuff that people end up not wanting anyway. Engage and measure.
- If you have a successful MVP, iterate and measure. Add feature B and see how users user or like it. Repeat for C, and so forth.
- If you’re in a situation like mine and you have a successful product, but it has a narrow focus, you can’t just add feature B. In my case the new product adds lots of extra information but it will take me time to build and test everything. Not a lot of new features, but they require a lot of work. Start with a signup page with your existing user base and see if they will be interested. This is also hard in my case because most of my user base is anonymous to me.
- Know your audience. For me, my audience is developers and engineers. They frequent Hacker News, so that is my target.
- When you have limited “eyeball resources” as I call them, focus your resources towards one target. In my case, I would get the most views if I was able to get to page 1 or 2 of Hacker News for a couple hours. Your target may be completely different.
- If you’re going to do what I did, don’t tweet/email people to the direct link. Ask them nicely to find your post on the /newest page and vote from there. Its a bit gamey, but if someone actively goes out of their way to find your post and upvote you that’s a really positive indicator even if you don’t make it to the front page. Otherwise, your post is going to go down down down. Also, don’t do this to game Hacker News with your spam. I hate even advocating this technique, so use it with care and respect.
- If your launch fails to meet your expectations/needs, consider following up with a detailed post about what you did. It might work. (I don’t know yet if this one will.)
For those who skipped directly to this point, let me summarize for you.
In order to first determine if there would be enough interest to invest the time to build the Pro service, I decided to do a soft launch using just a signup page to gauge interest. This went spectacularly wrong.
I don’t have a lot of resources in terms of Twitter followers and email lists to promote things. To get the data I want, I was hoping to use a post on Hacker News to get perhaps a few hundred views to get some commentary, signups, or no interest at all. Any combination of those would tell me whether I should proceed or not.
I directed the handful of tweets and emails I was able to send to the HN post. Turns out this seems to have activated a voting ring detector and pushed the post so far down Satan wouldn’t find it. I inadvertently created something I hate passionately, spam.
On the plus side, the feedback I *did* get was all positive and resulted in 2 signups, but the sample size was too small to be conclusive.
I’m writing this post for several reasons. To document this process, to help others working on their own products, and possibly to get enough feedback on the idea, http://getjsonip.com, to determine whether or not to move forward.