Abstract This is my personal web page. It's not scientific at all, though I took the opportunity to honour my research background by making it look like a scientific paper. It may contain more useful information than the average paper of many conferences, though, but that's not a hard challenge to achieve. Each section is a kind of blog post.
Written on Jan 2, 2013. I'll outline in this section what we call the Theory of Mountains™ inside Traity. For those of you who have not heard of AB testing, it's just an optimization technique in which half of a website's users sees a design alternative and the other half sees a different one. If you have properly defined the key metrics for your company, you'll choose the alternative that optimizes such metrics.
Metrics are functions whose values can be explored by choosing alternatives in your application to optimize your business. This is what a growth hacker would do, and, if you take a look at the possible values of a metric, as shown in the next figure, you'll get why we call this the Theory of Mountains™:
In that picture you can observe the outcome of several design approaches on a specific metric such as the conversion rate of a landing page. The x axis doesn't have a scientific meaning, but helps to understand the idea behind. The picture shows three mountains that represent the value of a relevant metric according to the design of your web application and other factors.
Additionally, I've highlighted three "exploration points" A, B, and C. You could think of them as something like "A is the landing page with sans-serif and red font, B is the landing page with black serif font, and C is a minimalistic landing page with just a picture". Therefore, each point represents a version of our website.
A mistake when doing AB-testing is overexploring a mountain and never trying different ones. An example of overexploring a mountain can be testing a conversion button's text with the cases "Try it now!" and "Try it now free!". Of course, the word "free" has a powerful effect on users' minds which deserves being tested. If, however, you have not explored completely different cases, you're getting stuck in the same mountain.
Let's go back to the picture with the mountains. Imagine we're building a social network about pets. We release a landing page so that people can signup and get a particular conversion rate at point B. By tweaking messages, testing colours and layout, we could get to design A and obtain a better conversion rate. But doing that too early in the life of the startup would limit your view of the metric landscape.
Instead, testing very different approaches would lead to huge differences that make the exploration more interesting. If, for example, you had tested a landing page about pet sharing, you might have been able to reach the peak at C after some optimization.
So when it comes to building a startup and explore mountains in the best, leanest way, here's the methodology I'd suggest:
Identify a problem. Startups need to solve problems, and we experience things that don't work well every day. Stick to a big pain that motivates you and devote your startup to solve it. Paul Graham can explain this much better, and has an awesome post on the subject (Google for "how to get startup ideas paul graham").
Define the success metric. E.g., if you plan to create a dating site, the main metric would be the number of successful matchings. Optimizing this main metric will tell you that your startup is doing something useful and solving the problem you decided to tackle.
Split the main metric progressively. The metric of successful matchings could be split into the number of meetings a user has and the percentage of meetings that result in a couple. Other submetrics would be the number of users in the website, and the amount of meetings that these users have, etc.
Create a Minimum Viable Product. An MVP should be built in a weekend. If you need more than a weekend, you're going too far and assuming too many hypotheses you've not proven yet. An MVP can be a video, sketches, or (best) a crappy, ugly website that has the minimum functionality of what you want to achieve. This MVP will position your startup in one mountain of your main metric landscape, according to the design decisions (or the lack of them) that you've taken when building it.
Learn. Your metrics define the potential levers that your startup can use to boost your main metric. E.g., you could observe that your dating startup has many users but they don't date, which is the explanation of a poor number of successful matchings. Or maybe users date, but the dates are not successful. It's crucial to know what part of the chain is failing. Knowledge is power, and by measuring, knowing your funnel, and sticking to real data you're able to make decisions that truly improve your startup.
Change. The knowledge acquired will allow you to try new, different approaches to explore other mountains. Just try them, measure their performance and stick to the most promising one. Explore different marketing channels, different UIs, different landing pages, and different design approaches to each point of your user flow.
Climb. Only when the most promising approaches have been identified is the time to climb the chosen mountain. Optimize your marketing channel, your viral acquisition strategy, and your flamboyant landing page only when you've measured they're better for your particular case than other approaches.
That's essentially my particular Theory of Mountains™. Remember that satisfaction comes not from being at the top, but from the path to it, so enjoy the journey, have fun, and get things done.
Written on Jan 2, 2013. I wanted to use a two-column layout for this page to resemble a scientific paper. Of course, I could have used a single-column, single-page layout, but that's not nearly as fun to code.
There's not a standard way in HTML to build a two-column layout in which text fills up one column and continues on the next one. Moreover, there's not a single way to split the content across several pages once one is filled up.
At the time of writing this section, there's not a way to link to a specific section in the paper. There's not column balancing in the last page, either. Also, the whole "blog" is loaded everytime, which would be an issue when there's a lot of content. A solution could be using dynamic loading of the different sections with infinite scrolling. I'll care about that once the site grows, as right now there's not much content to show.
I'm CTO at Traity, the worldwide 360-feedback evaluation. I have a PhD in Computer Science and consider myself mainly a hacker and entrepreneur. I've been coding since I was a kid (mainly videogames) and getting involved in entrepreneurial projects since my university days. I'm currently based in Silicon Valley (Mountain View).
Future work includes updating this page, although that might happen monthly or even yearly. You can follow me on Twitter (@researcheneur) or reach me by email (firstname.lastname@example.org). Nope, no hyperlinks: this is just a paper and papers don't have hyperlinks.
Feel free to contact me if you're an entrepreneur, founder, designer, coder, angel investor, researcher, magician, couch hog, rockstar, or plain and dull human being. I can't guarantee an answer, as time's precious and inboxes have spam.