“Sharique Hasan and Aaron ‘Ronnie’ Chatterji, both strategy professors at Duke University’s Fuqua School of Business, studied more than 35,000 high-tech startups founded between 2008 and 2013. The researchers found startups that use a series of randomized controlled trials to test options for product features – otherwise known as A/B testing – are more successful, on average, than startups that don’t follow this type of data-driven approach.”
“Experimentation is a cornerstone of agile software delivery. Yet I’ve noticed some confusion over the definition of an experiment and how the process is carried out. Today, we will examine the basic steps, how other professions in science and healthcare address experimentation, common factors, and finally, a quick exercise to bring all the concepts together.”
“While top economic performers are already significantly ahead of their peers in specific digital capabilities (automation, for example), they are also moving much more quickly than their peers in key business areas. Top tech companies, for example, share test-and-learn findings across the business, reallocate digital talent, and use multiple sources of insight about customers weekly, while average performers do those tasks monthly.”
I first wanted to share some thoughts on why this approach works and why we are so focused on experimentation velocity. HelloFresh is a company with users counting in millions and high user traffic, so we get a lot of data quickly, allowing experiments to reach statistical significance in a matter of weeks and in some cases, days. That drives the cost of a single experiment down and makes it financially justifiable to experiment often.
In this era of internet of things, business development and optimization remarkably lies in the power of learning and adapting, in which A/B testing is essentially the core. We use A/B test to analyze and understand how things perform against a controlled baseline. A decent years back, A/B tests were metaphorically referred as oil or fuel of the future, and more recently the comparison has been with sunlight because like solar rays, they will be everywhere and underlie everything.
“I have been writing this article based on and after reading the book: “Trustworthy online controlled experiments — A practical guide to A/B testing”, by Kohavi, Tang and Xu, probably the most helpful business AND technical book. The motivation is to share the beautiful story of how science plays a vital role in business today more than ever and encourage product owners to get closer to engineers so that they have fun while building successful growth for their organizations.”p>
“Executives must play a proactive role in making sure their digital design functions in the best interests of users. Doing so has the potential to give companies a deeper and more positive relationship with its customers. Brands that design their sites to exploit consumer behavioral bias (or those that fail to recognize that they do so unknowingly) might benefit in the short term, but the damage done to their reputation will do lasting harm.”