That venerable bastion of journalistic excellence, The New York Times, has recently been under even more intense scrutiny by content marketers everywhere after the leak of their Innovation Report a few weeks ago. I’m not going to go through it here, and if you haven’t read it, you can find some insightful articles at the end of this post.
Instead, I’d like to shine the spotlight on a section of their report which looked at the issue of experimentation (something which comes second nature to every competent SEO).
The New York Times has gotten some great reviews on innovative article formats such as ‘Snowfall’. But as the report says, true innovation is more than just about trying something new.
It is about coming up with a scientific process ‘for proving new concepts and constantly tweaking them to be as successful as possible’. This is something that we at CLOUDROCK have been advocating for quite a while.
We believe that the whole purpose of experimentation is not just to try something new, one time. That is both difficult and time-consuming, even if the results are worth it at the time.
It is about coming up with something (for example, a new content format, tool, or marketing process) that can be integrated into your current workflow, is easily repeated and can be scaled. This in turn allows even further experimentation and refinement.
This bears repeating – an experiment is a success when it introduces a new way of doing things that is easily repeated, scalable and provides better results. Conversely, even if the experiment does not produce those results, I wouldn’t go so far as to call it a failure – provided that insights have been gained and that the new knowledge has been distributed to the relevant people in the organization.
To illustrate the point, the dialect quiz built by NYT proved to be its most successful piece of content in its history, with over 21 million pageviews. One could argue quite successfully that the effort was worth it.
But its competitors over at Buzzfeed developed a template that allowed them to produce quiz after quiz easily, without needing a developer. How many quizzes have they made since then? I’m quite sure Buzzfeed has garnered more than 21 million pageviews on all their quiz pages so far.
As another example, it took the NYT hundreds of hours to hand code Snow Fall. I can’t argue with the results; it’s absolutely amazing and I love it.
However, there are already companies such as Scrollkit (acquired by Automattic) and Natively that are trying to create platforms that would allow the average user to develop visual stories such as Snow Fall in a fraction of the time, and without those ninja coding skills. What kind of content would those master storytellers at the NYT be able to produce on a regular basis if they had access to such tools?
This experimental approach is not only applicable to tools and website features, but to marketing activities as well. This is especially so for digital marketing tactics.
It gives our team a warm fuzzy feeling to find the strategy that we’ve been using for ourselves and recommending to clients being validated by NYT. Some of their recommended best practices for experimentation include:
- Launch quickly, based on the minimum viable product model
- Set your goals and understand how your new experiment will affect your KPIs
- Only declare an experiment a success or failure after a debrief, and show the insights gained to improve future efforts
- Encourage a culture of experimentation by rewarding people for showing initiative and not punishing failure
- Don’t be afraid to kill off mediocre efforts, but be transparent on the reasons behind the decision
- Plan for – and invest in – reiterations
- Make it easier to launch an experiment than to block one
Obviously, we don’t do it on as large a scale as NYT. But I’d like to think that our approach and process is similar, and that someday we might be able to prove it on a larger scale.
As promised, if you’re looking the leaked New York Times Innovation Report, you can find it here and here.