Using NPS feedback: Zola case study

Some interesting titbits on how Zola is using NPS feedback to improve product -

• - Bucket NPS feedback by theme. The issues that prevent people from picking Zola, the deal breakers, are the most important.

• - Look at feedback coming from detractors - they are frequently clearer guides than promoters

• - Carefully select when to seek NPS feedback. Zola solicits feedback monthly and at milestone. Create a habit to survey at least a portion of your users every month.

• - Circle back to each person who leaves comments. It’s an exercise not only in closing the loop, but also strengthening the community one member at a time.

• - To keep all employees informed and current, present detailed results from each month’s NPS survey in an All Hands meeting.

Find link here- http://firstround.com/review/heres-why-zola-cherishes-its-nps-detractors-as-much-as-its-promoters/

The Netflix HERMES Test: Quality Subtitling at Scale

Netflix has launched an online subtitling and testing platform, HERMES[1]. It’s open for both fulfillment partners and individuals. The first step in enrolling for subtitling job is to pass series of tests and once selected, the subtitlers will be recommended to Netflix vendors or work with Netflix directly in future [2]. 

As per a Medium post[3] by their Director of Globalization, the aim of this is for Netflix to maintain a standard across all of their third party translation partners to ensure constant quality and reliability. 

[1]: https://tests.hermes.nflx.io/

[2]: https://backlothelp.netflix.com/hc/en-us/articles/115005277788-After-I-complete-the-test-how-will-I-know-if-I-ve-been-selected-to-do-subtitling-work-for-Netflix-

[3]: https://medium.com/netflix-techblog/the-netflix-hermes-test-quality-subtitling-at-scale-dccea2682aef

Book review- Product Leadership

I had a chance to read “Product Leadership: How Top Product Managers Launch Awesome Products and Build Successful Teams” by Richard Banfield, Martin Eriksson, Nate Walkingshaw. What inspired me to read this book was interviews with more than 100 product managers and how they built awesome products. Unfortunately, this book didn’t live up to my expectation. More details down below but first some pros of the book.

This book is good compilation of what product management is all about. It even takes it a step further to explain product leadership and share how it’s different from product management. If one is starting out in product management, this can be a good beginning point.

Now on why I think this could have been better. First, there is nothing new in this book. If you have spent few years in product management and have read some books already (likes of Lean Startup, The Four Steps to the Epiphany, Hard things about hard things) or read/watched online stuff (likes of Sam Altman series), the only thing that this book serves is to refresh some of those concepts.

Second, I felt that the writing style was too verbose, making it boring to read. Few ideas were repeated throughout the book. For example, part 2 of the book calls out how a PM is different across various organizational states, that is startup, emerging and enterprise, but there doesn’t seem to be much difference in the ways a PM would perform his job. Instead of this, exact difference in each of these would have been a better way of putting things together.

Third, the USP of this book was interviews with more than 100 Product Managers. These interview snippets spread across the book seem to be just touching the surface with no real-life examples of why they feel so and/or any practical actionable items for readers. This makes all concepts and ideas look weak and half-cooked.

Fourth, some of the concepts attributed to product leadership are true for general management. Example of these include, how to hire a team, buy vs build decisions, how to work with external partners etc.

Overall 2 out of 5 for me.

North Star Metric and a Check Metric

The North Star Metric is the single metric that best captures the core value that your product delivers to customers. Optimizing your efforts to grow this metric is key to driving sustainable growth across your full customer base. 

However, NSM is not sufficient. Growing it in an unsustainable or unconstrained way can lead to challenging situations and inhibit long-term growth. For example, the NSM for an e-commerce company is orders. When the company runs a price promotion, orders get a nice boost. 

Welcome Check Metric - It’s a metric that constrains the NSM and ensures that the NSM grows in a way that is sustainable and creates long-term value. Example, for an e-commerce company, a possible check metric could be Gross Profit, which leads to unit economics staying sustainable and strong.

Read more about Check Metric here- https://medium.com/@gokulrajaram/the-second-most-important-metric-for-every-company-df958ff8c5ec 

What I learned from reading thousands of app reviews

Dropbox PM read thousands of reviews about their app and collated some interesting learnings Find below some interesting excerpts –

• 70% of our app reviews were either 5-star or 1-star reviews.
• People want to know what’s going on - After we stopped writing release notes, 12% of reviewers complained about our generic release notes.
• Thanks to our app reviews, I learned all sorts of interesting ways people use Dropbox.
• Ratings differ a lot by country - In the United States, we have roughly the same amount of 5-star reviews as 1-star reviews. But in Japan, we have almost twice as many 1-star reviews as 5-star reviews. My best guess is that there are two things at play here: Translation quality, cultural bias.

Here is the link to complete article - https://medium.com/dropbox-design/are-app-reviews-worth-reading-518d3211872f