Using NPS feedback: Zola case study

Some interesting titbits on how Zola is using NPS feedback to improve product -

• - Bucket NPS feedback by theme. The issues that prevent people from picking Zola, the deal breakers, are the most important.

• - Look at feedback coming from detractors - they are frequently clearer guides than promoters

• - Carefully select when to seek NPS feedback. Zola solicits feedback monthly and at milestone. Create a habit to survey at least a portion of your users every month.

• - Circle back to each person who leaves comments. It’s an exercise not only in closing the loop, but also strengthening the community one member at a time.

• - To keep all employees informed and current, present detailed results from each month’s NPS survey in an All Hands meeting.

Find link here- http://firstround.com/review/heres-why-zola-cherishes-its-nps-detractors-as-much-as-its-promoters/

The Netflix HERMES Test: Quality Subtitling at Scale

Netflix has launched an online subtitling and testing platform, HERMES[1]. It’s open for both fulfillment partners and individuals. The first step in enrolling for subtitling job is to pass series of tests and once selected, the subtitlers will be recommended to Netflix vendors or work with Netflix directly in future [2]. 

As per a Medium post[3] by their Director of Globalization, the aim of this is for Netflix to maintain a standard across all of their third party translation partners to ensure constant quality and reliability. 

[1]: https://tests.hermes.nflx.io/

[2]: https://backlothelp.netflix.com/hc/en-us/articles/115005277788-After-I-complete-the-test-how-will-I-know-if-I-ve-been-selected-to-do-subtitling-work-for-Netflix-

[3]: https://medium.com/netflix-techblog/the-netflix-hermes-test-quality-subtitling-at-scale-dccea2682aef

Book review- Product Leadership

I had a chance to read “Product Leadership: How Top Product Managers Launch Awesome Products and Build Successful Teams” by Richard Banfield, Martin Eriksson, Nate Walkingshaw. What inspired me to read this book was interviews with more than 100 product managers and how they built awesome products. Unfortunately, this book didn’t live up to my expectation. More details down below but first some pros of the book.

This book is good compilation of what product management is all about. It even takes it a step further to explain product leadership and share how it’s different from product management. If one is starting out in product management, this can be a good beginning point.

Now on why I think this could have been better. First, there is nothing new in this book. If you have spent few years in product management and have read some books already (likes of Lean Startup, The Four Steps to the Epiphany, Hard things about hard things) or read/watched online stuff (likes of Sam Altman series), the only thing that this book serves is to refresh some of those concepts.

Second, I felt that the writing style was too verbose, making it boring to read. Few ideas were repeated throughout the book. For example, part 2 of the book calls out how a PM is different across various organizational states, that is startup, emerging and enterprise, but there doesn’t seem to be much difference in the ways a PM would perform his job. Instead of this, exact difference in each of these would have been a better way of putting things together.

Third, the USP of this book was interviews with more than 100 Product Managers. These interview snippets spread across the book seem to be just touching the surface with no real-life examples of why they feel so and/or any practical actionable items for readers. This makes all concepts and ideas look weak and half-cooked.

Fourth, some of the concepts attributed to product leadership are true for general management. Example of these include, how to hire a team, buy vs build decisions, how to work with external partners etc.

Overall 2 out of 5 for me.

North Star Metric and a Check Metric

The North Star Metric is the single metric that best captures the core value that your product delivers to customers. Optimizing your efforts to grow this metric is key to driving sustainable growth across your full customer base. 

However, NSM is not sufficient. Growing it in an unsustainable or unconstrained way can lead to challenging situations and inhibit long-term growth. For example, the NSM for an e-commerce company is orders. When the company runs a price promotion, orders get a nice boost. 

Welcome Check Metric - It’s a metric that constrains the NSM and ensures that the NSM grows in a way that is sustainable and creates long-term value. Example, for an e-commerce company, a possible check metric could be Gross Profit, which leads to unit economics staying sustainable and strong.

Read more about Check Metric here- https://medium.com/@gokulrajaram/the-second-most-important-metric-for-every-company-df958ff8c5ec 

What I learned from reading thousands of app reviews

Dropbox PM read thousands of reviews about their app and collated some interesting learnings Find below some interesting excerpts –

• 70% of our app reviews were either 5-star or 1-star reviews.
• People want to know what’s going on - After we stopped writing release notes, 12% of reviewers complained about our generic release notes.
• Thanks to our app reviews, I learned all sorts of interesting ways people use Dropbox.
• Ratings differ a lot by country - In the United States, we have roughly the same amount of 5-star reviews as 1-star reviews. But in Japan, we have almost twice as many 1-star reviews as 5-star reviews. My best guess is that there are two things at play here: Translation quality, cultural bias.

Here is the link to complete article - https://medium.com/dropbox-design/are-app-reviews-worth-reading-518d3211872f

THE END OF APPS: How Apple, Facebook, and Google are ushering in the next evolution of app experiences, and what that means for businesses

• Users are downloading apps less often and spending their time in fewer of the apps that they choose to install.
• 85% of users’ time is spent with just five apps
• 80% of all app users churn within 90 days
• That's a big problem for Google, Apple, Facebook, and the companies that rely on their app 

See this report from Business Insider how each tech giant has devised an approach that plays to its strengths.

How Big Data and ML helped Trump win

Nice article on how data from our digital footprint was used to make targeted campaigning for Trump - https://motherboard.vice.com/en_us/article/mg9vvn/how-our-likes-helped-trump-win 

The origin of approach

• The approach that Kosinski and his colleagues developed over the next few years was actually quite simple. First, they provided test subjects with a questionnaire in the form of an online quiz. From their responses, the psychologists calculated the personal Big Five values of respondents. Kosinski's team then compared the results with all sorts of other online data from the subjects: what they "liked," shared or posted on Facebook, or what gender, age, place of residence they specified, for example. This enabled the researchers to connect the dots and make correlations.

• Remarkably reliable deductions could be drawn from simple online actions. For example, men who "liked" the cosmetics brand MAC were slightly more likely to be gay; one of the best indicators for heterosexuality was "liking" Wu-Tang Clan. Followers of Lady Gaga were most probably extroverts, while those who "liked" philosophy tended to be introverts. While each piece of such information is too weak to produce a reliable prediction, when tens, hundreds, or thousands of individual data points are combined, the resulting predictions become really accurate.

•  In 2012, Kosinski proved that on the basis of an average of 68 Facebook "likes" by a user, it was possible to predict their skin color (with 95 percent accuracy), their sexual orientation (88 percent accuracy), and their affiliation to the Democratic or Republican party (85 percent).

Cambridge Analytica, a UK based firm, was hired to manage Trump's campaign. This is how they took Kosinski's approach and created a similar model - 

•  "At Cambridge," he said, "we were able to form a model to predict the personality of every single adult in the United States of America." The hall is captivated. According to Nix, the success of Cambridge Analytica's marketing is based on a combination of three elements: behavioral science using the OCEAN Model, Big Data analysis, and ad targeting. Ad targeting is personalized advertising, aligned as accurately as possible to the personality of an individual consumer.

• Nix candidly explains how his company does this. First, Cambridge Analytica buys personal data from a range of different sources, like land registries, automotive data, shopping data, bonus cards, club memberships, what magazines you read, what churches you attend. Nix displays the logos of globally active data brokers like Acxiom and Experian—in the US, almost all personal data is for sale. For example, if you want to know where Jewish women live, you can simply buy this information, phone numbers included. Now Cambridge Analytica aggregates this data with the electoral rolls of the Republican party and online data and calculates a Big Five personality profile.