Author Archives: Vivek Vijayan

About Vivek Vijayan

A Technology Product Manager - I derive satisfaction when my product intelligently makes life easier for users.

How can Product Managers extract value from Data Scientists

5374735875_8232501d71_z

Product Managers are supposed to be metrics driven and are used to extracting and analysing data to understand how the product has been performing – for the customers/users and for the business. This is usually done as an operational exercise on a day to day basis or sometimes to figure out the cause of something unusual happening on the product. This is a typical usage of dash-boarding and business intelligence; perfectly fine if you do not (or cannot) collect data beyond a few key metrics.

Today, most enterprise and consumer products collect and store  enormous amount of data on the usage of the products. The usefulness of this data goes much beyond dashboards and business intelligence visualisations. Product Managers – who are serious about their products should take the help of Data Scientists – specialists who know to combine statistics, data mining, analytics to extract knowledge out of data. Usually some Product Managers find it difficult to find out when and how to collaborate with a Data Scientist. My experience with a few products which collect “huge” amount of data has been that Data Scientist can help in two situations:

a) When you know what you want out of your data  and b) When you want to explore about what to do with your data

When you know what you want out of your data: This is a part of product development or operations. Products need slicing and dicing of data to understand user behaviour. Without a Data Scientist one would end-up looking at the major metrics and struggle to understand several underlying behaviours. You need a data scientist to figure out reasons for the symptoms and institutionalise a framework such that it is visualised and understood. There are also cases where products need to separate the wheat from the chaff and the accuracy of achieving that measures the success of the product;Data Scientists help you in working with the development team to ensure that it is done at scale with minimum errors. In hypothesis based experiments, the expert helps you to make sure that you reject or accept the hypothesis with data that is statistically significant.

An example of this type of project would be when you want to extract FAQs from mailing lists – where you need to device a model to efficiently generate the FAQs from a pile of user generated content.

When you want to explore about what to do with your data: Customer or user interviews usually bring back a truck load of anecdotes, ideas, issues and requests. Similarly market and competition throws at you plenty of data points. They say, anecdotes are plural of data, but such things are easier to say. A Data Scientist can help you find the hidden meaning in the data which can help you create a product roadmap. This is more of an exploratory exercise where you need to work more closely with the Data Scientist than in the previous case.

An example of this type of project would be when anecdotal evidence suggests that high bounce rate of users for your web application is due to bad first time user experience; before you embark on improving the experience, data might be able to tell you if such a pattern really exists in at least some form.

Finally, do not forget that Product Manager is the custodian of the product. Data Scientists are specialists who can help you achieving the product goals just like how a Software Engineer or an Architect would.

Picture courtesy: Bragg

Product Manager: you are what you measure

Product Managers don various avatars; that is the nature of the work. However, the product metrics that you measure defines what you really do.

4311409835_51c05f07ca_z

 

What you measure

Product Manager type

  •  Rate at which features are churned out or feature backlog is consumed/burnt
  • Complexity of the features added to the product
  • This role is essentially that of a Program Manager and a Project Manager as you are getting things done and helping ship products/features
  • Number of new users of the product (for consumer products) and number of businesses signed-up for trial or some sort of an initial engagement (for enterprise products)
  • Effectiveness of the new-users of the product – cost incurred to attract a new user/business
  • A Product Marketing role – where you promote your product to the right target segment and get the best prospective customers/consumers to use the product
  • User engagement (say bounce rate, time spent, click-throughs etc) for consumer products and value addition for businesses (cost saved, increased productivity) for enterprise products
  • Experiments run with clear hypothesis and product discoveries made
  • The heart of this role is that of a Product Manager – who is responsible for engaging the users of the product in every manner possible and extracting the maximum value. Also, one who runs selected experiments to discover ‘what next?’

In small organizations (mostly startups) – all these roles are packed into one and in large organizations – these roles are watertight compartments. A Product Manager who dons multiple hats should make sure that he/she does not spend disproportionate time on only one activity lest you run the risk of not really being a Product Manager in practice.

Essentially, you are what you measure.

Photo Credit

3 Questions to Product Manager Tyler Strand

 

DSCN0265We start the interview series of Product Mantra in 2016 with a young technologist and product leader Tyler Strand, who is very clear in his thoughts about various aspects of product management. Product Mantra asked Tyler three questions.

Product Mantra: Do you see the role of a Product Manager undergoing an evolutionary change, say in the next 5 years?

Strand: We’re at the beginning of a transition towards corporations viewing technology as a first class citizen and an ongoing investment. I see a lot of (older, bigger, less-technical) companies hiring what I’d call project managers and asking them to liaise between, say, an offshore development firm and a design consultant to get an app built. As digital products become more central to traditional businesses, I think we’ll see the requirements for a PM role shift more towards what you’d read in a Facebook or Google PM job description. More technical, more opinionated, more holistic. The position will become more widely about defining and executing on a product strategy and less about coordinating between parties and keeping a project on schedule, which is how it’s often seen today.

Product Mantra: Is there merit in saying Product Managers should be master of at least one skill – say data analytics or UX or software development or something else?

Strand: One of the things I like about having PMs be masters of at least one skill is that it makes them flexible. When a PM’s daily todo list is slow, they can jump into a specific project and help move things forward that way. I also think seeing a PM in the weeds is huge for building credibility with the team. When a PM is only “managing”, they are at risk of being criticized for doing nothing. That said, there are plenty of products and teams, particularly larger ones, that necessitate full time product management from at least one person. So there’s definitely a time and a place for a PM who doesn’t have mastery over a more “concrete” skill.

Product Mantra:  At what stage should a startup hire a Product Manager and why?

Strand: In early stage startups, a founder will usually play the role of a PM while getting the product off the ground. The founders are the ones guiding the product, staffing the team, and ultimately deciding what gets built and what doesn’t. As the company grows, founders have to think about more than just the product itself. Recruiting, business development, marketing, sales, new product development, fundraising, and legal become more time consuming, making the need for a dedicated PM critical. Exactly when this happens will depend on what the company does and the skill set of its founders. In general, I’d say once the team is about 20 people is when a company should start thinking about making a formal PM hire.

Thanks a lot for your time Tyler.

Tyler is a Computer Science graduate from Stanford University and co-founded hostess.fm, while he was still a student. In his present avatar he is a technologist and product manager. His post on Medium “What Do You Do, Exactly?”  is a great read for anyone who wants to know what product managers do or for even product managers who are finding it a challenge to articulate what they really do. Follow Tyler’s tweets here.

What do people search about “Product Management”?

Curiosity to find what people search for about product management, led me to dig a bit into what people searched online in the last one year. I split the the search queries into these categories:

  • Job Search like “product management interview questions”
  • Software tools like “product lifecycle management tools”
  • Skill upgrade like “product management certification”
  • Informational like “what does a product manager do?”

Share of different categories of search queries

image.png

 

Cost of search engine advertising for different categories

Not surprisingly enough, the average cost for search advertising is the highest for software tools, but a bit surprising was that the least was for jobs. The normalised data is shown below:

image (1).png

Top queries that saw the maximum increase in the last year

Queries Increase over previous year
digital product management +50%
agile product management +40%
product management internship +40%
Product manager salary +40%

Top four queries in different categories

Top search queries in Informational category
product lifecycle management
what is product management
product portfolio management
associate product manager
Top search queries in Job Search category
product manager salary
product manager jobs
product manager interview questions
product manager resume
Top search queries in Skill Upgrade category
product management certification
product management training
product management courses
product management books
Top search queries in Software Tools category
product lifecycle management software
product data management software
free product management software
product portfolio management software

Looking at the data above as a whole, I have a few observations:

  • Product Management seems to be getting matured. I was able to categorise 90% of the 500+ unique queries into the above categories unambiguously
  • Jobs are being searched heavily but advertisement costs are the lowest – perhaps head hunters and companies are a bit weary of getting way too many unsuitable candidates?
  • “Associate Product Manager” – makes it to the top 4 in the category – shows great interest in the stream and to make it while you are young
  • I expected “lean” to be as popular as “agile” as far as product management is concerned, but “agile” beats “lean” by a very wide margin

Perhaps a similar exercise every six months might reveal how product management has been evolving over time.

 

Shadow Product Manager – salvaging out of the situation

There are scores of situations where a Product Manager might find herself/himself getting entangled completely in tactical aspects of product management because the situation demands it or there are simply too much chaos in the organ223670395_7ca0c7a061_zization due to a disorganised way of working. There are also situations when one is with an off-shore development team, “far away” from the customers and key decision makers. It is easy to get completely disoriented and frustrated at the situation. There are ways to salvage out of the “situation” of being a “Shadow Product Manager”:

  1. In-depth data analysis: They say, data is the plural of anecdotes. I have found that in chaos, people miss out in-depth data analysis; there is always some signal hidden in the pile of data for you to figure out a feature for your road map – at least a clear hypothesis. Once, I remember that analysis of support tickets over a period of time told me that users were not rebooting the machine after installing our application as most users postponed doing that. We worked to build something that required no restart and made life much simpler for the user.
  2. Collect anecdotal evidences via social media: There are always keen users of your product who are upset when something does not work and make their displeasure known through social media – but are more than happy to help if you collaborate with them. Do not react to each such signals, but pick and choose whom you want to respond to.
  3. Get into the good books of one or two sales folks: Sales personnel meet the customers all the while but lack of product makes them either succumb to pressure or to overcommit and harass the product manager. However, once you befriend them  by helping them understand the product better, you can get involved in sales calls – a lot of them are remote and get connected to the buyer (a typical case of B2B product).

Where most often I as a Product Manager have flawed is to assume that product folks who are geographically near the customer would have spoken to the customer and gathered anecdotal evidences. I have seen instances where not even a decent email was exchanged and the customer was happy when I spoke to them over phone.  There is really nothing like a ‘shadow product manager’ – there are only ‘self motivated product managers’ and ‘complaining product managers”.

Photo Credit: Hamid Saber

Interview: 3 Questions to Product Manager Rohini

We are back with the monthly interview series. All our past interviews were with veterans of the industry. This time we have an young and emerging thought leader of Product Management – Rohini Venkatraman of So You Want to Manage a Product fame. Here we go:

Product Mantra: In your first stint as a Product Manager, did you discover that there were misconceptions about Product Management? If so, can you elaborate on the biggest misconception.Rohini

Rohini: In my first role as a Product Manager, one of my tasks was to submit an application to the Mac App Store. As part of this process, I was asked to design our sales page. I took my assignment literally and jumped right in to get my hands dirty with Photoshop. After creating a page I thought was beyond awesome, I emailed the page to my manager. His response was not dripping with praise as I expected. Instead it read, “Great. Did this come from our designer? I’d like to push her on color scheme.” Designer?! I asked my computer. What is he talking about?  This is how I learned that, especially at a larger corporation, a product manager does not create visual designs. She also does not write code. Your designer is your design expert. Your engineer is your programming expert. And, you, the product manager, are the expert on whether the design and functionality meet the specific user need at hand.

Product Mantra: What should a Product Manager be cautious about?

Rohini: As the product manager for a small business lending product, my goal was to improve the small business financing process. I spent a lot of time talking to small business owners about their attempts to receive financing. When asked what could be different about the process, many said that they needed the process to be faster — the long process was disruptive to their business. Based on this feedback, my team tested an idea where small businesses could apply and get pre-approved for financing online in minutes. The shocker? Even when they were quickly approved for a loan, businesses didn’t jump at the money. As we started to observe their behaviors and probe further, it became clear that while people said that the process needed to be faster, what they really needed was education around their options and guidance in choosing the best one. Had we built out a product solely based on what people said they needed, we would have ignored the opportunity to build a product that people actually needed. Among other things like looking both ways before crossing the street, a product manager should be cautious about distinguishing between needs and wants of key stakeholders. Not just needs and wants of customers, but also those of engineers, designers, and management. This can be difficult because needs and wants are often conflated. People tend to express wants as needs. It’s up to the product manager to tease the two apart, then prioritize the needs. Needs are the minimum requirements that solve a big problem for customers, that achieve team goals, and that validate the business opportunity. Wants can wait. Without clearly making this distinction, you risk wasting a lot of company time and resources and building something that nobody wants or needs.

Product Mantra: Intuition is very important for a good Product Manager. How do you develop intuition?

Rohini: To me, this question is really asking, “How do (good) product managers always seem to have the right answer?” I’ll let you in on a secret: It’s not intuition. Good product managers are continually doing their research — competitor analyses to understand the market in which their product lives, user research to profoundly understand the customer problem at hand, facilitiating team conversations to understand technical paths and design decisions, organizing cross-functional discussions to understand implications on the business — so that they know everything there is to know about their product and roadmap at any given point. This means that when they need to make a quick decision, they are fully equipped with all the information they need to make the best decision possible given time and resources. When you’ve consumed a lot of data, you can feel pretty confident trusting your gut.

Product Mantra: Thanks Rohini for participating in our interview series and providing your insightful answers.
Rohini has a neat blog where Product Management is one of the many things she writes about. You can also follow her lively tweets.

Three question for Product Manager Teresa Torres

We start the Q & A series of 2015 with Teresa Torres of ProductTalk fame. As usual – just three Teresa-Torresquestions and unedited answers!

Product Mantra: Gut feeling and data-driven approach – when do you prefer one over the other?

Teresa: I don’t prefer one over the other. Both are important and need to play a role in decision-making. Gut feelings or intuition is usually the result of pattern matching based on our prior experience. We notice something in a current situation that reminds us of past experience and we generate a solution based on what worked in our past. Whether or not this is a good solution depends on whether the current situation is similar to our past experience in relevant ways.

The challenge is our past experience might be similar in superficial ways and different in significant ways, meaning that a solution that worked in a past experience may be irrelevant to our current situation. We aren’t very good at recognizing when this occurs. Our intuition finds a solution quickly and we aren’t very good at slowing down and asking in which ways is this situation similar and different.

This is where a data-driven approach can help. We should listen to our intuition, but we shouldn’t trust it blindly. Instead, we should design experiments to test whether or not our proposed solution works in the current situation.

The optimal form of decision making is to listen to our intuition to generate insights and then to use a data-driven approach to test those insights.

Product Mantra: Intuition is very important for a good Product Manager. How do you develop intuition?

Teresa: Intuition comes from experience and reflection. Both are critical. We all know people who have years of experience who stopped improving long ago. And similarly, we all know people who seem experienced beyond their years. The difference is often reflection.

It’s not enough to log 10,000 hours of practice. In fact, the research that suggests it takes 10,000 hours of practice to develop expertise probably doesn’t apply to fields like product management. With the type of complex business problems that we tend to face, it’s hard to get expert coaching and it’s even harder to get real-time feedback, both of which are necessary for the deliberate practice required by the so-called “10,000 hour rule.”

You can’t just log experience. You have to take the time to reflect on your experience. This means you need to grow your awareness around how you make decisions and when you tend to be wrong. For product managers, I recommend doing the following for each new product idea:

  • Write down what impact you expect the product change to have.
  • Estimate an exact amount with a rationale for why.
  • Design an experiment to test your thinking.
  • Track your results.
  • Compare the actual outcomes to your estimated outcomes.
  • Do the work to understand the gap between the actual outcomes and your expected outcomes.

If you do this over and over gain, your intuition will improve. But remember, a finely tuned intuition doesn’t mean you can ever stop experimenting (as you won’t know when you are wrong), it just means your cycles will get shorter – you’ll know what to test sooner, your tests will get better, and you’ll start to move more quickly.

We need to let go of the idea that as product managers we can be right more often than not. Instead, we need to assume we’ll be wrong and adjust our methods to account for this.

I suspect the real question behind this question is how to we get better at product management. Product management is a broad function and it’s impossible to build expertise in every aspect of it. I recommend getting good at the basic fundamentals which I define as empathy, active listening, curiosity, intellectual honesty, statistical competence, root-cause analysis, visual communication, and abductive reasoning; cultivating the right mindsets such as being human-centered, experimental, collaborative, and metacognitive; and picking one or two areas of depth to develop deep expertise. You can read more about my philosophy on developing product expertise here.

ProductMantra: What was your New Year resolution for 2014 as a Product Manager? Based on that how do you frame one for 2015.

Teresa: I don’t set New Year resolutions. Too much research suggests that they don’t work. Instead, I set learning goals.

In 2013, I wanted to learn about content marketing and get better at cohort analytics. That year I worked at AfterCollege  I built out a content marketing team that is building awareness and growing the student audience and I implemented cohort analytics that accelerated our rate of learning and allowed us to get traction with a new product much faster than we otherwise would have been able to do.

In 2014, I worked as a full-time consultant coaching product teams on how to integrate user research, experimentation, and meaningful metrics into the product development process to help them make better product decisions. My goal for the year was to invest 100% of my effort into making this a viable business and this led to a variety of learning goals around how to support a growing consulting business. I also invested heavily in growing my statistical knowledge so that I was better equipped to coach my clients on understanding their experiment results.

Heading into 2015, my consulting business is strong and I’m less concerned about growing my business and I’m shifting my focus to my bigger goal of how do we invest in product management as a function – how do we get better at building products. As a consultant, I get access to the way different companies build products. I can see what’s working and what’s not across several companies. This year I plan to focus my energy into translating those insights into more writing and more teaching.

I also want to get better at the skills that underlie both of those. I’m approaching writing as a craft and investing in my skills by reading and writing more. With teaching, I’m investing in teaching opportunities that extend beyond what I do in my blog or a one hour talk. I’m doing more workshops and I’m experimenting with new course formats. I want to help more people get better at building products, so this year I’m experimenting with how to reach more people in ways that allow them to practice the craft of product management.

ProductMantra: Thanks a lot Teresa.

Teresa is a product coach helping teams adopt user-focused, hypothesis driven product development practices. You can read her views here and follow her tweets here.