If you ask any Product Manager on what consists his/her daily routine at work, you will in a majority of the cases hear monitoring the dashboard as one of the starting items on any given day. As Product Managers we must be on the top of the dashboard. A recent incident makes me think, if we really should on a day-to-day basis “worry” about the change in metrics.

A few weeks back, I was palpitating looking a key metric which which was showing a downward trend since the last 3 days for no apparent reason. I drew a series of hypothesis and started chasing every hypothesis – some were a wild goose chase – with none of the hypothesis turning turning true. I looked at the external environment – nothing sudden was happening in the market. I looked at the data of the previous year – nothing really was co-relating with the present one. I was advised by my senior to wait and watch as there seems to be no apparent reason – sometimes there are short waves that come and go for no apparent reason and breaking your head over something of that sort might be a waste of time. I patiently waited. The metric came back to normal. Did I have to really deep dive and waste my time? Alternatively, if it was an upward trend, would I have created some hypothesis from thin air co-relating it with some changes we had made much earlier? We must look at the dashboard, not for blips, but as a whole over a well defined period of time. Well, if you had just launched a AB test you might be tempted, but even in that case until you get a statistically relevant number – do you really want to react?

Next time, when you ask a Product Manager what their daily tasks are and if you get a response saying “monitoring the dashboard and ensuring if things were OK”, ask them what happens if “things do not seem OK”. That will give you an insight into how an organisation works – impulsive and reactive  or well planned and organised.

Photo credit: Scania Group