User Avatar

Visit me to learn about Tech Careers, Product Management More. Experienced operator; Beginner online writer. Dad.

5 points checklist to help you decide if ML is a good fit for your next project

"Software Is Eating the World, but AI Is Going to Eat Software." As NVIDIA CEO described 2017. And yet, not every product problem is a good fit to be solved by AI. As a Product Manager, you need to focus the team on the user problem and the business outcome. AI/ML is a powerful tool at your disposal.

Many jump on the bandwagon of ML without thoroughly assessing if their problem is the right one and if their organization is ready.

Use this quick checklist to assess if your product problem is solvable by ML:

The problem requires making a lot of decisions.

It takes a couple of editors to determine the order each day of articles in the printed version of a paper. However, it will be infeasible for Twitter to employ editors to sort its feed per person. Examples for such problems include:

  • Recommendations: People who bought this product also bought this, people you may know, feed ranking, etc.

  • Pattern recognition: Email spam labels, determining if a picture contains a face, etc.  

Each decision is a low consequence. 

If we get it wrong once in a while, no one will die. ML isn't perfect. While it performs many tasks better than humans, it can't drive a car on a snowy day or land an airplane. High stakes usually (still) involve humans.

Humans can agree on the "right" decision to make

Either the decision has a right answer, ("ground truth"). We can agree this email is Spam, and the other isn't, or we agree on the outcome of the decision. For example, by watching this video, the user is more likely to come back to our platform tomorrow.

Your organization has access to or can generate (or buy) a lot of relevant past data.

While there are some developments in this area, and new techniques like GAN generate data to learn from, you need a strategy to collect data in most cases. Options include:

  1. Collect data in the product (Video completion on YouTube)

  2. Human Raters (Google Search, TikTok Integrity); or Drivers (Waymo)

  3. Buy data from other companies or trained models.

This is key; without data, your models are worthless.

Your organization is committed to this path; you have the talent (or can hire) & resources, and we understand this will take time to get right.

For example, at Google, we developed a model to determine how important each landmark in the world is for tourists (Search: "Things to do at <city>"). We had a world-class team and leveraged all google advanced AI and troves of data. Still, it took us a year to get a result we put in front of users. And, another year of iteration to a level where we proud of. We heavily utilized rates and travel editors for ground truth data.

On top of this, you need to retrain models periodically, keep your data fresh, and don't forget about technical debt.

I hope this note will help you determine if your product problem and organization ready to use AI. I'd be super grateful if you drop a line on Twitter for any feedback or share your learnings!