Three paths to AI

I’ve consistently advocated a product-centered approach to AI, data science, and machine learning projects (which I just group together as “AI” for convenience). At their core, AI products are decision-making tools. They either make decisions or provide information to humans to help them make decisions.

As you think about what product to build, consider how the output of your AI system will be consumed. There are three different types of AI product:

• One-off reports
• Scheduled batch jobs
• Real-time systems

I’ll briefly discuss these three types of product, some pros and cons of each, and offer some advice about which to choose.

One-off reports

The first type of AI product is a one-off report that answers a specific question. This report is often a PowerPoint presentation or document. When the question has been answered and the answer has been communicated, the work is done.

Reports that may are suitable for AI have a few characteristics in common:

They produce answers to specific questions, and those answers inform a number of important long-range decisions.

The problem doesn’t change very frequently. In other words, the answers contained in the report won’t become outdated in the near term.

The project benefits from using a larger quantity and a wider range of data than have been used by the business in the past.

One instance of this kind of AI project is a customer segmentation report. It’s common for marketing professionals to design customer personas based on interviews, surveys, or other data that’s generated by the company. Customers are segmented according to those personas, and marketing campaigns are designed to target specific segments.

I often hear from marketing teams that they’re fairly happy with their existing segmentation categories, but they still wonder whether there’s a better way. They wonder whether there is a more data-driven approach that can utilize more of the information that the business already has about its customers. In this case, it’s natural to think about AI techniques, because those techniques typically enable the business to use a wider variety of data sources. They also enable a range of sophisticated clustering techniques to discover an optimal set of customer segments.

For this problem, it makes sense to deliver a one-off report based on research conducted by a data scientist who uses AI methods. It’s likely that there are better customer personas and segments which can be uncovered. If there are, then this insight would go into a report which is communicated to your marketing and sales teams.

Scheduled batch jobs

For the one-off report, it’s important that the problem domain doesn’t change too frequently. Otherwise, the insights become outdated too quickly to justify the effort that went into developing the report. For problems which do change relatively frequently, it’s sometimes useful to think about creating a scheduled batch job.

For example, suppose you have the ability to use AI techniques to predict which products are likely to be your best sellers. However, these products change frequently — for instance, perhaps you sell clothing, and new items are being produced all the time. Clearly, you don’t want a one-off report that will become outdated in a few weeks. Instead, you’d like to produce the same type of report every few weeks.

Scheduled batch jobs do this by producing similar analyses on a regular cadence. They are distinguished from one-off reports because they require automation and engineering work. Although it’s natural to assume that there’s not much difference in effort from one-off reports to scheduled batch jobs, this unfortunately isn’t the case. If a data scientist or engineer is producing a one-off report, they don’t have any reason to be concerned with ensuring that their work is repeatable. As a result, they will have collected their data and written code in a somewhat haphazard way, and their process will be undocumented.

In order to make the jump from one-off reports to batch jobs, it’s not acceptable to do things in a haphazard manner. There has to be a well-engineered data pipeline that retrieves the most up-to-date available data, cleans it, and runs it through a model. The output can’t be copy-pasted into a report by a human; it has to be output automatically into a human-readable format. All of the usual software engineering best practices have to be followed, including automated testing and monitoring. If these practices aren’t followed, you run the risk of producing erroneous output without knowing it. Your business decisions could be guided by defective analyses.

In short, the scheduled batch job moves into the realm of engineering and requires exactly the same level of support as other engineering products. So there is a big increase in effort and resources as soon as you introduce even this modest amount of automation.

Real-time systems

The most sophisticated type of AI product makes large numbers of decisions in real-time, the instant the decision is needed.

A familiar example of this is a fraud detection model used by a credit card company. The instant you swipe your credit card, information is sent to a model to detect fraudulent activity. That model makes a determination based on information about your purchase, the vendor, the amount, your geographic location, and other factors. It returns a result in real-time to approve or deny the transaction.

We interact with these real-time AI systems every day. Facebook decides which posts to show you in which order based on such a system; and Netflix uses real-time models to recommend movies to you. Any website that tailors its content on-the-fly based on specific information about you is probably driven by a real-time AI system.

In general, a real-time system requires roughly three or four times the engineering effort of scheduled batch jobs. The extra engineering effort comes from three different sources.

First, these systems have to be consumed by another system in your tech stack. For example, Netflix’s movie recommender is consumed by Netflix’s website. So you’re no longer building a single system; you’re building at least two.

The second source of extra engineering effort comes from the data requirements of the model. In order to work in real-time, the model has to be able to retrieve data very quickly. Not every database is able to accommodate this requirement. For example, if the data exists only in a data warehouse, it’s unlikely to be able to look up and send specific data quickly enough because warehouses aren’t designed for this purpose. So you might need to architect an entirely new system to store and quickly retrieve your data.

Third, you need to create the infrastructure required to host and run your model. This is often the biggest stumbling-block for new AI initiatives. A real-time model has to be hosted like a website or other service. It has to be able to listen for requests from your other systems, calculate an output, and send the output back to whichever system made the request. It needs to do this at scale, and have failover so that when a machine inevitably fails or there’s a network error, another machine can pick up the request.

Selecting a project

There’s no single answer to the question, “Which of these three systems should my business build?” It depends entirely on what kinds of questions need to be answered by the AI system. To properly answer the question, it’s crucial to have good product managers on the case.

But in general, there are some trade-offs to consider and pitfalls to avoid.

One-off reports require the least engineering effort, but they have very limited value. I generally advise not going this route if the business doesn’t already have momentum toward using AI. The counter-argument is that creating a one-off report is a good way to dip your toes in the water, so to speak, and get some experience without investing too much up-front. But I think this argument is unsound. One-off reports rarely have enough value to justify the effort required to produce them, and so your stakeholders will see AI producing limited value. And this takes the wind out of your sails, so to speak.

Furthermore, one-off reports don’t teach people very much about how to deploy AI systems at scale. It’s crucial when starting on the journey to AI that your people get experience with the unique technical challenges of deploying these systems. But there is no such opportunity when creating these reports.

So although I don’t advocate doing a “big bang,” high-risk project right away, I do advocate starting with a project that requires some engineering resources. If there’s an opportunity to build a scheduled batch job system that uses AI under the hood, and the system can create enough value for your business, then it’s a reasonable place to start. Unless you’re in the enviable position of being able to hire a team of experienced AI practitioners right away (or you can drop a few million dollars on consultant fees), the first couple of projects should be chosen to provide experience for the engineers who will ultimately own the implementation of other AI products in the future.

For some businesses, the only type of AI product that can provide significant value is a real-time system. If that’s the case, then you’ll have to invest a lot in engineering before you even see an AI product being developed, and you’ll need to think a lot about how to mitigate the project’s risk.