Major Challenges for Machine Learning Projects

Although scientists, engineers, and business mavens agree we might have finally entered the golden age of artificial intelligence when planning a machine learning project you have to be ready to face much more obstacles than you think.

Deep learning algorithms like AlphaGo are breaking one frontier after another, proving that machines can already be able to play complex games “thinking out” their moves. Automation has more applications than ever before: from email classification, music, and video suggestions, through image recognition, predictive maintenance in factories, to automatic disease detection, driverless cars, and independent humanoid robots.

If this in-depth educational content on implementing AI in the business setting is useful for you, subscribe to our Enterprise AI mailing list to be alerted when we release new material. 

Understand the limits of contemporary machine learning technology

Many companies face the challenge of educating customers on the possible applications of their innovative technology. Machine learning engineers face the opposite. Entrepreneurs, designers, and managers overestimate the present capabilities of machine learning. They expect the algorithms to learn quickly and deliver precise predictions to complex queries. They expect wizardry.

Because of the hype and media buzz about the near coming of general superintelligence, people started to perceive AI as a magic wand that will quickly solve all problems – be it automatic face recognition or assessing the financial risk of a loan in less than a second. It’s not that easy. Not at all.

In fact, commercial use of machine learning, especially deep learning methods, is relatively new. They require vast sets of properly organized and prepared data to provide accurate answers to the questions we want to ask them. A business working on a practical machine learning application needs to invest time, resources, and take substantial risks.

A typical artificial neural network has millions of parameters; some can have hundreds of millions. A training set usually consists of tens of thousands of records. While a network is capable of remembering the training set and giving answers with 100 percent accuracy, it may prove completely useless when given new data. The mechanism is called overfitting (or overtraining) and is just one of limits to current deep learning algorithms.

The black box problem

The early stages of machine learning belonged to relatively simple, shallow methods. For example, a decision tree algorithm acted strictly according to the rules its supervisors taught it: “if something is oval and green, there’s a probability P it’s a cucumber.” These models weren’t very good at identifying a cucumber in a picture, but at least everyone knew how they work.

Deep Learning algorithms are different. They build a hierarchical representation of data – layers that allow them to create their own understanding. After analyzing large sets of data, neural networks can learn how to recognize cucumbers with astounding accuracy. The problem is that their supervisors – the machine learning engineers or data scientists – don’t know exactly how they do it. The problem is called a black box.

Artificial Intelligence supervisors understand the input (the data that the algorithm analyses) and the output (the decision it makes). While the engineers are able to understand how a single prediction was made, it is very difficult to understand how the whole model works.

Some AI researchers, agree with Google’s Ali Rahimi, who claims that machine learning has recently become a new form of “alchemy”, and the entire field has become a black box. It is a significant obstacle in the development of other AI applications like medicine, driverless cars, or automatic assessment of credit rating. What if an algorithm’s diagnosis is wrong? How will a car manufacturer explain the behavior of the autopilot when a fatal accident happens? How will a bank answer a customer’s complaint?

The black box is a challenge for in-app recommendation services. It turns out that web application users feel more comfortable when they know more or less how the automatic suggestions work. That is why many big data companies, like Netflix, reveal some of their trade secrets.

The research shows artificial intelligence usually causes fear and other negative emotions in people. People are afraid of an object looking and behaving “almost like a human.” The phenomena is called “uncanny valley”.

We accept machines that act like machines, but not the ones that do the human stuff, like talking, smiling, singing or painting. Of course, this may change with time, as new generations grow up in a digital environment, where they interact with robots and algorithms.

Talent deficit

Although many people are attracted to the machine learning industry, there are still very few specialists that can develop this technology. A good data scientist who understands machine learning hardly ever has sufficient knowledge of software engineering.

The problem is drastic. The Chinese tech giant Tencent estimated at the end of 2017 that there were just about 300,000 researchers and practitioners dealing with AI worldwide. Element AI, nn independent company, estimates that “fewer than 10,000 people have the skills necessary to tackle serious artificial intelligence research”. Machine learning engineers and data scientists are top priority recruits for the most prominent players such as Google, Amazon, Microsoft, or Facebook.

It makes salaries in artificial intelligence field skyrocket, but also makes the average quality of specialists available on the market plummet. We wrote about general tech brain drain before. With machine learning, the problem seems to be much worse.

According to NYT in the US, people with just a few years of experience in artificial intelligence projects earned in up to $500,000 per year in 2017, while the best will get as much as NBA superstars. In a court filing in 2016, Google revealed that one of the leaders of its self-driving-car division earned $120 million in incentives before he left for Google’s competitor – Uber.

Data is not free at all

As I mentioned above, to train a machine learning model, you need big sets of data. It may seem that it’s not a problem anymore, since everyone can afford to store and process petabytes of information. While storage may be cheap, it requires time to collect a sufficient amount of data. Moreover, buying ready sets of data is expensive.

There are also problems of a different nature. Preparing data for algorithm training is a complicated process. Here’s an interesting post on how it is done. You need to know what problem you want your algorithm to solve, because you will need to plan classification, clustering, regression, and ranking ahead. You need to establish data collection mechanisms and consistent formatting. Then you have to reduce data with attribute sampling, record sampling, or aggregating. You need to decompose the data and rescale it. It is a complex task that requires skilled engineers and time. So even if you have infinite disk space, the process is expensive.

If you plan to use personal data, you will probably face additional challenges. People around the world are more and more aware of the importance of protecting their privacy. They may be unwilling to share them with you or issue a formal complaint if when they realize you did it, even if you obtained all they gave you their consent.

Personal data and big data activities have also become more difficult, risky and costly with the introduction of new regulations protecting personal data, such as the famous European General Data Protection Regulation.

The technology is very young

Once again, from the outside, it looks like a fairytale. The biggest tech corporations are spending money on open source frameworks for everyone. The Alphabet Inc. (former Google) offers TensorFlow, while Microsoft cooperates with Facebook developing Open Neural Network Exchange (ONNX). These systems are powered by data provided by business and individual users all around the world.

However, all these environments are very young. The first version of TensorFlow was released in February 2017, while PyTorch, another popular library, came out in October 2017. Web application frameworks are much, much older – Ruby on Rails is 14 years old, and the Python-based Django is 13 years old.

On one hand young technology uses the most contemporary solutions, on the other, it may not be production-ready, or be borderline production ready.

You need time to achieve any satisfying results and planning is difficult

Traditional enterprise software development is pretty straightforward. You have your business goals, functionalities, choose technology to build it, and assume it will take some months to release a working version.

In machine learning development has more layers. The engineers are writing a program that will generate a program, which will learn to perform the actions you planned when setting your business goals. Just adding these one or two levels makes everything much more complicated.

Machine learning takes much more time. You have to gather and prepare data, then train the algorithm. There are much more uncertainties. That is why, while in traditional website or application development an experienced team can estimate the time quite precisely, a machine learning project used for example to provide product recommendations can take much less or much more time than expected. Why? Because even the best machine learning engineers don’t know how the deep learning networks will behave when analyzing different sets of data. It also means that the machine learning engineers and data scientists cannot guarantee that the training process of a model can be replicated.

It’s very likely machine learning will soon reach the point when it’s a common technology. Nevertheless, engaging in a AI project is a high risk, high reward enterprise. You need to be patient, plan carefully, respect the challenges this innovative technology brings, and find people who truly understand machine learning and are not trying to sell you an empty promise.

This article was originally published on Netguru and re-published to TOPBOTS with permission from the author.

Enjoy this article? Sign up for more updates on AI business applications.

We’ll let you know when we release more technical education.