top of page

Can I trust the results of an AI?

We are using more and more devices and services that collects data and when we combine this data with Machine Learning we are able to make predictions, sometimes extraordinary predictions. But can I trust the results of an AI?

When a self driving car is part of an accident it becomes headline news, although human drivers makes mistakes all the time. Is it right that we hold human drivers and machines to different standards? We all allow humans to make mistakes at times, but to extend that compassion to machines feels unnatural.

AI is getting deeper and deeper into our lives, most of the time without us realizing it. And now we are starting to tackle problems that don't have any clear solution. Take a quire simple example of an AI that can detect a cat in an image, if we show an image of just a cat tail to that AI should it still mark that image as containing a cat?

The quality of a decision

This is basically the problem of sensitivity vs specificity, or "how good are we at detecting a cat?" vs "how good we are at detecting when our image doesn't contain a cat?". We can have maximum sensitivity if the detector always thinks there is a cat, but then the specificity will be really low (lots of false positives). Once we have a trained AI a human will need to take part in how to use the detector to get an acceptable weighing between sensitivity and specificity.

In order to have both high sensitivity and high specificity at the same time we need a really smart AI, that places all images in either the “true positive” or “true negative” category, and for that we need data to train on. It used to be that we needed a lot of data for training neural networks (for image detection preferably hundred thousand or millions of images), but recent advances allows us to train on one big set of data and use general insights from this training and retrain for our specific problem.

Reusing previous knowledge

Think of how a toddler learns to recognize objects, at first it must be immensely hard to understand even where a cat begins and ends in a picture, but by looking at many pictures (or real cats) the toddler learns to recognize simple objects. Once the toddler has learned to recognize cats, then he or she will be much faster at learning to recognize lions, by using insights from how a cat looks.

For AI this means that we start by training on a large dataset (like the ImageNet set with more than 14 million images). The AI will use this dataset to learn features like detecting fur, hands, buildings, vehicles, and so on. Later we retrain on our specific problem (like differencing between cucumbers and zucchini) and then we can get away with a much smaller data set (sometimes just ten images from each category).

Another trick we can use is to expand the data we have, an image of a cucumber will still be an image of a cucumber if I mirror it, change the brightness or skew it. Creating clever ways to expand your data can be a key ingredient in training an AI.

In the end we must still spend lots of (manual) work in massaging our data into a way so the AI can make sense of it. I think of it as how different teachers have different teaching capabilities. A good teacher (data scientist) can make a student (neural network) understand concepts better by presenting the information in smart ways.

The AI:s internal goal

When training the AI we solve an optimization problem: we want to minimize wrong predictions. When an AI solution is deployed in the real world we will move that optimization into the real world, which might cause unintended side effects. The classical example is a police station that uses AI for determining where to send patrol cars. If AI is trained on where there are lots of arrests, it will make more police cars go there and invariably even more arrests will be made in the same area, creating a feedback loop between the real world and the AI.

We need to consider that the AI just appears smart, it doesn't actually understand the real world. This is also why GDPR contains a section on the "right to non-automatic decision making". AI and humans need to work together, making use of each others strengths.


So what about the good parts? AI allows us to raise the lower bar. An AI medical doctor may not be as good as a human doctor on all tasks, but for the tasks that we can automate we will raise the lower bar, so you should never get worse diagnosis than what we can train an AI for. AI will help us automate lots of common tasks (think of Gmail's Quick Reply), or adopting a cookbook recipe to the contents in your fridge. We will get autonomous cars and 24/7 customer support via virtual assistants so we never need to wait in another telephone queue.

The future is bright, we just need to allow room for humans in the society processes we are currently creating. It is still humans that decide how to interpret the results from AI and the best results come when AI and people work together.

Framtiden ser ljus ut, men vi måste se till att vi människor får plats i de processer som vi håller på att skapa med AI. I slutändan är det människor som måste bestämma hur vi ska tolka resultaten från AI och bäst blir det när människor jobbar tillsammans med AI.

By Erik Man



bottom of page