Are we prepared to live in world where algorithms decide on life and death?

Basis of this is an episode of philosophieHow to Solve a Moral Dilemma. The episode discusses the trolley problem in the context of self driving cars (approximately twelve-and-a-half mins in).

Quoting from Wikipedia, the trolley problem is:

The trolley problem is a series of thought experiments in ethics and psychology, involving stylized ethical dilemmas of whether to sacrifice one person to save a larger number.

In the context of self driving cars, this poses uncomfortable questions on what happens in case of an accident. What happens if there is a decision to be made who has to die? I.e. either the three people on the side of the street or the one in the middle of the street. Or the people sitting in the car?

And who is responsible? In some jurisdictions the answer has been: there has to be human driver that can override in such situations. The driver is then responsible. But why have self driving cars then? As the driver has to be permanently ensuring that no accident can happen.

What happens in jurisdictions where complete self driving cars are allowed? The Artificial Intelligence (AI) controlling the car makes the final decision on what happens. Hence the AI is making the decision over life and death of humans.

The unfortunate truth about AIs is that they are nothing other than extremely complex computer algorithms that have been trained on some input data to perform some task. They are that complex that we don’t understand how exactly how they make their decisions and we are unable to diagnosis and understand their decision making after the fact.

Can we instead somehow prevent car accidents?

We will see a growing reorganisation of the urban landscape towards making it easier for self-driving cars to navigate our streets. As interestingly enough, jaywalking was made an offence by lobbying of the early car industry.

However it is doubtful that there will no more car accidents or deaths related to self driving cars. Just imagine children suddenly running onto streets. Unless of course cars, similar to underground trains, are placed underground. However it is highly impractical to build tunnels everywhere were people wish to go.

The questions remain: are we fine with AI making decisions on the life and death of humans and are we fine with not being able to reconstruct why the AI made the decision?

I will deal with the latter, the former has already been decided by our continued research into self driving cars.

How to deal with the ‘why’?

To consider this question, we also have to establish who is responsible for an accident. The obvious answer is the manufacturer. But it’s not that simple. Road conditions also play a role. Is the state of the road bad? Was there a stop sign missing or vandalised? There are all sorts of mitigating conditions.

As a comparison I will consider what happens when planes crash or fail.

Planes, Trains and Automobiles

When planes crash killing hundreds of people, the insurance companies have to pay compensation to families and loved ones of the victims. To decide which insurance companies, the fault of the crash needs to be established. So there is a definite interest to get to the bottom of what caused a plane to crash/fail.

Expensive recovery operations to retrieve planes from the bottom of oceans (not to mention finding them first) are launched. Then potentially whole fleets of planes are grounded because manufacturers are found to be at fault.

If it turns out it was pilot error, the airline pays the compensation. If the anti-icing routines of planes at airports were the cause, then the airport pays.

The same thing will happen with self-driving cars. Eventually there will be blackboxes in self-driving cars and entire fleets will be grounded until either hardware or software upgrades can be made.

However how can an AI be debugged? Normally if an AI is making wrong decisions, it’s training data is checked for anomalies and the entire AI is retrained. This an expensive task. In addition, it doesn’t mean the AI won’t make other errors.

Self reflection?

By self reflection I mean a meta-communication system with an AI. A technology similar to human speech amongst humans. Will we be motivated to build a communication system for AIs so that we can explain and discuss complex scenarios with an AI? A kind of diagnostic communication protocol where the AI could self reflect on itself to answer the question “why”.

Having a self reflection mechanism is something that would obviously make diagnosis simpler. However how can we then modify the AI to correct the behaviour? This leads to the need of a self-healing system for an AI. Self-healing since the AI would make the changes itself to itself.

Combining a self-reflecting and self-healing mechanism would mean that an AI could make changes to itself — without our intervention. This is probably the point where humankind reaches singularity and as described by Nick Bostrom, singularity is very hard to control.

Simply put: singularity is the point where technology is as intelligent as us. The danger is that it won’t take the technology very long to be more intelligent than us. Thereby reducing us to second place in the intelligence contest.

So are self-driving cars on the road that leads us to singularity?