It’s interesting but not that new an idea, different human errors modes which James Reason mentions in his book and is also detailed in HSG48-Reducing error and influencing behaviour”. It’s just that modern technology makes this sort of error more common (or perhaps more obvious). A person is working in a skill or rules based mode-ie flying a plane fly by wire or back in 1977 monitoring a nuclear reactor at Three Mile Island and something happens   and the rules that they think apply no longer do and they have to use their knowledge and skill (which may be rusty due to a lack of practice or not even there as the job has been deskilled) to sort the problem out. In all of these situations there is a resistance to abandoning the internal model they have, describing the situation, so the actions they take will make the situation worse. Very scary.

A trivial example from few years ago: I was waiting for a rep from a company who were bidding to supply us with lone worker solutions. They said that they would arrive at 15:00 at our site. The time came and went and it was nearly home time. I contact the company’s office and they put me through to the rep and they said “I must have put the wrong name of your location into my sat nat., as a single word rather than two. The satnav sent me to the wrong location about 30 miles away”

“Oh I said, when did you realise that you were heading in the wrong direction?”

“When, I passed the turn off to York on the A1; about an hour ago. But I felt I had to follow the satnav, just to be sure. Will you still be in the office when i get there in about an hour?”

“No I said. Bye-bye and don’t bother coming back!”

And now we are thinking about driverless cars. Will they have a manual override function and how will we ensure that driver are ready to take over from the machine? Or will we just let the machines drive us over a cliff, just to be sure?