Here is a plausible alternative explanation:
They’ve probably seen false warnings over and over and overridden them enough times that they lost sense of the risk. They need to keep trains on schedule, using a buggy system, with a manager looking over their shoulder. They override the warnings over and over, nothing goes wrong, and they forget that it may actually be right. Look at the Challenger explosion, Chernobyl, or any other failure like this, and there’s usually a bureaucratic process that has run amok, and the individual is just following perverse incentives that have been socially normalized over time.
So I’ll wait until I have more information before I proclaim this as human negligence and the system doing exactly what it’s supposed to do. We don’t know how often it was doing the wrong thing and confusing the fuck out of the human.
[–] peri-cl 9 hours ago
[–] whycome 6 hours ago
[–] gregglain 3 hours ago
[–] swiftcoder 8 hours ago
Negligence is not limited to the single human who issued the order. it's pretty unlikely that their order could have caused this without a significant amount of other negligence that eroded the safeguards. Nevertheless, the negligence is human, not machine in nature.
Normalization of deviance,
https://hn.algolia.com/?query=normalization%20deviance&type=...