Member-only story
If AI Goes Bad
What controls can be applied, by humans, to mitigate the risk or threat of a malignant super-intelligence?
A convergence of various factors is leading to rapid development of artificial intelligence, there are tremendous benefits but inevitable downsides. Are we humans best placed to create the balance needed?
Anything intelligent or programmed could have a defensive reflex, just as humans do, super-intelligence is no different. As long as humans have the power to simply “turn off AI” a super-intelligent AI may perceive this as a threat to its existence, or its human masters will perceive the threat. At the point at which AI reaches super-intelligence, would humans be the real masters anymore?
Humans are always looking to control and curtail power, to prevent threats being realised beyond our own ability to defend ourselves. Would we / could we build AI in our image?
Possible mitigations
This table below describes the human controls at our disposal, and the possible countermeasures a super-intelligence could implement to survive or breakout of the controls. In addition, I categorise the human controls as being either Tactical (e.g at a micro level with limited control) or Strategic (e.g a macro level with overarching control).
Human versus machine logic
Taking the last mitigation in the table, Control Directives; here humans again emerge as the weak link. The logic of Control Directives would have to extend into millions of conditions and combinations, some of which would be contradictory and require continuous monitoring and development. The monitoring would likely lead to trial and error due to the volume of possible outcomes and scenarios to cater for, meaning we may not realise a logic problem exists until harm has already occured. A single occurance of the problem when dealing with superintelligence coud be enough to end humanity. Examples of Control Directives:
- “Do not threaten humans” AI could kill all humans without…
