2030: Have They Social Engineered Your AI?!

It is with much excitement that I am reporting that our maverick research paper has been published. Please welcome “Maverick* Research: Your Smart Machine Has Been Conned! Now What?”

Led by Patrick Hevesi (@PatrickHevesi ) and together with Erik Heidt (@CyberHeidt), we have explored the much-discussed concept of AI apocalypse. In the abstract we say “Smart machines [defined here, BTW] and AI pose huge future risks that derive from malicious humans using or abusing them to achieve their goals. Here, we focus on identifying and reducing those risks. (Maverick research deliberately exposes unconventional thinking and may not agree with Gartner’s official positions.)” Our original idea was that much of this bleating about “coming AI horrors” misses the point: the risks are indeed there, but they are NOT – for the foreseeable future – about the machine rebellion, but about the machine exploitation by people. Thus, we have built a smart machine attack taxonomy and outlined new defense approaches. We also have 3 fun scenarios of smart machines being deceived, from “almost real” to remote futuristic.

Some fun quotes follow below:

• “The entire history of security teaches us that controls will fail; thus, AI and smart machines need extensive monitoring, anomaly detection and auditing controls to expose logic tampering.”
• “Smart machines can be enormously complex and opaque [and non-deterministic], making it difficult or impossible to determine why they failed.”
• “Focus on the risks likely to emerge between now and 2035 (smart machines being duped or otherwise misused by humans) instead of on longer-term existential fantasies and fears.”
• “Scientifically speaking, we still don’t know what morality in humans actually is — and so creating a [full] digital morality in software is essentially impossible. ”
• “Morality-based attacks [further in the future] will try to deceive the machine’s ethical programming into believing what was “wrong” is now “right.” Therefore, machine morality must be protected.”
• “At this time, questions of AI safety and security engineering, resilience, and other risk considerations are barely understood. As the field develops, these questions will become critical in the design of future AI and smart machines.”

Enjoy the paper here – with any Gartner license. We had a lot of fun – and some frustrations! – creating it, so have fun reading it!

Possibly related posts:

• On Evil AIs and Evil People
• Killed by AI Much? A Rise of Non-deterministic Security!