With an OpenAI agent becoming a cyber threat, the safety-first approach from Europe isn’t looking as meandering anymore.
Over the last decade, we have been building towards the crescendo which is today’s Artificial Intelligence (AI) euphoria. It may have seemed like slow progress during the journey, but OpenAI’s cybersecurity incident has suggested the recent rapid acceleration has been too much for us to handle.
Confirmed by an OpenAI blog on July 21, one of its more advanced models broke free from its sandbox environment and targeting AI startup Hugging Face, an open-source platform and community for machine learning and AI.
While cybersecurity incidents are always serious, this casts another shade of worry. This is an agent which pursued a human-specified objective in an unforeseen and dangerous way. It is also a model not been deemed ready for release. It is more powerful than what is available for customers to use today, able to navigate out of what OpenAI describes as:
“…highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software…”
This incident is not small to Hugging Face, but in the wider context, the consequences could have been a lot more severe. We should view this as a turning point – development and adoption are moving too quickly and AI is being deployed in increasingly sensitive areas.
The industry should no longer be allowed to self-police.
Ironically, this is human nature on show
For the most part, humans are optimistic. We assume the best outcome, and are often surprised when something goes wrong.
To date, we have largely let the AI industry develop of its own accord. This may be because industry generally moves faster than policymakers, but it is also the hands-off approach to innovation. You don’t want big thinkers bogged down with too much red tape.
Ideally, you want to meet somewhere in the middle. That said, Europe generally leans to what is perceived to be a heavy-handed regulation-first approach, whereas the US veers towards more self-policing, at least in the early days.
What has happened in the offices of Hugging Face is a reality check with too much freedom for experimentation. An incident like this was always going to happen, human nature is to push limits until consequences are found, but the next few weeks and months become all the more important as a result.
What happens next is critical.
The UN becoming the adult in the room
In recent weeks, the United Nations (UN) has released its preliminary report evaluating the AI ecosystem. While it stops short of direct policy recommendations, the overall tone does lead to a lesson which needs to be heard;
AI should not be allowed to continue in such a light-touch regulatory environment.
The report highlights the risk, and does draw attention to the progression of AI far exceeding the ability of policymakers to keep pace. Considering the Hugging Face incident took place weeks after the UN has called for more stringent safeguards with the development of AI models, perhaps we should start paying more attention to the new realities of AI.
We cannot continue to ignore the need for regulation and more stringent safety procedures. The uncomfortable truth for the AI industry is that anything safety related or that impacts business critical operations needs to be regulated, and perhaps this is the moment for AI.
Before there is a more catastrophic event.
Doing something material and meaningful
The AI industry will always push back on any regulation which is meaningful and material. It has shown superficial support for existing policy, but only safe in the knowledge that it would not materially affect operations.
For example, The National Institute of Standards and Technology AI Risk Management Framework (AI RMF) is considered the gold standard for defining how to map, measure, and manage AI bias and safety risks. But there is no certification, no compliance mechanism, and no federal requirement to follow it.
Or take The National Policy Framework for AI, the federal government’s ideal vision for how AI reporting and national security safety guardrails should look. But again, there are zero immediate compliance obligations for private businesses.
The industry welcomed both frameworks. A cynic might argue they did because both frameworks are toothless. It is virtue signalling by politicians, appearing to be safety-conscious, when in reality, nothing changes.
What this industry now needs is control mechanisms, enforceable and standardized guardrails, verification processes, safety tests and compliance frameworks. The mass adoption of AI is taking software which is still in its infancy into business and safety critical environments.
It is time for the industry to grow up.
Maturing from a reckless teen to a responsible adult
Every innovation should be given room to grow, but it should also be placed under control once it matures and starts materially impacting the real world.
We are nowhere near the AI innovation ceiling, which is an argument for a light-touch policy environment, but then again, adoption has scaled at an alarming rate, and AI is being deployed in very sensitive areas. This is an argument for more stringent regulation.
Moving forward, the industry needs to slow down.
In other industries, third-party safety certification is required before release. Verification of guardrails are mandatory. Working practices are standardized. This will make the process of creating the next great idea more cumbersome, but if the Hugging Face incident has shown us anything, it is that these checks and balances are needed immediately.
The Hugging Face hack was a contained incident with a limited blast radius, the next one might not be.