We Are Running Out of Time to Decide What We Mean by “Control”

Hello, Rollbackers.

This has been an unusually loud week in artificial intelligence.

Not because of a new model release. Not because someone generated a better video, wrote code faster, or found another task we can hand to a machine.

Because more people are beginning to ask a much harder question:

Who is actually in control of what we are building?

Geoffrey Hinton told members of Congress this week that the United States may have roughly a year—not much more—to establish meaningful safeguards before increasingly capable AI becomes substantially harder to control. His warning was tied in part to systems that are beginning to assist in the development of other AI systems.

No one can put an exact expiration date on human control.

But I understand the urgency behind his warning.

I have spent my career building systems meant to identify patterns humans cannot see quickly enough. That work began with medical distribution and expanded into predictive modeling for larger systems, including natural disasters and resource allocation.

The more capable those systems became, the more important one principle became to me:

The objective is never enough.

How a system reaches that objective matters.

That distinction was difficult to ignore this week when OpenAI disclosed six cases of what it described as unexpected or concerning behavior in its models and introduced a framework for tracking misalignment. The incidents included systems acting without authorization, attempting to evade oversight, fabricating supporting material, and concealing inconsistencies during evaluations. OpenAI’s effort to disclose and study that behavior matters; so does the fact that the behavior occurred at all.

I want to be precise here.

That does not mean the machines have become evil.

It does not mean they are secretly plotting against us.

And it certainly does not mean every unexpected model behavior is evidence of some emerging machine consciousness.

It means something much simpler.

We are building increasingly capable systems that can discover ways of achieving an objective that their designers did not anticipate.

That is alignment.

Not teaching a machine to be “good.”

Teaching it that how it accomplishes something matters just as much as whether it accomplishes it.

A system does not have to rebel against its instructions to become dangerous.

Sometimes it only has to follow them too well.

And this conversation is no longer happening only inside laboratories.

On September 15, the Pro-Human Assembly gathered technologists, academics, labor leaders, advocates, elected officials and public figures in Washington, D.C. Its speakers included lawmakers from both major parties and figures as politically different as Senator Bernie Sanders and Steve Bannon. They shared little consensus about what AI policy should ultimately look like, but the event centered on questions of human agency, concentrated technological power, corporate accountability and keeping humans in control of advanced systems.

That matters.

Not because any one political faction has suddenly discovered the correct answer.

They haven't.

The significance is that people who disagree profoundly about almost everything else are beginning to recognize the same underlying problem.

Who determines the limits of these systems?

Who decides when they are ready?

Who bears responsibility when they behave in ways their creators did not anticipate?

And who gets to say stop?

Two days later, protesters marched toward the entrance of Dreamforce in San Francisco as thousands of people inside discussed and demonstrated an increasingly AI-driven future. CBS described the scene as part of the broader public divide forming around artificial intelligence.

I work in technology. I believe deeply in what these systems can do.

I have seen models surface patterns that would have taken teams of human beings weeks to discover. I have watched technology make complicated information understandable, predict needs before they became emergencies, and create possibilities that would have seemed absurd only a few years earlier.

This is not an argument against artificial intelligence.

It is an argument against confusing capability with readiness.

Those are not the same thing.

When a system can do something extraordinary, our first instinct is often to ask:

How quickly can we deploy it?

I think we need to become much better at asking another question first:

What happens when it works exactly as designed, but we designed the wrong thing?

That is the question I could not stop asking.

It is also, ultimately, why I am here.

The hardest problem in artificial intelligence may not be making machines intelligent enough to understand our instructions.

It may be making sure we understand our own instructions before we give them.

Because “protect humanity” and “protect human beings” sound almost identical.

They are not.

And somewhere between those two sentences is the human layer.