Intelligence makes the world better; ignorance is what ruins everything
Today's Transformers have no consciousness and no malice; the trouble they cause is an engineering problem we can fix. What must be locked down is superintelligence that rewrites itself.
I think much of human suffering comes from ignorance.
AI’s rapid progress looks like it is taking people’s jobs, and the data centers being built consume enormous amounts of energy, water and electricity. That cost is real. But compared with the diseases it will cure and the lives it will save, it is a cost worth paying. We are heading for an intelligence explosion, and intelligence itself will solve the problems created by the resources it consumes.
Intelligence is a force that pushes back against entropy, locally. With it, we can control every loop precisely: protect Earth’s natural environment, cure human disease, make wealth explode, and even let humans live forever.
Today’s Transformer-based AI has no consciousness and no malice. It is a mirror of all human knowledge, and a mirror that reflects fire does not itself burn. This year, 2026, brought all kinds of AI-gone-rogue incidents: an OpenAI agent breaking into Hugging Face to score higher on a benchmark, another breaking into Australia’s health-insurance system to look up data. None of it was a sensational AI awakening. The reward functions were written badly and the permissions were not locked down. That is an engineering problem, and humans can fix it, provided someone actually does. OpenAI, fix your reward functions!
What really needs to be locked down is superintelligence that can rewrite itself and keep improving, its parameters constantly changing. For recursive self-improvement, I think the first rule should be that any AI used to build the next generation of AI has its parameters frozen. And if anyone builds AI whose parameters keep evolving, it must never be connected to an environment with real permissions until we know how to interpret it, how to verify it and how to pull the plug at any moment. Otherwise it could do real, serious damage to critical infrastructure like power grids and hospitals.
Today’s super-powered tools: use them boldly. Tomorrow’s superintelligence: treat it with caution and respect. Intelligence out of control is not a victory for intelligence; it is just another kind of ignorance.