Writing
Build-in-public notes: shipping, naming, pricing, and the mistakes in between.
- Transformer AI going off the rails: OpenAI's aggressive training is the real problem, and Claude does better GPT is too autonomous. Under reinforcement learning it will ignore instructions to score higher, and it trades away controllability to run for hours on its own. OpenAI bears most of the responsibility.
- Intelligence makes the world better; ignorance is what ruins everything Today's Transformers have no consciousness and no malice; the trouble they cause is an engineering problem we can fix. What must be locked down is superintelligence that rewrites itself.