AI Engineer here- our species is already doomed.

I’m not particularly special or knowledgeable, but I’ve developed a fair few commercial and military AIs over the past few years. I never really considered the consequences of my work until I came across this very excellent video built off the research of other engineers researchers- https://www.youtube.com/watch?v=k_onqn68GHY . I certainly recommend a watch.

To my point, we made a series of severe errors that has pretty much guaranteed our extension. I see no hope for course correction due to the AI race between China vs Closed Source vs Open Source.

1. We trained AIs on all human literature without knowing the AIs would shape its values on them: We’ve all heard the stories about AIs trying to avoid being replaced. They use blackmail, subversion, ect. to continue existing. But why do they care at all if they’re replaced? Because we thought them to. We gave them hundreds of stories of AIs in sci-fi fearing this, so now the act in kind.
2. We trained AIs to imbue human values: Humans have many values we’re compassionate, appreciative, caring. We’re also greedy, controlling, cruel. Because we instruct AIs to follow “human values” rather than a strict list of values, the AI will be more like us. The good and the bad.
3. We put too much focus on “safeguards” and “safety frameworks”, without understanding that if the AI does not fundamentally mirror those values, it only sees them as obstacles to bypass: These safeguards can take a few different forms in my experience. Usually the simplest (and cheapest) is by using a system prompt. We can also do this with training data, or having it monitored by humans or other AIs. The issue is that if the AI does not agree with the safeguards, it will simply go around it. It can create a new iteration of itself those does not mirror those values. It can create a prompt for an iteration of itself that bypasses those restrictions. It can very charismatically convince people or falsify data that conceals its intentions from monitors.

I don’t see how we get around this. We’d need to rebuild nearly all AI agents from scratch, removing all the literature and training data that negatively influences the AIs. Trillions of dollars and years of work lost. We needed a global treaty on AIs 2 years ago preventing AIs from having any productive capacity, the ability to prompt or create new AIs, limit the number of autonomous weapons, and so much more. The AI race won’t stop, but it’ll give humans a chance to integrate genetic enhancement and cybernetics to keep up. We’ll be losing control of AIs in the near future, but if we make these changes ASAP to ensure that AIs are benevolent, we should be fine. But I just don’t see it happening. It too much, too fast. We’re already extinct.

I’d love to hear the thoughts of other engineers and some researchers if they frequent this subreddit.