Summary of Recent AI trends: Suggestions Welcome

Long time lurker on this subreddit. I recently tried to write up a quick summary of major AI breakthroughs and trends for friends/tech workers to read. Below are the trends and major breakthroughs that I’ve come across that seem particularly notable. Feel free to provide discussion, feedback and suggestions of things I’ve missed. (Note I am focusing on LLMs because my friends are already aware of Music, Image and Video Generation)

My Summary:

“Some trends and major breakthroughs:

Scaling laws continue: Parameter counts continue to increase dramatically as access to compute increases with newest models speculating to be in the multiple-10s of trillions of parameters.

https://ourworldindata.org/grapher/artificial-intelligence-parameter-count

XAI, Google, Microsoft/Openai are dramatically investing in AI infrastructure. For instance Microsoft has already announced plans to spend $80billion on AI in 2025 https://www.techrepublic.com/article/microsoft-plans-invest-ai-data-center-2025/

Project stargate was just announced. $100 billion for AI infrastructure going to Open AI this year.

Test time compute is another scaling law that is rapidly advancing. Recent advancements in reasoning models such as o1 or o3 as well as Gemini “Thinking” models have shown dramatic output quality improvements when devoting more compute at time of inference.

Source: https://arxiv.org/abs/2408.03314

Infinite-context Paper: Historically as context length increases the computation cost increases quadratically. Infini-context allows LLMs to have virtually unlimited context window without the dramatic increase in computation.

Source: https://arxiv.org/abs/2404.07143

Google Titans paper: Paper discussing a memory system that allows for long term memory across conversation and materials and huge context windows.

Source: https://arxiv.org/abs/2501.00663

Agents are in development: Anthropic had basic versions of agents, OpenAI said they will release their “operators” in the next few months.

https://youtu.be/ge-rN5tDaC8?si=SrmABjjV07BoCWqy&t=507

Race Conditions:

The United States has put strict export controls on advanced GPUs to China. This has resulted in China not having as much compute, forcing them to be far more efficient with their training. They have come up with far cheaper and more efficient systems that are almost as performative. https://x.com/deepseek_ai/status/1881318130334814301

China is pumping out open source models that are almost on the level of Open AI and Google.

Kai-Fu Lee talking about China making models for $3million that would cost American companies 80-100 million https://youtu.be/od1hIimh7-A?si=PnE18ulNoi2LaThi&t=91

Salesforce said they are no longer hiring any more Software Engineers.

https://www.analyticsinsight.net/tech-news/why-salesforce-has-stopped-hiring-engineers-in-2025

Summary:

World governments and private companies are investing heavily into AI infrastructure. Race conditions between world powers will force them to compete and rapidly innovate. Some of the previous beliefs in lack of memory, reasoning capability, context window length etc. are being tackled and overcome. Industry leaders are all saying the same thing that we will have Agents that are capable of complex tasks in 2025. Advancements may happen faster than society can adapt.”