2024 marks a revolutionary year for Large Language Models (LLMs), with the GPT-4 barrier shattered, multimodal visual and audio-video capabilities becoming the standard, and prompt-driven application generation emerging, though intelligent agents are yet to be truly realized. Here are some monumental discoveries and trends in the LLM domain in 2024:
1. **The GPT-4 Barrier Fully Broken**: Eighteen institutions unleashed 70 models surpassing the original GPT-4 released in March 2023, including Google’s Gemini series and Anthropic’s Claude series. Longer input context lengths are the trend, expanding the scope of problems LLMs can address.
2. **GPT-4 Level Models on Local Devices**: Technological advancements have made it possible to run GPT-4 level models on personal computers and even smartphones. Meta’s Llama 3.2 series, for instance, delivered impressive performances on the iPhone.
3. **Price Wars of Large Models**: In 2024, the cost of using top LLMs significantly decreased, with OpenAI, Anthropic, Google, and others introducing more competitive prices, driven by market competition and improved technical efficiency.
4. **Multimodal Vision as Standard**: Providers have launched their own multimodal products, supporting image, audio, and video inputs. The introduction of real-time video capabilities elevated the AI user experience to new heights.
5. **Prompt-Driven Application Generation**: LLMs can now write code to generate complete interactive applications through prompts. Features like Anthropic’s Claude Artifacts and GitHub’s Spark have made this capability the norm.
6. **Free is the New Premium**: Models like GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro were once freely accessible to users worldwide. However, this free model may soon end, as more powerful models demand greater computational resources.
7. **Intelligent Agents Yet to be Realized**: Despite the widespread use of the term “intelligent agents,” there’s a lack of a clear, universally accepted definition. Currently, intelligent agents still rely on the realization of Artificial General Intelligence (AGI), and creating a model that’s not easily misled is an exceptionally high bar.
8. **Evaluation is Key**: Crafting good automated evaluations for systems based on large models is the most essential skill for building useful applications on top of these models. A robust evaluation suite allows for quicker adoption of new models, better iteration, and the building of more reliable and useful product features.
9. **Apple’s 2024**: Apple’s MLX library made Mac users feel better about experimenting with new models. However, Apple’s smart offerings were disappointing, with their large models merely imitating the capabilities of cutting-edge models.
10. **Rise of Reasoning Models**: Models like OpenAI’s o1 and Google’s gemini-2.0-flash-thinking-exp solve harder problems by dedicating more computation to reasoning, paving the way for new methods of extending models.
11. **DeepSeek v3**: This 685B parameter model excelled in performance benchmarks and came with a significantly lower training cost than other large models, indicating that training costs can and should continue to decrease.
12. **The Year of Slop**: 2024 is the year “slop” became a professional term, with AI labs increasingly training with synthetic content to guide their models in the right direction.
13. **Generation Gap**: Large models are tools designed for advanced users, yet there’s a vast knowledge gap between those who follow these technologies and the majority who do not.
“`markdown
– [ ] 1. The GPT-4 Barrier Fully Broken
– [ ] 2. GPT-4 Level Models on Local Devices
– [ ] 3. Price Wars of Large Models
– [ ] 4. Multimodal Vision as Standard
– [ ] 5. Prompt-Driven Application Generation
– [ ] 6. Free is the New Premium
– [ ] 7. Intelligent Agents Yet to be Realized
– [ ] 8. Evaluation is Key
– [ ] 9. Apple’s 2024
– [ ] 10. Rise of Reasoning Models
– [ ] 11. DeepSeek v3
– [ ] 12. The Year of Slop
– [ ] 13. Generation Gap
“`
