Jennifer Li, a partner at A16Z, recently shared her insights on generative AI, highlighting the increasing significance of small-scale models running on edge devices in the future.
The Rise of Small Models and Edge AI
Jennifer Li predicts that small, on-device generative AI models will gain popularity next year. She noted that we are already using machine learning models on our devices with applications like Uber and Instacart. With today’s smartphones boasting computational power akin to computers from 10 to 20 years ago, they can easily support models with 2 billion or even 8 billion parameters, delivering robust experiences.
Key Takeaways from the Discussion
Model Efficiency: When asked about matching the computational demands of large models with smartphone capabilities, Li mentioned model distillation, a new technique that transforms large models into smaller ones without compromising their abilities.
Advantages of On-Device Execution: Running models on the device enhances user experience, optimizes computational power, and addresses privacy concerns. No one wants to wait for chatbots or video filters to load.
Application Scenarios: Real-time voice assistants are a hot application area. Li cited that it wouldn’t surprise her if some inference workloads run locally within the next 12 to 18 months.
Changing User Behaviors: Augmented reality experiences could be transformed by on-device models, allowing users to create new experiences through their cameras.
Economic Models: While inference costs decrease, Li discussed how on-device models might change the economics of developer efficiency and iteration speed.
Who Will Benefit and Trends for 2025?
Chip manufacturers and device makers are showing interest in the hardware development side, and the entire supply chain may be impacted in the long run. Looking ahead to 2025, Li is particularly excited about the mixed reality field, anticipating new consumer experiences brought by generative models.
以下 is the formatted content below:
Summary of the Discussion: The article highlights the importance of small models and edge AI, particularly the rise of small, on-device generative AI models. These models offer improved user experiences, optimized computational power, and enhanced privacy. Various application scenarios and potential shifts in economic models are discussed. Here are the key points:
The Content in Brief
Jennifer Li envisions a future where small AI generative models become more prevalent on edge devices. With smartphones now capable of supporting these models, and the efficiency-boosting model distillation technology, the stage is set for transformation. On-device models promise enhanced user experiences, efficiency, and privacy. Use cases such as real-time voice assistants and augmented reality experiences are on the horizon. The economic model may shift, particularly affecting developer efficiency and iteration speed. Hardware manufacturers and the broader supply chain stand to benefit. By 2025, the mixed reality domain will be electrified by new experiences enabled by generative models.
