On November 19, 2024, Nvidia CEO Jensen Huang gave an interview. He believes that Nvidia has changed the way of computing, significantly reducing the marginal cost of computing. Modern computing has become a new “AI factory.” In the future, data centers will evolve into places that generate AI content, and the generated “tokens” can be reconstructed into various intelligences.
Huang pointed out that in the future, there will be a large number of agents specific to each SaaS platform. For example, Salesforce, SAP, and Nvidia’s Omniverse will all have unique agents working in synergy with their respective tool ecosystems. These platforms will not be disrupted; instead, they are fertile ground for agent innovation.
For the next ten years, Huang’s goal is to increase performance by 2 to 3 times annually in scale rather than at the chip level, while reducing costs and energy consumption. The new scaling method involves co-design and the full stack, such as converting from different data precisions and promoting compression by treating the network as a computing structure. NVLink is the key to achieving low latency and high throughput. Facing the challenge of extended inference time, a balance needs to be struck between the two to create excellent products.
Customers are concerned about the interchangeability of infrastructure between large-scale training and inference. Nvidia’s advantage lies in the fact that the infrastructure built for training is also excellent in inference. It can perform model distillation, synthetic data generation, etc. There will be models of various sizes, and the software thinking mode remains unchanged.
Nvidia builds systems in the way of building a complete data center. After full-stack optimization, modular components are sold in a decoupled manner to achieve “build software once and run it everywhere.” For example, rapidly building a cluster for X.AI demonstrates the power of cooperation, and the “data center as a product” plan can set up a data center in a short time. Larger-scale superclusters in the future face many challenges, but they are worth trying. The reward for reinventing intelligence is huge.
Huang is very satisfied with the digital AI chip designer. He believes that Hopper cannot be built without AI. Engineers have limited time and cannot explore enough space. Nvidia has redefined computing, reducing the marginal cost of computing and creating a new factory for generating AI. This may be a new industry that every country and company needs.
Huang is very excited about “embodiment.” He believes it is close to artificial general robots and can convert tokens into specific actions. Autonomous driving cars and humanoid robots are ready-made systems. In the future, there will be various AI employees, and companies may have both biological employees and AI employees. He hopes that chip design will be the most intelligent. SaaS platforms will not be disrupted, and a large number of agents specialized in each platform will appear and collaborate to solve problems.
Huang believes that on the basis of science and engineering, AI is changing the underlying work of science and computer science. Every scientific field is adopting new methods. Changes in computing technology stacks and the way of writing software will affect everything. He uses AI to learn and verify knowledge every day. Algorithms are very effective in all fields, which is exciting.
