At Computex 2024, Nvidia and Microsoft announced a groundbreaking partnership to bring AI applications to RTX graphics cards through the development of a new Application Programming Interface (API). This collaboration addresses the previous limitation of running AI applications solely on Neural Processing Units (NPUs). The API empowers developers to harness the superior AI processing capabilities of GPUs, which typically outperform NPUs, even in low-end models. This opens up the possibility of running AI applications on PCs that don’t meet the Copilot+ PC requirements.
The partnership also introduces retrieval-augmented generation (RAG) capabilities to the Copilot runtime. RAG provides the AI model with access to specific information locally, enabling it to generate more relevant and helpful solutions. This feature was showcased in Nvidia’s Chat with RTX earlier this year.
In addition to the API, Nvidia unveiled the RTX AI Toolkit, a suite of tools and SDKs designed for developers. This toolkit empowers developers to optimize AI models for specific applications, resulting in models that are four times faster and three times smaller compared to those created using open-source solutions.
The partnership between Nvidia and Microsoft marks a significant step forward in the development of AI applications. With the availability of powerful hardware and tailored software tools, the future holds promising advancements in AI applications for end users.
