Microsoft has unveiled a groundbreaking new technology for video streaming, dubbed ‘Gaze Based Video Stream Processing,’ that promises to dramatically enhance the quality and stability of video calls. This innovative solution utilizes AI to mimic the natural way humans perceive the world, focusing on details in the center of our vision while blurring peripheral areas.
Think of it like foveated rendering in VR gaming, where the areas you’re looking at are rendered in high detail, while the background is simplified to conserve processing power. Microsoft’s patent takes this concept and applies it to video conferencing platforms like Microsoft Teams, Zoom, and Discord.
The system works by intelligently predicting where a user’s gaze is directed. Based on this prediction, the video stream processor dynamically adjusts the quality of video streams, prioritizing the area being viewed. This results in a higher quality video stream for the user’s focal point while reducing the quality of streams outside their focus.
The immediate benefit? Reduced bandwidth requirements, leading to a more stable and efficient video conferencing experience. Imagine three video streams, each at different resolutions: two at 1080p and one at 720p. With traditional video streaming, all three streams are presented at full quality, consuming a significant amount of bandwidth.
However, with ‘Gaze Based Video Stream Processing,’ the stream the user is focused on remains at a crisp 1080p 60 FPS, while the other two streams drop to a lower 360p 24 FPS. This intelligent bandwidth allocation ensures that resources are allocated where they are needed most, resulting in a more seamless and high-quality video experience.
Powered by AI and a neural network, this groundbreaking technology is poised to revolutionize video conferencing, making it more efficient and enjoyable. With the potential for integration into Copilot+ PCs equipped with NPUs, Microsoft Teams could be the first platform to adopt this game-changing innovation, setting a new standard for video calls in the digital age.