Copilot Vision is a new AI assistant that also sees everything on your Windows 11 display

Microsoft's new opt-in Copilot Vision might sound like Recall 2.0 but it's a full AI agent and voice-controlled assistant designed for Windows 11.

Copilot Vision is a new AI assistant that also sees everything on your Windows 11 display
Comment IconFacebook IconX IconReddit Icon
Senior Editor
Published
1 minute & 45 seconds read time
TL;DR: Microsoft's Copilot Vision enhances Windows 11 with AI-powered screen analysis, offering real-time guidance, document review, and app tutorials via voice or text commands. This opt-in feature integrates deeply with Office apps and improves productivity, navigation, and gaming, marking a significant step toward AI-driven PC experiences.

With Siri, Google, Alexa, and other voice-activated digital assistants, it's not exactly surprising to hear that Microsoft's Copilot is adding voice controls. However, Copilot Vision is something different, in that it's an AI-powered agent that can see what it is you're doing in Windows 11 and offer advice, assistance, guidance, tips, and more.

Microsoft believes that Copilot Vision is the next step in making every Windows 11 PC an AI PC. All you need to do is say "Hey Copilot" to engage with the AI agent and then use natural language to do everything from answer questions about something on-screen, analyzing documents to find something specific, or provide tips while you're playing a game.

In addition to being able to see what's on your screen, Copilot Vision will also have "full app context" for Word, Excel, and PowerPoint, so it can review an entire document or deck without being limited to just what's on screen. Copilot Vision is a new opt-in feature, and surprisingly, it's now available "in all markets where Copilot is offered."

Assuming it all works as intended, Copilot Vision does sound like a powerful PC tool. One of the more interesting features is its ability to teach you and guide you through using various apps and software tools by simply asking Copilot, "Show me how." An example could be after opening a photo in an image editor, you could ask Copilot to show you how to change the lighting or change the size.

Even though Copilot Vision is opt-in and Microsoft is adamant that it's backed by robust security and privacy, it is definitely starting to look like the future of Windows will include AI agents and Copilot as a key part of both navigation and productivity. In addition to Copilot Vision's launch, Microsoft is replacing the search bar on the taskbar with an Ask Copilot bar that can be interacted with via voice or text. Microsoft says that this new Copilot-powered search bar makes "finding apps, files, and settings faster and easier, with results appearing and updating instantly as you type."

And then there's Gaming Copilot (Beta) for the ROG Xbox Ally handheld and Copilot Actions for Windows Insiders, which completes actions on your PC like extracting files, finding documents, or even creating one - with direct access to local files on your Windows device.