Microsoft Copilot in Windows 11 gains voice, vision, and the power to act

Microsoft has announced Windows 11 update, which will bring voice, vision, and agentic AI task execution to all Windows 11 PCs, even if they are not Copilot Plus PCs

Microsoft has announced that Windows 11 users, irrespective of whether they have a Copilot Plus PC or not, will be able to turn their PC into an AI PC. The US technology company has introduced voice and visual capabilities alongside tools that allow the AI assistant to perform tasks on behalf of users, even on regular Windows 11 PCs, calling it a step for AI to become an operating system. Here is all you need to know:

Copilot Vision on all Windows 11 PCs

Microsoft said its Copilot Vision is now available in all markets where Windows 11 Copilot is offered. It allows the assistant to analyse what’s displayed on the screen, with options for users to either share their entire desktop or individual applications.

 

Key features include:

• Full desktop and app sharing: Copilot can analyse content, provide insights, answer questions, and coach users through tasks aloud. It can give tips for creative projects, improve resumes, or guide users while navigating games.
• Highlights: Users can ask Copilot “show me how” for a specific task, and it will highlight exactly where to click within the app. This can be used for photo editing, improving lighting, or checking travel itineraries.
• Full app context in Microsoft Office: Microsoft said that when Word, Excel, or PowerPoint files are shared, Copilot can see beyond what’s on the screen. For instance, it can review a full PowerPoint deck without requiring users to flip through slides manually.
• Text-in, Text-out (coming soon for Windows Insiders): Vision will support text-based interaction, allowing users to converse via chat instead of voice. Previously, guidance was only offered aloud.

Copilot Voice on all Windows 11 PCs

Windows 11 users can now activate Copilot using the wake phrase “Hey, Copilot”, previously limited to Copilot Plus PCs. Voice interaction allows the assistant to understand commands, provide guidance, and offer step-by-step instructions or direct actions within apps and files.

 

Key features of Copilot Voice:

• Making the AI assistant work for you: Users can engage with Copilot via voice for dictation, note-taking, transcriptions, searches, and accessibility features like Voice Access and Voice Typing. 
• Wake and goodbye commands: The assistant can be started with “Hey, Copilot” and ended with “Goodbye,” the ‘X’ button, or automatically after inactivity. A chime confirms both the start and end of a session.

Notably, this feature allows voice conversations, making AI interaction easier even for users who are unfamiliar with how to give AI prompts.

Other features being tested

In addition to these interactive features, Microsoft is previewing agentic capabilities through Copilot Labs for Windows Insiders. These allow Copilot to take actions on local files and applications, such as organising documents, extracting information from PDFs, or performing repetitive tasks. Users can monitor the process, intervene at any time, or let the assistant handle tasks in the background.

 

Microsoft is also expanding Copilot’s integration with services like OneDrive, Outlook, Google Drive, Gmail, and Calendar. Users can query connected content using natural language and even export results directly into Word, Excel, or PowerPoint. The company has introduced AI-powered workflows for creating websites from documents, editing videos through File Explorer, and scheduling Zoom meetings without switching apps.

 

Other enhancements include a new taskbar integration for quick access to Copilot, improved local search, and experimental features through Copilot Labs. Most of these capabilities are initially available to Windows Insiders currently, while voice activation and Copilot Vision are rolling out more broadly.