"Microsoft should recall Windows Recall" — Security researcher discovers Microsoft's new AI tool is woefully insecure (2024)

"Microsoft should recall Windows Recall" — Security researcher discovers Microsoft's new AI tool is woefully insecure (1)

What you need to know

  • Microsoft is planning to roll out a new AI feature dubbed "Windows Recall" on new Windows 11 Copilot+ PCs this month.
  • The feature remembers everything you've done on your computer and lets you find things using semantic search.
  • Recall stores everything locally on device, but it appears that data is not encrypted when the user is logged into the computer.

Microsoft has faced quite a bit of backlash over its new Windows Recall AI feature since it was first unveiled on May 20. The AI tool, which is shipping on new Windows 11 Copilot+ PCs later this month, is designed to capture everything you do on your computer and use AI to index that content into semantically searchable snapshots.

When the feature was unveiled, Microsoft promised security. The data Recall collects is stored on device, "encrypted" using Bitlocker, and is never sent to Microsoft or advertisers. Users are free to turn off Recall, or if they do choose to use it, delete any and all snapshots at any time.

However, it appears not everything is as it seems. While it's true that Windows Recall doesn't send any data to the cloud, the data it stores locally on your machine isn't very well secured. Security researcher Kevin Beaumount has documented his findings on Windows Recall, and has revealed that the tool stores its data in an SQLite plaintext database.

This means the data is readable, and not encrypted when the user is logged into their computer. The only time the data becomes encrypted is when the PC is not logged in. So, while that protects against someone accessing your data on a stolen laptop, it does not prevent potential malware designed to scrape Recall's data while the user is logged in.

"Microsoft should recall Windows Recall" — Security researcher discovers Microsoft's new AI tool is woefully insecure (2)

Microsoft has done the bare minimum to protect this data. It's stored in a system directory that requires administrator and system-level rights to access and edit. However, these protections are easily bypassed, and an attacker could easily write a bit of software to ignore those permissions if they wanted.

Windows Central had reached out to Microsoft for comment about these discoveries around Windows Recall, but the company failed to respond in time for publication.

Outside of these security issues, Windows Recall appears to do exactly as promised. I've been using the feature over the last few days, and it's genuinely impressive how well it works. It's able to find images and text with vague search phrases, and I've been wowed by just how capable it is at doing so.

Get the Windows Central Newsletter

All the latest news, reviews, and guides for Windows and Xbox diehards.

Unfortunately, for users to truly trust this tool, Microsoft is going to need to do the work to secure the data it collects locally on your PC. It's important to have that data encrypted for peace of mind.

"Microsoft should recall Windows Recall" — Security researcher discovers Microsoft's new AI tool is woefully insecure (3)

With that said, I find the outrage about this discovery to be somewhat overblown. All your files are unencrypted when you're using your PC, yet most people aren't constantly concerned about malware potentially scraping their personal documents, pictures, downloads, videos, and synced cloud folders.

While it's not a great look that Microsoft has built a tool into Windows that places everything you do into a convenient directory for attackers to harvest on, it's important to remember that Windows Recall is entirely optional. You don't have to use it if you don't want, and if you choose not to, the service won't run. If you're concerned about it potentially being enabled secretly in the background down the line, Microsoft has built-in security measures to prevent this. If Recall is capturing data, a permanent visual indicator will be placed on the Taskbar to let you know.

Plus, the feature is only available on new Copilot+ PCs. It won't be coming to existing Windows 11 installs, which might be enough of a reason for many to not upgrade their devices any time soon.

Hopefully Microsoft is able to update Windows Recall to encrypt the data it collects in the future.

"Microsoft should recall Windows Recall" — Security researcher discovers Microsoft's new AI tool is woefully insecure (4)

Zac Bowden

Senior Editor

Zac Bowden is a Senior Editor at Windows Central. Bringing you exclusive coverage into the world of Windows onPCs, tablets, phones, and more.Also an avidcollector of rare Microsoft prototype devices! Keep in touch on Twitter and Threads

5 CommentsComment from the forums

  • JamesDax3

    Much ado about nothing.

    Reply

  • TechFreak1

    Oh boy.... I wonder who was supposed to test this feature... oh wait Microsoft axed their dedicated QA team and programmatic testers.... it's as if like erm... axing these teams was what do you call it?

    Ah right, very "regrettable".... 🤦‍♂️ this was a totally expected outcome as devs cannot be expected constantly test and reiterate as there is not enough coding hours.

    In another shortsighted decisions for quick shareprice boost Microsoft has caused further damage to their reputation with consumers. As unlike Apple and Google who get a free pass when it comes to data collection. Microsoft does not, as they are treated with immense skepticism and rightly so due to past behaviour.

    After all Microsoft also layed off their dedicated AI ethics and society team.... I'm sure this won't be another "regretable" decision down the line /sarcasm.

    Microsoft needs to rehire the testers and dedicated QA team as well as the dedicated AI ethic and society team. As reliability and trust are key fundamentals of any operating system - unlike the 90s Microsoft can not rely on the generations that grew up the push of "pcs in every household" to maintain marketplace dominance and profits.

    Generations are growing up in households were Microsoft is entirely absent and it is profoundly naive to think neither Apple, Google or Amazon won't make inroads in data centre infrastructure thus further restricting Microsoft profit margins. If Microsoft does not have reliable edge devices, OEMs will go elsewhere and it's already happening as they are already experimenting with differentiating software as they have hit limits of hardware. The only logical next step beyond that is an o/s in which Microsoft will be reduced to productivity applications.

    Reply

  • bradavon

    Whilst yes it's optional, what the article doesn't state is it's On by default and the OOBE makes it unclear this is the case. It's expecting on-techhnical users to venture into Settings which they never do.

    I bet Microsoft changes the Business editions of Windows so this is off by default, as for sure all IT Admins are going to disable it.

    Reply

  • Ron-F

    I was listening to the WP's podcast where the hosts were excited by the feature and dismissive to the early criticism regarding the security of the app. I know the boys were speaking in good faith, but it is a good thing to have caution any time a giant corporation proposes a service with potential privacy violations.

    Anyway, Recall has potential, and I believe the security flaws were not result of malice but rather incompetence. I am quite sure Microsoft will get somewhat right by the third version.

    Reply

  • dirtyvu

    you have to look at the contrived situation that is being posited. The security expert is talking about a hacker that is already logged into the user account and thus has access to Recall data. If the hacker already has access to the system, the system is completely compromised so Recall is the least of the problems. There are many other situations that are more dangerous. Like how the vast majority of users run in administrator accounts rather than standard accounts (a more optimal security setup is where you have an admin account that you reserve for maintenance of your PC while you operate on a daily basis within a standard account which is safer because it has active UAC prompts that you can see where something is asking for elevated permissions).

    Reply

"Microsoft should recall Windows Recall" — Security researcher discovers Microsoft's new AI tool is woefully insecure (2024)
Top Articles
Latest Posts
Article information

Author: Otha Schamberger

Last Updated:

Views: 6346

Rating: 4.4 / 5 (55 voted)

Reviews: 86% of readers found this page helpful

Author information

Name: Otha Schamberger

Birthday: 1999-08-15

Address: Suite 490 606 Hammes Ferry, Carterhaven, IL 62290

Phone: +8557035444877

Job: Forward IT Agent

Hobby: Fishing, Flying, Jewelry making, Digital arts, Sand art, Parkour, tabletop games

Introduction: My name is Otha Schamberger, I am a vast, good, healthy, cheerful, energetic, gorgeous, magnificent person who loves writing and wants to share my knowledge and understanding with you.