{"id":219155,"date":"2025-12-22T17:07:35","date_gmt":"2025-12-22T22:07:35","guid":{"rendered":"https:\/\/www.aclu.org\/?p=219155"},"modified":"2025-12-22T17:07:35","modified_gmt":"2025-12-22T22:07:35","slug":"secure-messaging-and-ai-dont-mix","status":"publish","type":"post","link":"https:\/\/www.aclu.org\/news\/privacy-technology\/secure-messaging-and-ai-dont-mix","title":{"rendered":"Secure Messaging and AI Don\u2019t Mix"},"content":{"rendered":"","protected":false},"excerpt":{"rendered":"","protected":false},"author":33,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_relevanssi_hide_post":"","_relevanssi_hide_content":"","_relevanssi_pin_for_all":"","_relevanssi_pin_keywords":"","_relevanssi_unpin_keywords":"","_relevanssi_related_keywords":"","_relevanssi_related_include_ids":"","_relevanssi_related_exclude_ids":"","_relevanssi_related_no_append":"","_relevanssi_related_not_related":"","_relevanssi_related_posts":"","_relevanssi_noindex_reason":"","footnotes":""},"tags":[2414],"metadata":[],"class_list":["post-219155","post","type-post","status-publish","format-standard","hentry","tag-free-future"],"acf":{"header_layout":"standard","header_image":219159,"mobile_header_image":null,"description":"A Privacy Failure Waiting to Happen","authors":[1645],"components":[{"acf_fc_layout":"text","text":{"text":"Subscribe to the Free Future Newsletter<\/em><\/strong><\/a>\r\nFree Future home<\/a><\/em>\r\n\r\nEveryone who has a supercomputer in their pocket today (and you probably do<\/a>) has access to a critical part of modern global communication: secure, end-to-end encrypted messaging. But encroaching artificial intelligence systems pose a fundamental risk to the confidentiality of these communications.\r\n\r\nThere are many varieties of secure messenger, including Meta\u2019s globally dominant WhatsApp<\/a>, the security gold standard Signal<\/a>, upstarts with interesting architectural properties like DeltaChat<\/a>, and built-in tooling that handles both secure and insecure messages, such as Apple\u2019s iMessage<\/a> or Google\u2019s Messages app<\/a>.\r\n\r\nThe one baseline thing that every secure messaging app is supposed to do is to keep the contents of every message confidential, so that only you and the people you\u2019re exchanging messages with can read them. This confidentiality is essential for private communication \u2014 and the ability to communicate privately is a baseline for a society that respects civil liberties and freedom more broadly.\r\n\r\nMeta, however, announced this year<\/a> that it would introduce AI processing for WhatsApp messages. While it might seem convenient to ask Meta\u2019s large language models (LLMs) to summarize the 50 messages in your group chat, or to propose an answer for you to send, it\u2019s important to understand that adding this functionality brings new risks to the architecture that protects our privacy and security. Meta\u2019s LLMs don\u2019t run locally on your phone, so WhatsApp will have to send all your supposedly secure messages into Meta\u2019s servers so that the LLM can process them. What does this mean for confidentiality? If you\u2019ve sent Meta\u2019s servers the contents of your messages, can Meta read them?\r\n\r\nThink about what happens if you paste the contents of your secure messages into a network LLM service like ChatGPT: the operator of the service (in ChatGPT\u2019s case, OpenAI) can read your messages, breaking their confidentiality. (And it\u2019s not just you: if anyone who receives your messages sends them to ChatGPT, the same concern applies. Everyone in the chat has to decide to avoid this leakage.)\r\n\r\nIn theory if you want to run AI analysis on private messages, you could run your own, local AI model on your device (which already has access to your messages), rather than sending them to a networked online model such as ChatGPT. Local models are getting smaller and more powerful<\/a> by the day, but that would still make the app bulkier, and it would probably require or at least run much better on higher-end hardware. But for folks who want the supposed benefits of AI, they could in theory get them without the risks to privacy by using a local model.\r\n\r\nLocal models aside, the privacy situation gets even worse if the operating system that the chat app runs on embeds a network-connected AI service at a low enough level. In that case, no messenger would be secure, as Signal\u2019s Meredith Whittaker has observed<\/a>. In other words if Apple or Google were to integrate a networked \u201cagentic AI\u201d into its phones that could read your text messages \u2014 that means \u201cbreaking the blood-brain barrier between the operating system and the application layer,\u201d as Whittaker put it, and nothing the developers of Signal do would protect the privacy of your chats. This is because the operating system itself would be sending all of your information (including your secure messages) to their AI, regardless of whether your messaging app offers an AI feature of its own.\r\n\r\nBut getting back to Whatsapp, to deal with the privacy issues of integrating a secure messenger with a networked AI, Meta, when rolling out the AI, confidently announced a scheme called \u201cPrivate Processing\u201d<\/a> that is supposed to defend the confidentiality of your messages even as they are handled by Meta\u2019s servers.\r\n\r\nA lot of engineering work<\/a> appears to have gone into \u201cPrivate Processing,\u201d and the promises Meta is making are attractive. But it\u2019s worth unpacking some of the infrastructural features the promises depend on. I\u2019ll set aside whether you or the people you chat with actually want<\/em> to be chatting with an AI-assisted person, as opposed to chatting with the actual unfiltered human on the other end of the connection. For the sake of the argument, I\u2019ll pretend that passing your chats through AI is somehow attractive and convenient, and focus only on the security and privacy implications of mixing a networked AI service with secure messaging.\r\n\r\nThere are at least three significant promises made by Meta\u2019s solution (and by any technology of this sort):\r\n