CanaryBench: Stress Testing Privacy Leakage in Cluster-Level Conversation Summaries
arXiv:2601.18834v1 Announce Type: new Abstract: Aggregate analytics over conversational data are increasingly used for safety monitoring, governance, and product analysis in large language model systems. A common practice is to embed conversations, cluster them, and publish short textual summaries describing each cluster. While raw conversations may never be exposed, these derived summaries can still pose privacy risks if they contain personally identifying information (PII) or uniquely traceable strings copied from individual conversations. We introduce CanaryBench, a simple and […]