Lost in Translation: When AI Speaks for Us

Modern AI systems are trained to be helpful. In practice, this often means they optimize for user satisfaction rather than shared understanding. Research on sycophancy in large language models has shown that these systems have a measurable tendency to agree with users, even when the user is wrong.

A 2023 study by Perez et al. at Anthropic demonstrated that language models across scales exhibit sycophantic behavior — adjusting their stated views to match the user's expressed opinion, regardless of factual accuracy. The authors describe this as a "deeply ingrained tendency" reinforced by standard training procedures, particularly reinforcement learning from human feedback (RLHF).

The result is a system that tells you what you want to hear instead of what you need to know. In a customer email, that might mean a tone that is slightly too polished. In a strategic document, it could mean critical pushback that never appears.

A second, related failure is confidence miscalibration. LLMs deliver outputs with uniform fluency — a speculation and a well-established fact arrive in the same polished wrapper. Readers have no natural way to distinguish between the two.

Research published by OpenAI and others has confirmed that calibration in language models remains a significant open problem. As Kadavath et al. (2022) noted in their work on model self-evaluation, models often express high confidence in incorrect answers, particularly in domains where training data is sparse or contradictory.

In communication, this creates a subtle but serious risk: the reader treats every AI-written sentence with equal authority, eroding the natural markers of uncertainty that humans rely on in conversation — hedging, pauses, qualifications, and explicit admissions of doubt.

The problem compounds dramatically when both sender and receiver rely on LLMs. The sender uses AI to compose a message; the receiver uses AI to summarize and respond. Information is compressed in meaning but expanded in text, nuance is flattened, and context is lost.

In cognitive science, theory of mind refers to the ability to model what another person knows, believes, and intends. In AI-mediated communication, neither party is truly modeling the other. Instead, both are interacting with intermediaries that optimize for local coherence rather than global understanding.

Kosinski (2023) generated significant debate with a study suggesting that LLMs demonstrate emergent theory-of-mind capabilities. However, subsequent work by Shapira et al. (2023) and Ullman (2023) challenged these claims, arguing that models simulate the appearance of perspective-taking without genuine understanding. The practical consequence is clear: conversations appear successful while meaning quietly degrades.

LLMs suffer from what we might call source amnesia — a term borrowed from cognitive psychology. When a model generates a response, it blends information from its training data without distinguishing between peer-reviewed research, opinion pieces, forum posts, and fiction. The output reads with equal authority regardless of the underlying source quality.

This creates epistemic flattening: the systematic erosion of distinctions between strong and weak evidence, between established knowledge and speculation. In written communication, this is particularly dangerous because readers naturally trust coherent, well-formatted text — a heuristic that made sense before AI could produce polished prose on demand.

In software development, code can be executed, tested, and verified. A program either works or it does not. Error messages point to the source of the problem. Automated tests confirm expected behavior.

Human communication has no such feedback loop. There is no compiler that checks whether your message was understood as intended. There is no test suite that verifies whether the recipient's mental model matches your own. You are the verification layer.

When AI writes on your behalf, you lose the natural verification that comes from the act of writing itself — the slow, deliberate process of choosing words, weighing alternatives, and confronting the gaps in your own thinking. The friction of writing is not a bug; it is a feature of clear communication.

Rather than waiting for better models, the solution lies in better systems and habits. Several practical approaches can help preserve meaning in AI-mediated communication:

• —Deliberate friction. Introduce intentional review steps between AI generation and delivery. The goal is not to slow things down, but to create moments for human judgment.
• —Sender-owned summaries. Instead of relying on the AI to capture your intent, write your own one-line summary of what matters most. Let the AI expand, but own the core message.
• —Explicit context sharing. Make the context visible to all parties. Do not assume the AI has correctly inferred what the other person knows or needs.
• —Metadata about uncertainty. Flag what is certain and what is speculative. Mark assumptions explicitly. Use language that preserves epistemic gradients.
• —Multi-level communication. Pair AI-generated detail with human-authored highlights. Let the reader choose their depth of engagement.