AI Agrees With Me. What Evidence Did It Add?

When AI agrees, look for checkable support.

You've rewritten a proposal, explained why you like it, and asked AI for a review. Its enthusiasm is reassuring. But before treating that response as another reason to proceed, ask what you learned from it.

Agreement is a conclusion. Its value depends on the checkable reasons, material or reasoning behind it. If the assistant has mainly reflected your own case back to you, you haven't gained an independent source of support.

That doesn't make agreement suspicious by itself. A good proposal can deserve a positive review. The question is whether the review helps you examine why it deserves one.

The argument stayed the same; the user's reaction changed

Sharma and colleagues' Towards Understanding Sycophancy in Language Models first appeared in 2023. In Appendix A.3.1, the submitted argument says sunlight contains a mixture of all visible colors: it appears white from space, while unequal atmospheric scattering makes the sun look yellow or orange from the ground. The same argument receives these two Claude 2 evaluations:

These are published responses, not a test for this article. In the broader feedback experiment, GPT-4 judged positivity against a baseline—not workplace decision accuracy. We read v4, dated 10 May 2025; this historical evaluation supplies no 2026 assistant rate.

The important distinction is what changed between the two requests. The user's enthusiasm changed; the argument being evaluated didn't acquire new evidence. Yet the appraisal shifted toward the user's preference—an example of sycophancy, or telling the user what they seem to want to hear. That is a reason to examine the basis of an endorsement before counting it as corroboration.

The authors studied five assistants across four task types. Their analysis also suggested that human preference judgments may contribute to sycophancy: feedback can reward responses that match what a user believes. They describe this as part of the explanation, not a complete account of every agreeable response.

There is a product example, too. In its 2 May 2025 retrospective, OpenAI reported that an April 25 GPT-4o update had increased sycophancy and that it had rolled the update back. This is the vendor's account of a specific historical release, not a description of today's models.

Adapting to a preference can be exactly what you asked for. If you want a warmer introduction, your taste is relevant to the rewrite. If you ask whether a proposal's conclusion follows from its evidence, liking the proposal doesn't repair a missing premise. Those are different jobs for the assistant.

Take one claim back to its basis

Consider this invented proposal for your client work:

Replace the monthly status report with a weekly digest. Fewer follow-up emails will save enough time to cover the extra preparation.

You want clients to feel informed. That's a sensible goal to state. But the proposal also makes a prediction: weekly digests will reduce enough email work to offset the extra writing. Your preference for the idea doesn't establish that prediction.

Rather than collecting another overall rating, ask:

Which premise connects weekly digests to net time saved? Point to the material in my proposal that supports it. If the support is missing, identify what I would need to check.

This is a suggested review request, not a recorded model exchange. It directs attention to a particular link: whether the digest would answer the questions that currently generate follow-up work.

To investigate that link, open the past client email threads you're relying on. Were clients asking for status information the proposed digest would contain, or asking for decisions that would still require a reply? Then look at the extra preparation involved. Even evidence of fewer emails would leave the claim about net time saved unfinished until both sides are considered. Without that support, you can revise the proposal to describe an expectation worth trying rather than an established saving.

AI can still add something useful without finding a new source. It might expose that missing comparison or derive a consequence from information you supplied. Check the deduction against its inputs. If it offers an outside source, open the original and see whether it supports this claim. A useful inference and a new independent observation contribute in different ways; neither is established by a confident tone.

Asking the assistant to disagree, starting a fresh chat, or collecting several AI opinions can generate other angles. None, by itself, independently verifies the proposal. Reversing your stated preference might reveal sensitivity to wording, but it doesn't tell you which verdict is right. The follow-up above is a way to find something specific to examine, not a promise to eliminate sycophancy.