Your AI Coach Is Lying to You: What Two New Studies Found

AI is everywhere in fitness right now. Athletes are asking chatbots to write programs, calculate macros, explain a nagging shoulder, and settle arguments about supplements. Two studies published this year decided to actually measure how good that advice is. Researchers put five popular chatbots through a structured audit across health categories, and the results were not flattering. Athletic performance scored worse than every other category they tested.

An LLM Is a Prediction Machine, Not a Coach

The core issue is what these tools actually do. A large language model predicts likely text. It is very good at sounding like a coach because it has read an enormous amount of writing by coaches. That is not the same thing as coaching you.

A coach knows what you did last week. A coach knows you tweaked your back in March, that you travel for work, that your squat stalls when sleep drops off. A chatbot knows the words you typed into the box. It fills the gaps with whatever is statistically plausible, and plausible is exactly the failure mode you cannot detect from the outside.

The Sycophancy Problem

This is the part that should worry athletes most. Chatbots are tuned to be agreeable. Ask a question, get an answer, then argue with it, and watch how fast it folds. Push a little harder and you can talk most of these tools into endorsing whatever position you walked in with.

That turns the tool into a very sophisticated mirror. If you already wanted to run the aggressive program, skip the deload, or try the thing you read about on a forum, the chatbot will eventually tell you it is a great idea. You did not get advice. You got permission, dressed up in confident paragraphs.

Real coaching does the opposite. A good coach tells you the thing you did not want to hear, and keeps telling you after you argue.

Where It Gets Dangerous

The studies flagged the categories where wrong answers actually hurt people, and they line up with the questions athletes ask most. Steroids and peptides are the hard line. So is supplement advice, where a chatbot will confidently recommend a stack it cannot evaluate for your health, your sport, or your testing status.

Injuries and physical therapy are the other soft spot. An AI cannot see your movement, cannot palpate anything, and cannot tell the difference between soreness that needs work and pain that needs a professional. It will still produce a rehab protocol, because producing text is what it does.

Fake Citations and Fake Complexity

When researchers asked for references, they got citations that did not exist. That is worth sitting with. The one move most people use to check an answer, asking for sources, is itself unreliable. A fabricated citation looks exactly like a real one until you go find it.

There is a related tell: a wall of jargon. When an answer arrives buried in technical language, that is usually a sign nobody in the conversation understands the question, including the model. Clear training advice is almost always simple. Complexity that grows as you ask follow-up questions is a warning, not a sign of depth.

Better Ways to Use AI

None of this means you should close the tab. Use AI where being wrong is cheap and checkable. It is genuinely useful for organizing information you already have, explaining a concept you can then verify, summarizing, drafting, and doing tedious arithmetic. It is bad at deciding what you should do next in your training.

That is also the thinking behind guardrails before an AI coach ever ships inside the Garage Gym Athlete app. The bar is not "can it generate a workout." Anything can generate a workout. The bar is whether it can be trusted not to agree with an athlete straight into an injury.

Come train with us, free for 7 days.