The Conversion Rate Optimization (CRO) world is full of big talk and empty promises. Agencies love to flash glossy case studies, dangle sky-high ROI numbers, and sell you on being “full service.” But here’s the truth: most don’t have the depth, rigor, or guts to actually move the needle on your revenue.

If you’re serious about growth, you need to know how to spot the fakes. Below are ten red flags that scream “don’t waste your budget here,” plus three yellow flags that aren’t dealbreakers on their own but should make you pause and question whether the agency really has the chops.

10 Red Flags You Should Know About

Unrealistic ROI Claims

Beware of agencies promising staggering financial returns with certainty. Be skeptical of case studies showcasing exact dollar figures in annualized revenue increases or stats since it is difficult to assign an exact revenue amount to CRO only.

Suspicious Pricing

If the cost is remarkably lower than anyone else, the quality and depth of service might be compromised.

Jack-Of-All Trades Syndrome

Be wary of agencies that claim expertise in all areas of marketing without clear specialization. Effective CRO requires focused expertise, not a superficial sweep of services.

Lack of Pre-Test Calculations

Question the basis for the number of tests an agency proposes each month. Look for agencies that provide detailed pre-test analyses and transparent methodologies.

Overpromising Results

Promises that sound too good to be true often are. Seek a balanced approach, not one that is overly aggressive or risky.

Vague Objectives

Ensure deliverables and goals are clearly defined and directly tied to your bottom line.

Insufficient User Research

Opt for agencies that prioritize continuous, actionable research to inform their strategies.

Content Quality Concerns

Evaluate whether their content demonstrates genuine expertise or just fills space. The depth and relevance of content can indicate an agency’s real understanding of CRO.

Outdated Testing Practices

Steer clear of agencies stuck on simplistic or outdated testing methods such as relying on a million A/B tests with one change per test (a pro should be more complex), for example.

Superficial CRO Understanding

Test their knowledge with specific, technical questions about methodologies and statistics. Genuine expertise is evident when an agency can discuss advanced concepts comprehensively. 

Bonus: 3 Yellow Flags to Consider

Overemphasis on AI Capabilities

AI should NOT be the main selling point for an agency you’re considering. It’s too complex, case-specific, and multidisciplinary to be primarily handled by AI. The work must still be human-driven.

Restrictive Contracts

Watch out for clauses that make you reliant on the agency for basic data ownership. While you don’t enter a partnership expecting it to end, make sure any future transition will protect your company and your data.

Non-Assertive Approach

Effective CRO requires bold, well-substantiated consultation—not just going along to get along.

‍

A CRO agency that can’t back up its claims with real expertise, research, and a disciplined process isn’t just a bad investment—it’s a liability. Don’t fall for cheap promises, vague deliverables, or teams that play it safe instead of taking a stand.

The right partner will challenge your assumptions, bring hard data to every conversation, and care as much about your bottom line as you do. Anything less isn’t optimization—it’s guesswork dressed up as CRO.

‍