How to Use AI for Competitor Analysis (and Check What It Tells You)

In a Harvard Business School experiment with 758 consultants, AI users were 19 percentage points less likely to reach the correct answer on one business task. That task was built to sit outside what the AI did well.

Competitor analysis contains tasks on both sides of that line. This guide shows where AI helps, which sources to collect, how to process them, how to verify the output and which legal lines to respect. The examples are hypothetical, and nothing here is legal advice.

Strengths and Blind Spots of AI Research

The study, published as Harvard Business School Working Paper 24-013, tested GPT-4 in 2023. On 18 tasks inside the AI's reach, consultants using it finished 12.2% more tasks, worked 25.1% faster and produced results rated over 40% higher in quality.

The outside-the-frontier task required combining interview notes with spreadsheet data to reach a business recommendation.

There, AI users were 19 percentage points less likely to be correct. The authors call this a jagged frontier: strong on some tasks, weak on similar-looking ones.

Read the limits too. It was one firm, one model from 2023 and one deliberately difficult task. Current tools differ. The lesson is that the line is hard to see from the inside, so you need checks that do not depend on trusting the output.

Research tasks carry a separate risk. The Tow Center tested eight AI search tools on 1,600 queries asking them to identify the source of a news excerpt, and, as the Columbia Journalism Review reported, they gave incorrect answers to more than 60% overall.

Error rates ran from 37% for the best tool to 94% for the worst, and two tools cited fabricated or broken links in over half of responses. It tested news excerpts in early 2025, so it warns about the mechanism, not today's rankings.

My working rule follows from both studies. Use AI for sorting, summarizing and drafting from documents you supply, and take care when it must recall facts, cite sources or combine numbers. Our guide on how to find, summarize and verify information covers the research side.

Begin at the Decision, Not the Tool

Competitor analysis is useful only when it feeds a decision. Before you open anything, write the decision in one sentence: whether to change a price, rewrite a homepage, add a feature or enter a market. The decision tells you which facts matter.

Then choose three to six competitors in three groups: direct rivals that sell the same thing, indirect rivals that solve the same problem differently, and substitutes such as doing nothing or using a spreadsheet. Customers compare across all three, and your analysis should too.

Finding them is a research job in itself. Ask recent customers who else they considered, and search the way a buyer would.

While you are still defining your market, choosing a profitable niche can show which rivals overlap with you in practice.

List five questions for each competitor and keep them identical across rivals: who they target, what they promise, how they price, which features they stress and what customers complain about. Identical questions are what make a comparison fair.

Take a hypothetical online coffee subscription with five competitors. The decision is whether to change its homepage promise. The five questions then map to pages and reviews you can collect, which is why the decision comes first.

Pairing Needs With Evidence

Each of the five questions has natural evidence behind it, and matching them in advance saves time. It also shows where a question has no good source, which is better to discover before you start than after you present:

• Who they target: homepage wording, case studies and the job titles in testimonials.
• What they promise: headlines, product pages and the first paragraph of their ads.
• How they price: pricing pages, plan comparison pages and published terms.
• Which features they stress: documentation, release notes and help center articles.
• What customers complain about: public reviews, forum threads and support replies.

Avoid questions without a source, such as how many customers a private rival has or what it earns. A model will answer them anyway, with confident figures that nothing supports. If a question has no source, leave it blank or label it a guess.

Search rivals need their own method. Which pages outrank yours is a separate question, and the basics in how to rank your blog on Google are a starting point. A general competitor review will not give you the data that question needs.

Collect Sources You Can Verify

AI works best on text you gather, so collect first. Useful sources include competitors' own pages, pricing pages, help centers, product documentation, customer reviews, job postings and, for public companies, annual reports. Each is public, dated and checkable.

For public rivals, the SEC's EDGAR guide explains that the annual report on Form 10-K includes audited financial statements, a discussion of material risk factors and management's discussion of results. Those sections often describe segments, customers and competitors in the company's own words.

Job postings reveal priorities, because hiring plans show where a company is investing. Reviews reveal gaps between promise and delivery.

Pricing pages show packaging. None of these needs secrets, and each reflects what the company chose to publish or what customers chose to say.

Rivals that publish little still leave traces. Private companies show their offers on websites, their customers in reviews and their hiring in job ads. Your own customers and lost deals are also evidence, and often the best, since they describe real choices.

Save what you collect. Copy the page text into a document with the URL and date, because pages change and your analysis must be reproducible. A dated archive also lets you see changes later, which is often more informative than a single snapshot.

AI search can help you find sources, but treat what it returns as leads. A comparison such as Perplexity compared with Google search shows how the tools differ. Open each source yourself before you rely on it.

Give the AI Documents, Not Open Questions

Open questions invite guessing. Asking a model what a rival charges or how many customers it has asks it to recall facts, which is where errors multiply. Giving it the pricing page and asking it to extract the plans is a far safer task.

Ask for structure. Request a list of plans, prices, limits and features as they appear in the text, with a short quote for each. Then compare the quotes with the page. A model that must quote is easier to check.

• Extract: "From the text below, list each plan, its stated limits and its features. Quote the exact words used for each item."
• Compare: "Using only the two documents provided, list differences in what each promises customers. Quote both."
• Classify: "Assign each review to one theme from this list, or to other. Return the review number and theme only."
• Flag gaps: "List claims in the text that nothing else in the text supports."

Notice the phrases using only the text provided and quote the exact words. They limit the model to what you gave it and make errors visible, because any invented detail will lack a matching quote.

Sorting many reviews into themes uses pattern-finding methods that our guide to how machine learning works explains for non-technical readers. You do not need the mathematics, but you do need to know that the themes are best guesses.

Keep your own confidential material out unless you have checked the tool's data terms. Competitor research rarely needs it, and one careless paste of internal plans into a public tool can create a problem that no analysis can fix.

A Worked Example Using Reviews

Take a hypothetical coffee competitor with 300 public reviews. You paste them in batches and ask the model to assign each to one theme. It returns support 75, shipping delays 66, quality 60, setup difficulty 54 and price 45.

That is 25% support, 22% shipping, 20% quality, 18% setup and 15% price. Reading 300 reviews at 20 seconds each takes about 100 minutes. Assigning themes with AI and checking a sample might take 35, which is my own estimate.

The sample check is the key step. Pick 30 reviews at random and read the assigned themes yourself. When 27 are right, the model scored 90% on your sample. That sounds high, but a sample of 30 is small.

A standard 95% confidence interval for 27 of 30 runs from about 74% to 97%. In other words, the true accuracy could plausibly be well below 90%. When the decision matters, check more reviews or sharpen the instructions and recheck.

Reviews also carry bias. People who write them tend to be unusually happy or unhappy, so they describe the extremes of a customer base. Treat the themes as signals about what stands out, not as a measure of how all customers feel.

Notice a second problem. The themes depend on the categories you gave it, and the model may merge distinct complaints. Spend time on the labels before the analysis, and test them on a handful of reviews first.

For the numbers side, see AI tools for data analysis and spreadsheets, which explains how to check totals and counts. A single wrong total can undo an otherwise careful analysis, so check the arithmetic yourself.

Audit Each Claim Before It Reaches a Slide

Treat the model's output as a draft from a fast, confident assistant who sometimes invents things. The fix is a claim ledger: a list of every factual statement in your analysis, with its source line and a status of checked or judgment.

Suppose a competitor brief has 40 claims. Twelve are quotes, 18 are numbers and 10 are inferences. Check all 30 quotes and numbers against the saved pages, and label the 10 inferences as judgments so readers know which is which.

Sampling helps when a list is long. If you check 20 claims and find no errors, the 95% upper limit on the true error rate is still about 14%. A clean sample reduces worry but does not remove it.

Quotes are the cheapest check, which is why prompts ask for them. A quote either appears on the page or it does not. Numbers come next, then inferences, which need your judgment about whether the evidence supports the claim.

Keep a log of the errors you find. If the model keeps misreading a particular kind of page or merging two themes, rewrite that prompt and recheck. Patterns in the errors tell you where the tool is weak for your material.

Some checks need no AI at all. Totals, counts and date comparisons can be done with ordinary formulas, and AI versus traditional software explains when each tool fits. Using the simpler tool where it works removes a source of error.

Be strictest about anything that will drive money. A wrong price in a pricing decision costs more than a wrong adjective in a messaging review. Match the depth of checking to the cost of being wrong, and write the rule down.

Common Errors That Spoil the Analysis

Small teams often repeat the same few mistakes, and most are easy to avoid once you know them. These are the ones to watch for first, and each can be caught with the checks described above:

• Asking for facts from memory: prices, customer counts and revenue figures should come from saved pages.
• Trusting a citation: a link proves nothing until you open it and find the passage.
• Mixing dates: a pricing page from last year and a review from this month do not describe the same product.
• Merging rivals: group competitors by type, because a direct rival and a substitute behave differently.
• Overclaiming: a theme in 300 reviews is not a market share, and a brief should not say it is.

Staying on the Right Side of the Line

Competitor research is normal when it uses lawful sources, but the edge is worth knowing. The Defend Trade Secrets Act, in 18 U.S.C. section 1839, says improper means include theft, bribery, misrepresentation, breach of a duty of secrecy and espionage.

The same section says improper means do not include reverse engineering, independent derivation or other lawful means of acquisition.

Reading public pages, buying a rival's product to study it and interviewing your own customers generally fall on the lawful side.

Misrepresentation deserves attention, because competitor research tempts people to pose as customers. Signing up under a false identity to see non-public pricing or documents can cross the line, and it may breach a site's terms. Use your real identity or stay with public sources.

Collection methods have their own rules. Many sites prohibit automated scraping in their terms of service, and the law differs by country and situation. I did not research scraping law here, so ask a lawyer before you automate collection at scale.

Do not seek confidential material from a competitor's former employees, and do not share competitively sensitive information, such as non-public prices, with competitors. These are general cautions, not legal advice, and the details vary by jurisdiction.

Keep the scope narrow while you learn these rules. A look at how small businesses are using AI shows that most start with one workflow, and that restraint also limits legal and accuracy risk.

From Findings to Decisions

Analysis becomes useful when it changes something. Summarize the findings as differences you can act on: who is targeted, what is promised, how plans are packaged and where customers are unhappy. Keep it to one page, because a long report rarely gets used.

Take the hypothetical coffee market again. Across five competitors, you record six claims made on their product and subscription pages, and count how many of the five rivals make each claim:

• Free shipping: 4 of 5 competitors offer it, so it is expected, not special.
• Pause or skip: 2 of 5 let customers pause a subscription.
• Roast date shown: 1 of 5 states it on the product page.
• Carbon-neutral claim: 3 of 5 make some claim, with little detail.
• Flexible frequency: 3 of 5 allow changes.
• Recycled packaging: 2 of 5 mention it.

The rare claim, roast date, is a candidate for a homepage message, because few rivals say it and customers can verify it. The common claim, free shipping, is table stakes. Counting how many rivals make a claim shows which messages are crowded.

Reviews add the other half. If complaints about support run at 25% for the biggest rival, a promise of faster responses may resonate, provided you can deliver it. A message you cannot support becomes your own review problem.

As a content strategist, I have found that the most useful result of a competitor review is rarely a new idea. It is a clearer view of where everyone sounds the same, because that is where a different claim stands out.

Tie the findings to strategy. A business plan that attracts investors usually explains why you win against named rivals, and your analysis should supply that evidence, not just a list of features.

Turn each finding into a test. If roast date is rare among rivals, change the homepage for a few weeks and compare sign-ups. A finding that no experiment can confirm is only an opinion, and it should be labeled that way in your brief.

Share the brief with the people who can use it. Sales can learn how rivals are positioned, support can see where rivals fall short, and product can see which features are crowded. A brief that stays with one person changes nothing.

Writing the One-Page Brief

A brief that fits on one page gets read. A simple structure keeps it honest, makes it easy to update each month and gives a busy colleague one place to look for the answer:

• Decision: the one choice the brief supports.
• Findings: three differences, each with its evidence.
• Confidence: how much was checked and what remains a judgment.
• Next test: the smallest experiment that would confirm or reject the main finding.

Aim the next test at differences that are hard to copy. If you want to build a competitive advantage, ask which of your strengths rivals could not match quickly, and which of theirs would take you years to replicate.

The confidence line is the one most briefs skip. Saying that 30 of 40 claims were checked and 10 are judgments lets a reader weigh the brief properly. It also protects you when a rival changes something after you wrote it.

A Monthly Rhythm

Competitors change, so a one-off report ages quickly. Set a monthly routine: re-save key pages, run the same extraction prompts, compare with last month and note what changed. Two hours a month is my estimate for a handful of rivals, or 24 hours a year.

Changes matter more than snapshots. A new pricing tier, a dropped feature or a shift in job postings signals a decision someone made. Record the date and your reading of why, and check later whether you were right.

If your analysis points to a new region, the next question changes. Choosing the right market to expand into is a different task, and it needs its own evidence before you commit any budget.

My own view is that AI is best treated as a fast reader and a poor witness. It can sort what you collect in minutes, but it should never be the source of a fact. Keep that division and the method holds up.

Start small. Pick one decision, three rivals and five questions, collect the pages, run the prompts and check a sample before you share anything. Repeat monthly for a quarter, then decide how much more analysis the decision needs.