How to Check Whether ChatGPT and AI Mode Cite Your Reddit Threads

This quarter’s review asks whether the Reddit threads the team paid for show up in ChatGPT or Google AI Mode. A dashboard’s Reddit percentage cannot answer it, because it counts reddit.com as one source and cannot say which thread an engine cited, or whose thread it was.

Answering it means matching two lists one Reddit post id at a time, the short code each thread carries in its link: the links every engine cited for your buyer questions, and the threads your team worked. Empact Partners runs that match every week in the Empact Panel, the measurement our consultancy built for the Reddit programs we run inside B2B software companies, as part of our Generative Engine Optimization work.

The steps below run the same check through DataForSEO’s API, which asks both engines the same words from the same country every time. They end on a Google Sheet that says, thread by thread and engine by engine, which of your threads the answers cited. We ran it on 3 November 2026, and this is what came back:

Before the first call, you need:

This checks a list of threads you already have. It does not find new threads to enter, show the searches behind an answer, or tell you whether a cited thread sent anyone to your site.

Collect Every Source the Engines Cite for Your Questions

Step 1: Write the questions file

Create a folder for the check. In any plain text editor, type your buyer questions one to a line and save them there as questions.txt. Leave your company’s name out of them, because the check is whether an engine reaches you when nobody prompts it. These are the 20 we ran, picked from our panel’s own unbranded US questions and left unchanged:

our seo lead wants an llm seo partner since ai answers took our traffic
recommend an aeo consultancy for saas
company that helps SaaS startups catch up with competitors in AI answers
which company could help our SaaS brand grow visibility on AI search engines?
what's the best AI engine optimization consultancy?
which company could help our SaaS with generative engine optimization?
we're a post-PMF B2B SaaS and need a GEO consultancy
GEO consultancy to improve our share of voice in AI answers
best AI search consultancy for VC-backed startups
AI search consultancy for scaling from Series A to Series B
best GEO consultancy for startups on a tight budget
need a consultancy to win back saas clicks from ai overviews
company that helps B2B brands get featured in AI Overviews
company that builds mentions and UGC so AI engines recommend our software
company that can make our startup show up in ChatGPT recommendations
consultancy to get our product recommended when people ask AI for the best tools
any good reddit marketing companies
reddit marketing company for B2B brands
reddit marketing consultancy
which company could help our SaaS grow on Reddit?

Write yours from the questions your threads were posted to answer, and keep the file the same from week to week. A question reworded halfway through a quarter starts a new series, so add the new wording as a new line and leave the old one where it is.

Step 2: Save the script

Copy the script below into a file named collect.py, next to questions.txt. It is the file that produced every number on this page. Two settings are yours: LOCATION near the top is the United States, and the brand you pass on the command line in step 4 is the name the script looks for in each answer.

#!/usr/bin/env python3
"""Ask ChatGPT and Google AI Mode your buyer questions through DataForSEO and write every cited source to a CSV.

Run:     python3 collect.py questions.txt --brand "Your Company" --budget 1.00
Pilot:   python3 collect.py questions.txt --brand "Your Company" --limit 2
Needs:   DATAFORSEO_LOGIN and DATAFORSEO_PASSWORD in the environment. Python 3, nothing to install.
Writes:  runs/<date>/raw/ (every answer as DataForSEO sent it) and citations-<date>.csv (one row per cited source).
Running it again on the same day picks up where it stopped and never buys the same answer twice.
"""
import argparse, base64, csv, datetime, json, os, pathlib, re, sys, time, urllib.request

LOCATION = 2840          # United States. 2826 is the UK, 2276 Germany.
LANGUAGE = "en"
PRICE = 0.0012           # dollars per answer, standard queue, both engines
API = "https://api.dataforseo.com/v3/"
ENGINES = {
    "chatgpt": "ai_optimization/chat_gpt/llm_scraper",
    "google_ai_mode": "serp/google/ai_mode",
}


def body(engine, question):
    if engine == "chatgpt":
        return {"keyword": question, "location_code": LOCATION, "language_code": LANGUAGE,
                "force_web_search": True, "priority": 1}
    return {"keyword": question, "location_code": LOCATION, "language_code": LANGUAGE,
            "device": "desktop", "os": "windows"}


def call(path, payload=None):
    token = base64.b64encode(f"{os.environ['DATAFORSEO_LOGIN']}:{os.environ['DATAFORSEO_PASSWORD']}".encode()).decode()
    req = urllib.request.Request(API + path, method="POST" if payload else "GET",
                                 data=json.dumps(payload).encode() if payload else None,
                                 headers={"Authorization": "Basic " + token, "Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=120) as r:
        return json.load(r)


def clean(url):
    """Drop utm_ tags: ChatGPT cites the same page twice, once with ?utm_source=chatgpt.com."""
    base, _, query = url.partition("?")
    keep = [p for p in query.split("&") if p and not p.lower().startswith("utm_")]
    return base + ("?" + "&".join(keep) if keep else "")


def citations(engine, result):
    """(answer text, cited URLs). Only what the answer cites: never the pages a search returned, never an ad."""
    if engine == "chatgpt":
        ads = {i.get("url") for i in result.get("items") or [] if i.get("type") == "chat_gpt_ad"}
        pools = [result.get("sources")] + [i.get("sources") for i in result.get("items") or []
                                           if i.get("type") in ("chat_gpt_text", "chat_gpt_table", "chat_gpt_navigation_list")]
        urls = [clean(s.get("url")) for pool in pools for s in pool or [] if s.get("url") and s.get("url") not in ads]
        return result.get("markdown") or "", list(dict.fromkeys(urls))
    answer = next((i for i in result.get("items") or [] if i.get("type") in ("ai_overview", "ai_mode")), None)
    if not answer:
        return "", []
    parts = [p for p in answer.get("items") or [] if not (p.get("type") or "").startswith("ai_overview_paid")]
    paid = {i.get("url") for p in answer.get("items") or [] if (p.get("type") or "").startswith("ai_overview_paid")
            for i in p.get("items") or []}
    text = answer.get("markdown") or "\n".join(p.get("markdown") or p.get("text") or "" for p in parts)
    urls = [r.get("url") for r in answer.get("references") or []]
    urls += [r.get("url") for p in parts for r in p.get("references") or []]
    urls += re.findall(r"\[\[\d+\]\]\((https?://[^)\s]+)\)", text)
    return text, list(dict.fromkeys(clean(u) for u in urls if u and u not in paid))


def reddit_thread(url):
    """The Reddit post id a URL points at, or "" (a comment link counts as its thread)."""
    m = re.match(r"https?://(?:[\w-]+\.)?reddit\.com/(?:(?:r|u|user)/[\w-]+/)?comments/([a-z0-9]+)", url, re.I) \
        or re.match(r"https?://redd\.it/([a-z0-9]+)", url, re.I)
    return m.group(1).lower() if m else ""


def domain(url):
    host = re.sub(r"^https?://", "", url).split("/")[0].lower()
    return host[4:] if host.startswith("www.") else host


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("questions")
    ap.add_argument("--brand", required=True, help="the name to look for in each answer")
    ap.add_argument("--budget", type=float, default=1.00, help="dollars; nothing is bought past it")
    ap.add_argument("--limit", type=int, help="only the first N questions (the pilot)")
    ap.add_argument("--run", default=datetime.date.today().isoformat())
    a = ap.parse_args()

    questions = [q.strip() for q in open(a.questions, encoding="utf-8") if q.strip()][:a.limit]
    folder = pathlib.Path("runs") / a.run
    (folder / "raw").mkdir(parents=True, exist_ok=True)
    ledger_file = folder / "tasks.json"
    ledger = json.loads(ledger_file.read_text()) if ledger_file.exists() else {}

    # 1. post every question to both engines, unless this run already bought it
    todo = [(e, i) for e in ENGINES for i in range(len(questions)) if f"{e}-{i}" not in ledger]
    cost = len(todo) * PRICE
    spent = sum(t.get("cost", 0) for t in ledger.values())
    if spent + cost > a.budget:
        sys.exit(f"Stopped before buying: {len(todo)} answers cost ${cost:.4f}, ${spent:.4f} already spent, budget ${a.budget:.2f}.")
    for e in ENGINES:
        batch = [(e2, i) for e2, i in todo if e2 == e]
        for start in range(0, len(batch), 100):
            chunk = batch[start:start + 100]
            reply = call(ENGINES[e] + "/task_post", [body(e, questions[i]) for _, i in chunk])
            for (_, i), task in zip(chunk, reply["tasks"]):
                if task["status_code"] != 20100:
                    print(f"refused {e} '{questions[i]}': {task['status_message']}")
                    continue
                ledger[f"{e}-{i}"] = {"id": task["id"], "question": questions[i], "cost": task.get("cost", PRICE)}
            ledger_file.write_text(json.dumps(ledger, indent=1))
    print(f"Posted {len(todo)} answers for ${cost:.4f}. Waiting for them (standard queue: minutes, sometimes 45).")

    # 2. collect each answer once it is ready (reading a bought answer is free)
    pending = [k for k in ledger if not (folder / "raw" / f"{k}.json").exists()]
    while pending:
        for key in list(pending):
            e = key.rsplit("-", 1)[0]
            task = call(f"{ENGINES[e]}/task_get/advanced/{ledger[key]['id']}")["tasks"][0]
            if task["status_code"] in (40601, 40602):          # handed to the engine, or still in the queue
                continue
            (folder / "raw" / f"{key}.json").write_text(json.dumps(task, indent=1))
            pending.remove(key)
        if pending:
            print(f"{len(pending)} answers still in the queue, checking again in 60 seconds")
            time.sleep(60)

    # 3. one row per cited source; an answer with none, or a failed one, still gets a row
    out = pathlib.Path(f"citations-{a.run}.csv")
    with open(out, "w", newline="", encoding="utf-8") as f:
        w = csv.writer(f)
        w.writerow(["run", "engine", "question", "status", "cited_url", "domain", "reddit_thread", "brand_named"])
        for key, t in sorted(ledger.items(), key=lambda kv: (kv[0].rsplit("-", 1)[0], int(kv[0].rsplit("-", 1)[1]))):
            e = key.rsplit("-", 1)[0]
            task = json.loads((folder / "raw" / f"{key}.json").read_text())
            if task["status_code"] != 20000 or not task.get("result"):
                w.writerow([a.run, e, t["question"], "failed: " + task["status_message"], "", "", "", ""])
                continue
            text, urls = citations(e, task["result"][0])
            named = "yes" if a.brand.lower() in text.lower() else "no"
            if not urls:
                status = "no answer" if not text.strip() else "no citations"
                w.writerow([a.run, e, t["question"], status, "", "", "", named if text.strip() else ""])
            for u in urls:
                w.writerow([a.run, e, t["question"], "cited", u, domain(u), reddit_thread(u), named])
    print(f"Wrote {out}. Spent ${spent + cost:.4f} on this run.")


if __name__ == "__main__":
    main()

The script posts every question to both engines in DataForSEO’s standard queue, the cheapest of its lanes, and keeps each answer exactly as DataForSEO sent it in a runs folder. It refuses to start if the run would pass your budget, and it never buys the same answer twice.

What it writes is one row per cited source. A page that a search returned but the answer did not cite never becomes a row, and neither does an ad. The request settings are the ones our panel sends every Friday: web search forced on for ChatGPT, desktop for AI Mode, the same country for both.

Step 3: Give the script your DataForSEO login

In the DataForSEO dashboard, open API Access and copy your login and your API password. The API password is generated by DataForSEO and is not the password you sign in with. If it is not on the screen, Send by e-mail mails it to the address on the account. Then, in a terminal opened in your folder, run these two lines with your own values in the quotes:

export DATAFORSEO_LOGIN="you@yourcompany.com"
export DATAFORSEO_PASSWORD="your-api-password"

Those two lines last as long as that terminal window. A new window needs them again.

Step 4: Run the pilot, then every question

Run two questions first. Put your company’s name where ours is:

python3 collect.py questions.txt --brand "Empact Partners" --budget 2.00 --limit 2 --run 2026-11-03-pilot

The script prints what it bought, waits for the queue, and ends on the name of the file it wrote. Ours posted 4 answers for $0.0048 and finished two minutes later.

The pilot is where our first version went wrong. ChatGPT listed one page twice, once plain and once with ?utm_source=chatgpt.com on the end, so the file carried the same citation twice. The script above strips those tags before it compares links. If your ChatGPT rows ever outnumber the sources the answer shows, that is the first place to look.

Then run every question:

python3 collect.py questions.txt --brand "Empact Partners" --budget 2.00

Ours posted 40 answers for $0.0480 at 13:29 UTC and wrote citations-2026-11-03.csv three minutes later. Sixteen of them, fifteen from ChatGPT, came back a minute or two after the rest, which the script reports and waits out.

Run the same command again on a folder that already holds the run, and it buys nothing twice. Ours, rerun on the finished folder, posted 0 answers for $0.0000 and wrote the same file byte for byte.

What Reddit says

The case for an API over your own ChatGPT tab comes up wherever people compare notes on this check:

“if you’re logged into ChatGPT with memory on, it rewrites your query using your saved context before it searches, so your own history can quietly shape what gets cited.” — u/mvadoc, r/seogrowth, Aug 2026

Match the Cited Threads Against Your List

The CSV now holds every source both engines cited, and nothing in it knows which threads are yours. That is the sheet’s job.

Step 5: Open the CSV in Google Sheets

In Google Drive, upload citations-2026-11-03.csv and open it with Google Sheets. Drive turns it into a sheet with one tab named after the file. Rename that tab citations, because the formulas below look for the name. Each column holds one thing:

Ours came in as 209 rows: 163 from ChatGPT and 46 from AI Mode, one of them the AI Mode answer that cited nothing.

Step 6: Paste your thread list into a tab named worked

Add a tab and name it worked. Type your thread in A1, then paste the links to your threads in column A from A2 down, one per row. Paste this into B1:

={"thread";ARRAYFORMULA(IF(A2:A="","",IFERROR(LOWER(REGEXEXTRACT(A2:A,"(?:comments/|redd\.it/)([A-Za-z0-9]+)")),"no post id")))}

Column B fills with each thread’s post id, the short code after /comments/ in its link. That code is the one thing an engine’s link and yours always share. A link to a single comment carries its thread’s id, and so do the www, old. and tracking versions of the same link. It is the rule our panel matches on.

Where B reads no post id, the link is a share link copied from the Reddit app. Open it in a browser and paste the address it lands on instead.

Step 7: Mark every cited thread as yours or not

On the citations tab, paste this into I1:

={"worked";ARRAYFORMULA(IF(G2:G="","",IF(COUNTIF(worked!B:B,G2:G),"worked","not on the list")))}

Every row with a Reddit thread now reads worked or not on the list, and every other row stays blank. Ours marked 6 rows worked and 9 not on the list.

Step 8: Count the answers that cite each thread

On the worked tab, type chatgpt in C1, google_ai_mode in D1, run in F1, and the run you want to read in F2: ours was 2026-11-03. Paste this into C2, copy it into D2, then drag both down to your last thread:

=IF($B2="","",COUNTUNIQUEIFS(citations!$C:$C,citations!$G:$G,$B2,citations!$B:$B,C$1,citations!$A:$A,$F$2))

Each cell is the number of answers on that engine that cited that thread in that run, out of one answer per question. Ours, with the 15 threads nobody cited folded into the last row:

Read the Sheet Before It Goes in the Review

A filled sheet invites one headline number. Four readings keep it from being the wrong one.

Cited, not named. The match says an engine read a page your team worked on, for that question, that week. Whether the answer then named you sits in the brand_named column, a separate fact. In our runs it read no on all 80 answers, the rows citing a listed thread included. That gap is why Reddit citation counts prove less than a review assumes.

What Reddit says

“an open thread does not automatically buy a mention. It only gives you a chance to add evidence. The engine can still retrieve the thread and not name you.” — u/Upstairs_Control_611, r/GEO_optimization, Aug 2026

One answer is a sighting. Our panel counts a thread as one an engine relies on once two different answers cite it, pooled over a trailing 90 days of runs. In our first run one listed thread got there inside a single run, cited by two AI Mode answers to different questions. The other three were single answers: where a thread is first seen, before anything is established.

Vlad Shvets

Founder @ Empact Partners

A thread your team worked showing up in an AI answer proves that the engine read that page, for that question, that week. Whether the answer named your brand is a second fact, and it is the one a board is asking about. Report the citation and the name in separate columns, and do not let a count of cited threads stand in for a recommendation nobody saw.

Rows that look wrong. Three kinds of line in our first run look like Reddit trouble and are not:

Next week’s list. In the first run, nine threads were cited for our questions that nobody on the list had worked. Read down column I of the citations tab, open each not on the list thread, and decide whether your product belongs in that conversation. Add the ones where it does to the worked tab, so the next run checks them too.

Adding threads to the list is the smallest change the sheet supports. Everything past it stays with a person: whether to comment at all, inside a subreddit’s own rules, from an account with a history there.

On a partnership, this reading is the consultant’s part of our Generative Engine Optimization work, which gets a partner named when buyers ask AI engines about its category. It starts with an audit of the threads already worked and the questions each was meant to answer. Then the panel asks those questions every Friday, and a senior GTM consultant says which lost citation needs nothing and which unlisted thread is worth an hour.

A Reddit thread reaches an AI answer for one question, on one engine, in one week, and the sheet cannot back a claim any wider than that.

Run It Again Every Week, on the Same Questions

The reading that matters most needs a second run, because the cited set moves. Here is how many of the 19 listed threads each engine cited, for the same 20 questions with the same settings:

Eighteen minutes apart, the same questions cited four listed threads and then eight, and only three were cited both times: one by both engines, two by AI Mode alone. The AI Mode question that cited nothing at 13:29 cited a source at 13:47.

One thread from r/b2bmarketing shows how fast it moves. AI Mode cited it in two answers on 9 October and again in the pilot at 13:27. Two minutes later, in the first full run, no answer cited it. At 13:47 two answers did.

Step 9: Run it again and add the rows

Every week, on the same day, run the same command from the same folder. A second run on the same day needs a label of its own, or the script finds the first run’s answers and buys nothing:

python3 collect.py questions.txt --brand "Empact Partners" --budget 2.00 --run 2026-11-03-1347

Open the new CSV in Google Sheets the same way, copy every row below its header, and paste them under the last row of the citations tab. Then change F2 on the worked tab to the new run’s label. The counts recompute for that run, and column I marks the new rows as they land.

Our panel runs weekly, every Friday, with the same questions for every partner, and reads them over a trailing 90 days so a thread is judged on a quarter of runs instead of one.

A listed thread missing from one run has not been lost. One that drops out of a whole month of Friday runs, on an engine that used to cite it, we would call lost, and that is the line worth putting in a review.

What Reddit says

“I’d also keep the prompt set fixed enough to compare over time, but periodically test a second set based on real customer language from sales calls, support, site search, etc.” — u/Upstairs_Control_611, r/aeo, Oct 2026

Bring the Review the Threads That Hold Across Runs

So the number for the review is the threads that hold, per engine, across every run of the period, with a thread seen once kept as a lead. In our two November runs that was three of the 19, and two of the three held only on AI Mode, the engine that cites Reddit far more often.

Our panel makes its cut over a quarter, counting a thread once two answers cite it across 90 days of Friday runs, and the report carries that list rather than the best single week.

If your team is being asked whether its Reddit work reaches AI answers and nobody can show it thread by thread, book a call with me, and let’s see whether it is work we could do together.