Question set

A question set is a fixed list of questions used to measure how AI assistants answer. It defines the denominator of every GEO metric, so swapping in a different batch of questions is the same as switching to a different ruler.

Some teams call it a prompt set or a test question library. Some AI visibility tools call them “tracked prompts.”

Why it comes first

Mention rate, first-recommendation rate and name confusion rate are all ratios of the form “appeared in X out of N questions.” If N is not fixed, the ratios cannot be compared. That is why the first step in any generative engine optimization (GEO) program is to write the questions down, number them and freeze them before measuring anything.

What a question set usually includes

A good set mixes several types of question, because each one tests something different:

Type	Example shape	What it tests
Category keyword	“Which firms offer GEO services in Shanghai?”	Whether the model considers you part of the category
Decision query	“How do I choose a GEO vendor?” / “Is it worth doing?”	Whether you appear when buyers are choosing
Long-tail question	Specific scenarios, low frequency, many variants	Coverage of real buyer situations
Branded	“What does [company] do?”	Whether the description is accurate
Disambiguation	Questions that include a confusable same-name entity	Whether the brand entity is identified correctly

Category questions deliberately do not name your brand. If every question contains your name, you will measure a mention rate close to 100% that means nothing.

Writing good questions

• Use buyer language, not marketing language. Draw on what sales hears, support tickets and real search queries. Questions that sound like your own slogans produce inflated results.
• Cover the buying journey. Early research, comparison and final decision questions behave differently.
• Include the markets you sell into. An exporter should test in English on the assistants its overseas buyers use, not only in Chinese on Doubao or Tencent Yuanbao.
• Keep enough questions. With too few, a single question flipping from one round to the next swamps any real change.

The rules you have to write down

Beyond the questions themselves, document the measurement rules:

• The exact wording of each question
• The list of platforms
• Whether web search is on or off
• How many times each question is sampled
• What counts as a valid answer
• How “first position” is judged, for example when an answer is an unordered list

Put these in a written specification and follow it line by line every time you retest. Without that, there is no real baseline and retest, only a series of unrelated snapshots.

Adding questions later

You will think of better questions after the first round. Add them, but track new questions separately and do not merge them into historical figures. Otherwise a change in the ratio may come from the new questions rather than from anything you did.

Limits

A question set is a constructed sample. It is not the real distribution of what users ask, which AI platforms do not make public. What a question set measures is the direction of change under fixed conditions, not market share.

Two failure modes are common: a set too small to separate signal from random variation, and a set written so close to your own copy that it measures how well the model repeats your brochure. Both produce numbers that look precise and say little.

If you want a starter question set for your category and a first baseline, request a free GEO audit.