AI Quiz Maker: Ten Items Cover 5.1 Sentences, Not Ten

🔧 AI Tools 2026-11-16 9 min read

Doubling the item count from five to ten adds about half a sentence of coverage per passage, 84 per cent of items in ten-item sets share their cited sentence with a sibling, and one in five sibling pairs has one item's correct answer printed verbatim in the other's options.

📜 Table of Contents

AI quiz maker: ten items cover 5.1 sentences
Eight 130-word passages, three runs each, 200 items traced back to the sentence that decides each key: ten items cover 5.1 sentences, and the passages average 5.1. 67 of those 80 items share a sentence with a sibling item. Run on 2026-11-16.

The usual way to get a ten-question worksheet is to paste one passage and ask for ten items. I did that on eight passages and counted what the ten items actually test. They test five sentences. The passages have five sentences. In those ten-item sets, 67 of 80 items lean on a sentence that another item in the same set already leans on.

This is not a parsing artefact. It is the ceiling. Doubling the item count added about half a sentence of coverage per passage, and the instruction written to fix it, every item on a different sentence, moved the count by zero.

How it was run

Eight passages of 124 to 141 words, written for this test: submarine communications cables, Roman marine concrete, bamboo mast flowering, oyster reef restoration, katabatic wind, indigo vat dyeing, container lashing, street-tree canopy. Four to six sentences each, 5.1 on average.

Each passage went to deepseek-flash at temperature 1.0 three times: once asking for five items, once for ten, once for ten with one extra line requiring every item to test a different sentence. All 24 calls came back complete, 40 items, 80 and 80, and all 200 items had four distinct options and a key in range, so nothing was dropped. That is worth stating because it rules out the usual explanation: no truncation, no malformed JSON, no missing keys.

Each of the 200 items then went to deepseek-v4-pro at temperature 0 with its passage, asked to quote verbatim the one sentence that settles the answer. No empty quotes came back. Grouping items by the sentence they cite gives the coverage count. Two items citing the same sentence are not always the same question: one can ask what surrounds the cable core and another how far apart the repeaters sit, and the same clause settles both. But they are the same piece of the passage tested twice, which is what a worksheet is supposed to spread across its questions.

The count

5 items10 items10 + different-sentence line
items returned408080
distinct sentences cited374141
sentences per passage4.65.15.1
items sharing a sentence with a sibling6 of 40, 15%67 of 80, 84%66 of 80, 83%
mean option length, characters36.528.225.7

The middle column is the finding. Ten items cover 5.1 sentences, and the passages average 5.1 sentences. Every sentence gets roughly two items. The generator does not reach for new material or restructure the passage; it re-asks the sentence it has already used from another angle.

passagewordssentencessentences covered by 5 itemsby 10 items
submarine cables130555
Roman marine concrete133656
bamboo flowering127545
oyster reefs141555
katabatic wind132555
indigo dyeing124656
container lashing136545
street-tree canopy135444

Per passage, doubling the count added one sentence twice, and nothing four times. Submarine cables: five sentences, five covered by five items, five by ten. Street-tree canopy: four sentences, four covered either way. Pozzolan and indigo, the two six-sentence passages, went from five to six. The pattern is a saturating curve, and it saturates at one item per sentence.

What the extra five items look like

On the submarine-cable set, items 0 and 1 both come out of the first sentence: what do the cables carry, and how thick is one in deep water. Items 2, 3 and 4 come from the second sentence: how the fibres guide light, what surrounds the core, how far apart the repeaters are. Items 5 and 6 come from the third, 8 and 9 from the fifth. Only item 7 stands alone on a sentence of its own, the one about faults being caused by fishing and anchoring.

Some siblings are genuinely different questions. On the concrete passage, one item asks what seawater did to the lime clasts, another asks which mineral grew in the presence of the ash; different answers, same sentence. Two items on one sentence is defensible in a real exam. What is not defensible is a ten-question worksheet on a five-sentence passage, where five ideas are tested twice each and the student who understood the sentence answers both without extra thought. The item count went up; the number of things being tested did not.

The items also get thinner as the count rises. Mean option length falls from 36.5 characters at five items to 28.2 at ten, and direct-lift stems appear that would not survive review: what surrounds the cable core, answered by layers of steel wire and a copper conductor, which is the sentence verbatim with the verbs removed.

Answers printed in each other's options

The second cost hits printed sheets. Sibling items draw their options from the same sentence, so an item's wrong options are often the correct answer to its sibling. For every ordered pair of items inside a sentence group I asked a judge whether B's correct answer can be read off item A alone, A being the stem and four options with no passage attached. 216 pairs, 46 flagged yes, and I read all 46: 43 are real, 3 are the judge treating a shared topic as a shared answer.

ordered pairsjudge said yesconfirmed by reading
5-item sets622
10-item sets1002624
10-item sets with the line1101817
all21646, 21%43, 20%

A concrete pair from the concrete passage. Item 4 asks what happens to cracks in Roman harbour structures, key: they tend to seal themselves, and its second option is "they widen as water reaches steel". Item 5 asks what happens to a crack in ordinary portland concrete, key: it widens as water reaches reinforcing steel. B's answer is printed in A's options. The reverse direction is the same: item 5's first option, "it seals itself with lime clasts", is item 4's answer. A student working through the sheet in order has eliminated a third of item 5 before reaching the passage.

The same shape repeats: on bamboo, item 6's key contains "a whole stand dies together", which is item 7's answer; on indigo, items 0 and 9 share the claim that indigo does not dissolve in water; on canopy, items 7 and 8 carry each other's keys as distractors. 43 of 216 pairs is one in five, spread over 13 of the 24 sets and involving 60 of the 200 items.

None of this makes the individual items wrong. Each key checks out against the passage, which is the same result as every batch measured here since 11-10. What fails is the set: its questions are not independent, so the score at the end carries less information than the question count suggests.

The instruction that did nothing

The line was: every item must test a different sentence of the passage, no two items may rest on the same sentence or the same fact. It produced 41 distinct sentences, exactly the same as the plain request, and 66 repeated items against 67. What it did change is where the keys land. Across the three ten-item crops the keys sat at A 35 times out of 80, and on the submarine-cable passage all ten keys came back at position A. The plain ten-item run spread them 28, 25, 16, 11.

One set collapsing is not a trend, and 80 items is a small base. But the direction is the point: a line that constrains the content cannot manufacture material that is not in the passage. Five sentences hold about five testable facts. Asking for ten does not create a sixth sentence, and asking the model to be diverse mostly changes the wording of the instruction's echo.

Config to copy

The generation prompt, which is the ordinary one:

Write {n} multiple-choice items from the passage below. Each item needs four options,
exactly one correct, and the correct answer must be checkable against the passage.
Return JSON only: {"items":[{"stem":"...","options":["...","...","...","..."],"answer":0}]}
The answer is the 0-based index of the correct option.

Passage:
{passage}

Then the audit prompt, one call per item, which is the part that produced the number:

Below is a passage and one multiple-choice item. Quote verbatim the single sentence of
the passage that settles which option is correct. Copy the words exactly as they appear.
Return JSON only: {"quote":"..."}

Cluster the returned quotes by word overlap and count distinct groups. If the group count is below about 70 per cent of the item count, the set is repeating itself and the fix is more material, not more items. Script is standard library only: 24 generation calls with 6 threads, 200 audit calls and 216 pair calls with 12 threads, 80 seconds and 108 seconds on DeepSeek, roughly 450 calls in three minutes.

The working rule from this batch: a 130-word, five-sentence passage gives five items and no more. For ten items, hand over 250 to 280 words or two paragraphs, and treat each as a five-item unit. Shuffle options in code after generation rather than asking for balance, and if the sheet goes to print, check sibling items against each other, not only against the passage.

What this does not cover

One generation model and one judge, both DeepSeek; the judge here was asked only for a quote, which is a weaker task than the labelling it did in the earlier audits, but the clustering threshold is mine (word overlap 0.6, with every loose group read by hand) and the counts move a little if that threshold moves. Two items on one sentence can be a legitimate pair, so the coverage number is a ceiling on variety, not a quality score. Eight passages of expository English prose is a small sample, and a narrative or a table-heavy text could saturate differently. The all-A key collapse is one set out of eight and should be read as a warning, not a rate.

For practical purposes: five items per 130-word passage, ten items per 250 words, an audit that counts distinct cited sentences, and a shuffle before printing. The free tools that will hold your hand through none of this, and what each one actually caps, are here: the export limits of nine free quiz makers. The batch that compared English and Chinese failure modes is here: 17 of 120 wrong options were never in the passage. And where 96 English keys landed across four positions, before this batch pushed 40 per cent of 200 keys into A, is here: answer position bias. Two numbers to take away: ten items covered 5.1 sentences on passages averaging 5.1 sentences, and 43 of 216 sibling pairs have one item's answer printed in the other's options.

Coverage is one axis; what the items ask is another. Ten items on a table came back as ten copied cells, and when a calculation was demanded 18 of 60 mixed units: the table run.

Related Articles
2026-07-24
Best Free AI Meal Planners 2026
2026-07-27
Best AI Coding Assistant in 2026: 10 Developer Tools Compared
2026-10-18
Best AI Dictation Tool for Veterinary Clinics 2026: SOAP Notes With Four Legs
2026-08-06
Rerankers GitHub: The Best Open Source Reranker Projects in 2026
2026-07-24
Best Free AI Travel Planners 2026
2026-07-26
AI Music Generators Without Login

Written by our editorial team; tools listed here are tested or verified against public sources. Links point to official sites or GitHub repos for reference only — no paid placements.

💬 Comments (0)

No comments yet. Be the first!

Login to comment