Linear-on-the-Fly Testing¶
A testing method that assembles a candidate-specific nonadaptive exam form from an item bank under shared content and difficulty requirements.
Core Idea¶
Linear-on-the-fly testing (LOFT) builds a candidate-specific exam form from an item bank under a common content and difficulty specification. The assembled questions are then given linearly: the examinee's earlier answers do not decide which later questions appear. The forms differ, but they are designed to measure the same target comparably.[ref-b5e57cbbbfe4][ref-e2e93358c9d5]
The method requires more than a computer or shuffled questions. NCEES uses it for year-round professional exams; onSET uses it for language placement. Both describe newly assembled forms subject to common topic and difficulty controls.[ref-b5e57cbbbfe4][ref-e2e93358c9d5]
Scope of Application¶
LOFT is useful when a testing program has an approved item bank and enough information about its items to build forms that satisfy one exam blueprint. It can serve licensure, certification or placement. NCEES, onSET and the Pharmacy Examining Board of Canada describe operational uses; their purposes and item types differ, but the form-assembly relation is shared.[ref-b5e57cbbbfe4][ref-e2e93358c9d5][^ref-a2ff4b3b6581]
The method does not itself prove that every form has identical measured difficulty, that every candidate sees a globally nonoverlapping set, or that cheating has been eliminated. Those are stronger program-specific claims. onSET reports Rasch calibration of its own bank; that calibration detail should not automatically be assigned to every LOFT program.[^ref-e2e93358c9d5]
Clarity¶
Ask three separate questions: Is each form assembled for a candidate? Does it meet the common specification? Is it delivered without response-driven item selection? A computer-based exam can fail the first test: NCEES contrasts its year-round LOFT exams with smaller single-day computer exams where everyone gets the same questions. An ordinary quiz with random items can fail the second. A computerized adaptive test fails the third because responses change later item selection.[^ref-b5e57cbbbfe4]
Also distinguish a design target from a validated result. Shared topic counts and similar difficulty are how operators describe the forms they intend to build. Whether the resulting scores support an intended decision still requires evidence for that particular exam.[ref-b5e57cbbbfe4][ref-e2e93358c9d5]
Manages Complexity¶
An item bank may allow many possible forms. LOFT turns that form-building task into a controlled procedure: choose eligible items, enforce the content and difficulty constraints, and deliver the resulting fixed set. The specification lets a program compare forms without manually authoring every possible combination.[ref-b5e57cbbbfe4][ref-e2e93358c9d5]
The procedure also exposes a capacity limit. A small bank or a tight content requirement can leave too few admissible combinations. More randomness alone does not repair a form that fails its blueprint; the bank and constraints must be examined together.
Abstract Reasoning¶
To assess a claimed LOFT system, identify the construct to be measured, the shared blueprint, the eligible item bank and the rule that assembles the form. Then check whether later items depend on prior answers. If they do, the system may be adaptive. If forms differ but have no comparable content or difficulty target, it may be randomization rather than LOFT.[ref-b5e57cbbbfe4][ref-e2e93358c9d5]
The form-assembly rule is a literal Algorithm component. That general Prime occurs in many other tasks, but those tasks are not LOFT unless they assemble psychometric test forms under shared requirements and deliver them linearly.
Knowledge Transfer¶
The method carries from engineering and surveying licensure exams to language placement C-tests. The question contents, stakes and scoring purposes change; candidate-specific assembly under common form requirements and nonadaptive delivery remain. PEBC's pharmacist qualifying exam supplies another documented use. The transfer is within testing, not a claim that any on-demand generator is LOFT.[ref-b5e57cbbbfe4][ref-e2e93358c9d5][^ref-a2ff4b3b6581]
Example¶
NCEES year-round exams. The operator describes a LOFT algorithm that gives each examinee a different complete question set for a given exam, with the same number of questions in each topic and similar relative difficulty. Mapped roles: blueprint → topic counts and difficulty; bank → eligible licensure items; assembler → candidate-specific form; linear delivery → questions not selected from earlier responses.[^ref-b5e57cbbbfe4]
onSET language placement. The operator describes a Rasch-calibrated C-test bank and a newly compiled task set for each participant, controlled for topic and total difficulty. Mapped roles: blueprint → placement task requirements; bank → calibrated C-tests; assembler → participant set; linear delivery → assembled set rather than response-adaptive item choice. The operator account states the method, not an independent validation of every outcome.[^ref-e2e93358c9d5]
Relationships to Other Abstractions¶
Current abstraction Linear-on-the-Fly Testing Domain-specific
Parents (1) — more general patterns this builds on
-
Linear-on-the-Fly Testing is part of Algorithm Prime
An assembly algorithm is an internal constituent of each LOFT administration.
Hierarchy paths (2) — routes to 2 parentless roots
- Linear-on-the-Fly Testing → Algorithm → Function (Mapping)
Neighborhood in Abstraction Space¶
Linear-on-the-Fly Testing sits in a sparse region of the domain-specific corpus (99th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.
Family — Unclustered & Miscellaneous (2551 abstractions)
Nearest neighbors
- Flexural Strength — 0.75
- Operator-precedence grammar — 0.74
- Second Normal Form — 0.74
- Linearizability — 0.74
- En ventre sa mere — 0.73
Computed from structural-signature embeddings · 2026-10-08
Not to Be Confused With¶
A common fixed computer form, casual quiz randomization, or computerized adaptive testing. LOFT also does not guarantee zero repeated items, perfect score equivalence, fairness or prevention of item theft. Those stronger claims need evidence beyond the method's name.[ref-b5e57cbbbfe4][ref-e2e93358c9d5][^ref-a2ff4b3b6581]
References¶
[^ref-b5e57cbbbfe4]: National Council of Examiners for Engineering and Surveying, “Computer-Based Testing (CBT)”, “Year-Round and Single-Day Exams.” Official exam-operator description of LOFT form assembly, common topic counts and relative difficulty, with fixed-form contrast. [^ref-e2e93358c9d5]: g.a.s.t. / TestDaF-Institut, “The design of the onSET”, research page, “Calibrated Item Bank” and “Standardized Test Administration.” Official language-placement operator description of its C-test bank, topic/difficulty controls and LOFT task sets. [^ref-a2ff4b3b6581]: Pharmacy Examining Board of Canada, “Linear-on-the-Fly Testing for the Pharmacist Qualifying Examination – Part I (MCQ)”, 5 October 2023. Official operator description of automatic unique-form assembly and its intended security benefit.