Linear-on-the-Fly Testing¶

A testing method that assembles a candidate-specific nonadaptive exam form from an item bank under shared content and difficulty requirements.

Version: v1 · 2026-10-07 · History
Domain-specific #: 13930
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomains: Psychometrics, Test Form Assembly → Experimental Design & Statistics
Aliases: LOFT

Core Idea¶

Linear-on-the-fly testing (LOFT) assembles an exam form for each examinee from an item bank under a common content and difficulty specification. The assembled form is then delivered linearly: later questions are not chosen in response to that examinee's earlier answers. The different forms are intended to be comparable measures of the same target, although their questions differ. This joins candidate-specific form assembly to nonadaptive delivery.^[1]^[2]

NCEES uses the method for year-round computer-based engineering and surveying exams, while onSET uses it for German and English language placement. Both operators describe a different compiled set for each participant subject to content or topic and difficulty controls. A course platform that merely shuffles questions lacks the documented equivalence controls and is not thereby a LOFT implementation.^[1]^[2]

Structural Signature¶

Signature: assessed construct and common exam specification → eligible item bank → constrained candidate-specific assembly → comparable target forms → nonadaptive linear delivery.

Exam specification. The intended measurement and blueprint state what each form must cover. NCEES requires the same counts in the same topics; onSET controls topic and task difficulty. Without a shared specification, different forms need not measure the same construct in a comparable way.^[1]^[2]
Eligible item bank. Approved questions or tasks carry the information needed to build forms. onSET describes a calibrated C-test bank; neither operator publishes the full bank or a universal minimum size.^[2]
Assembly algorithm. An effective procedure selects a form for a candidate from the bank while respecting the specification. It is an internal part of the testing method, rather than the entire assessment or the learner's response process.^[1]^[2]
Comparability target. Forms are designed for similar content and overall difficulty. This is a required design constraint, not proof that every form is exactly equivalent in observed performance or fair for every subgroup.^[1]^[2]
Linear delivery. The candidate works through the assembled form without response-contingent selection of later items. This is the decisive contrast with computerized adaptive testing, which updates the next probe using earlier responses.
Exposure distribution. Distinct forms can make wholesale question sharing less useful. Operators cite security as a reason for LOFT, but the method does not by itself establish that cheating or item theft has been prevented.^[3]

What It Is Not¶

LOFT is not simply computer-based testing. NCEES explicitly contrasts its year-round LOFT exams with lower-volume single-day computer exams in which examinees receive the same questions. Nor is it ordinary question-bank randomization: distinct forms without a controlled content and difficulty blueprint need not be comparable.^[1]

LOFT is also not computerized adaptive testing. An adaptive test selects subsequent items using evidence from responses already given. LOFT assembles a linear form for the individual without that response feedback. Both may use item banks and constraints, so the distinction is the timing and dependency of item selection, not the presence of a computer.

Scope of Application¶

The method applies where a program has enough approved items and item information to assemble multiple forms meeting one test specification. A professional licensing exam and a language placement test are source-attested, unlike carriers: the former assesses technical qualifications; the latter places language learners using C-tests. The per-candidate assembly relation remains the same.^[1]^[2]

LOFT does not require that every candidate receive a globally disjoint set of items, that a particular item-response model be used, or that the security benefit be measured at a fixed size. onSET describes Rasch calibration for its own item bank; that detail should not be imputed to NCEES or PEBC without evidence. Comparability remains a claim requiring program-specific validation.^[2]^[3]

Clarity¶

Separate three questions: Does the form differ? Does it meet the same blueprint? Do scores support the intended comparison? A different question set answers only the first. The operator descriptions address the second through topic and difficulty controls; actual score comparability is an empirical matter for each program. This prevents the word “unique” from being mistaken for “equivalent.”^[1]^[2]

Also separate when personalization occurs. LOFT gives each candidate a form assembled for that administration, but the form is not personalized to the candidate's estimated ability from ongoing answers. An adaptive test can share the same calibrated bank while changing questions mid-test. This distinction identifies the method even when the screen interface looks identical.

Manages Complexity¶

A test program may have many eligible questions and a large number of possible forms. LOFT compresses the assembly problem into a controlled procedure: start with the bank, impose content and difficulty constraints, select an admissible form, and deliver it. The program can reason about form coverage and exposure through those constraints rather than hand-authoring every possible combination.^[1]^[2]

The same frame reveals a limit. If the bank has too few items in a required topic or difficulty range, the desired diversity may conflict with the blueprint. It is then the pool or specification that needs attention; merely increasing randomness does not repair an infeasible or poorly matched form. This is a structural implication of the documented design, not an operator-reported failure rate.

Abstract Reasoning¶

To evaluate a claimed LOFT implementation, identify the target construct, the common form requirements, and the eligible item bank. Ask how an algorithm assembles each candidate's form, whether the resulting forms satisfy the same constraints, and whether item choice changes after responses arrive. If the last answer is yes, the system may be adaptive; if the assembly has no equivalence controls, it may be randomization alone.^[1]^[2]

The method can also guide a design audit: tight constraints make forms comparable but may narrow the feasible combinations; more variation can distribute exposure but must still respect the blueprint. Actual performance, fairness, and security must be tested with program evidence rather than inferred from the label “LOFT.”

Knowledge Transfer¶

The literal method transfers from NCEES's engineering and surveying examinations to onSET's language placement: the content and scoring purposes differ, but both assemble a candidate-specific linear form under a shared specification. PEBC's pharmacist qualifying exam gives another documented professional application, not a reason to generalize the method to all electronic exams.^[1]^[2]^[3]

The broader Algorithm Prime captures the finite assembly procedure that LOFT contains. Algorithms also select objects under constraints in scheduling and design, but those uses are not themselves LOFT unless the objects are psychometric test forms with this common-specification and linear-delivery relation. The name should not migrate on the strength of algorithmic resemblance alone.

Examples¶

NCEES year-round engineering and surveying exams¶

NCEES says its year-round computer-based exams use a LOFT algorithm: examinees for a given exam receive the same number of questions in the same topics, while the forms differ and have a similar relative difficulty. Its lower-volume single-day computer exams provide a useful fixed-form contrast.^[1]

Mapped back: construct/specification → the declared exam and topic counts; bank → eligible licensure questions; algorithm → per-examinee assembly; comparability → similar relative difficulty; linear delivery → assembled question set rather than response-driven selection; exposure → different complete forms. NCEES's public account does not reveal bank size or empirical equivalence estimates.

onSET language placement¶

onSET's German and English placement tests use C-test task sets from a calibrated bank. The operator describes a newly compiled set for each participant, with topic and difficulty considered and overall task-set difficulty held comparable.^[2]

Mapped back: construct/specification → language placement and controlled C-test content; bank → calibrated C-test tasks; algorithm → newly compiled participant set; comparability → controlled overall difficulty; linear delivery → an assembled task set rather than response-adaptive selection; exposure → different sets across participants. The operator account establishes its design and reported procedure, not an independent outcome validation.

Structural Tensions¶

Form diversity versus comparability. Making forms differ spreads use of an item bank, while requiring each form to meet the same content and difficulty targets restricts allowable combinations. In a finite bank these demands cannot be increased without limit at the same time. Relaxing the blueprint may produce weaker score comparability; narrowing the feasible set may increase repeated exposure. The diagnostic question is whether the available bank supports the intended number of distinct forms while meeting every required constraint. This is an analytic design tension, not a published estimate of its size for NCEES or onSET.^[1]^[2]

Structural–Framed Character¶

LOFT is a structural procedure within a framed measurement practice. Evaluative weight: the assembly rule is technical, but claims of fairness, comparability and security depend on valued uses of test scores. Human-practice dependence: examinees, test operators and score decisions supply the reason forms matter. Institutional origin: the method is specified by testing programs, not by a naturally occurring object. Vocabulary travel: “on the fly” can describe any real-time generation; outside psychometrics it does not carry the LOFT identity. Import versus recognition: one recognizes LOFT by examining the bank, blueprint, assembly timing and linear delivery; calling an unrelated generator “LOFT” would import a testing frame. The portable skeleton is the Algorithm constituent, whose procedure can occur elsewhere without the psychometric commitments. Its character: a domain-bound testing method with a precise algorithmic core and institution-dependent validity goals.^[1]^[2]

Structural Core vs. Domain Accent¶

The skeletal relation is constrained per-candidate assembly followed by nonadaptive delivery. The form-assembly algorithm is a literal internal component, which is why the proposed DAG edge to Algorithm is composition / part_of, with the parent constituent inside the child method. The item bank, exam blueprint, difficulty targets, examinee and score use make the relation specifically psychometric; remove them and an on-demand schedule or generated worksheet may still use an algorithm but is not LOFT.^[1]^[2]

This named method does not meet the Prime bar by merely appearing in licensure and language placement. Those are two kinds of testing, and its defining form and scoring constraints remain domain-bound. The reusable cross-domain part belongs to live Algorithm; possible cross-domain “constrained assembly” is a separate comparison question, not evidence that LOFT itself has become substrate independent.

This entry is part of Algorithm.

Algorithm is the proposed direct typed parent: a definite form-assembly procedure is an identity-bearing part of LOFT, so the edge is strict composition / part_of in the child-to-parent record. Constraint describes the admissible-form conditions, but a second direct edge is unnecessary when the algorithmic parent proof already captures the core assembly relation. Neither edge says every algorithm or constrained procedure is a test.

Computerized adaptive testing is a domain-specific neighbor, not a parent: it uses response feedback to choose later items. Educational Assessment is broader than this delivery method and can include portfolios or observation, while some LOFT uses are professional licensure. High-Stakes Testing concerns the consequence attached to a score, not how a form is assembled.

Relationships to Other Abstractions¶

Current abstraction Linear-on-the-Fly Testing Domain-specific

Parents (1) — more general patterns this builds on

Linear-on-the-Fly Testing is part of Algorithm Prime

An assembly algorithm is an internal constituent of each LOFT administration.

Hierarchy paths (2) — routes to 2 parentless roots

Linear-on-the-Fly Testing → Algorithm → Function (Mapping)

Show alternative path (1)

Neighborhood in Abstraction Space¶

Linear-on-the-Fly Testing sits in a sparse region of the domain-specific corpus (99^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (2551 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-08

Not to Be Confused With¶

A fixed common computer exam, randomized quizzes with no controlled equivalence, adaptive item selection after each answer, guaranteed zero item overlap, or a demonstrated reduction in cheating. The first three are different delivery methods; the last two are stronger outcome claims than the operators' descriptions establish. A LOFT implementation needs its own score-validity, fairness and security evidence for its intended use.^[1]^[2]^[3]

References¶

[1] National Council of Examiners for Engineering and Surveying, “Computer-Based Testing (CBT)”, “Year-Round and Single-Day Exams.” Official exam-operator description of LOFT form assembly, common topic counts and relative difficulty, with fixed-form contrast. registry ↩a ↩b ↩c ↩d ↩e ↩f ↩g ↩h ↩i ↩j ↩k ↩l ↩m ↩n ↩o ↩p

[2] g.a.s.t. / TestDaF-Institut, “The design of the onSET”, research page, “Calibrated Item Bank” and “Standardized Test Administration.” Official language-placement operator description of its C-test bank, topic/difficulty controls and LOFT task sets. registry ↩a ↩b ↩c ↩d ↩e ↩f ↩g ↩h ↩i ↩j ↩k ↩l ↩m ↩n ↩o ↩p ↩q

[3] Pharmacy Examining Board of Canada, “Linear-on-the-Fly Testing for the Pharmacist Qualifying Examination – Part I (MCQ)”, 5 October 2023. Official operator description of automatic unique-form assembly and its intended security benefit. registry ↩a ↩b ↩c ↩d