LLM discovery and synthetic transfer training¶
Part of Inverse Innovation with the Encyclopedia of Abstractions · LLM discovery and synthetic transfer training · Last revised August 2026
Field review date: 2026-08-03
Scope: LLM analogy and far-transfer benchmarks; relational and compositional evaluation; cross-domain scientific ideation; automated research systems; ontology/knowledge-graph generation; synthetic reasoning data; process supervision; self-distillation; contrastive and hard-negative training; held-out-domain evaluation.
Claim status key: Established = directly shown by the cited study; supported interpretation = synthesis across studies; provisional gap = not found in this scoped search, not proof of absence.
Executive conclusion¶
The broad claim that no one trains scientific-discovery models on decomposed problem-to-solution trajectories is no longer tenable. MOOSE-Star (Yang and Bing, ICML 2026) trains 7B models on 108,717 paper-derived decompositions of research background → sequential inspiration retrieval → hypothesis composition. Each hypothesis increment is represented as motivation, mechanism, and methodology. Its strict October 2025 temporal holdout, hard-negative retrieval experiments, and decoy-cluster ablation make it the strongest discovered training precedent for decomposition-based scientific discovery.
It is not, however, a demonstration of the Encyclopedia of Abstractions (EoA) transfer thesis. The papers and their cited inspirations are drawn largely from a bio-medical scientific corpus; the intermediate object is a paper-level “inspiration,” not a stable, reusable solution archetype; and the evaluation does not hold out target domains, archetypes, source–target domain pairings, or relation compositions. The task is retrospective reconstruction from a later paper and its citation structure, not controlled transfer of a mechanism from an unrelated source domain to a target problem.
Two other 2026 systems occupy the adjacent cells that MOOSE-Star leaves open:
- RLAD (ICLR 2026) trains a literal problem → natural-language reasoning abstraction → solution architecture and rewards abstractions by the improvement they induce in an abstraction-conditioned solver. Its evidence is on mathematical reasoning, not cross-domain scientific transfer.
- Unlocking LLM Creativity in Science through Analogical Reasoning (Shen, Druckmann, and Zou, May 2026 preprint) explicitly extracts objects and relations, constructs cross-domain analogies, searches the analogous domain for a solution, and maps that solution back. It reports large diversity/novelty gains and implements proposals on four biomedical problems, but the method is prompting and inference, not synthetic trajectory training.
The defensible answer to the focal question is therefore:
Supported interpretation: Work now exists on (a) synthetic training of decomposed scientific-discovery trajectories, (b) training abstraction-conditioned solvers, and © explicit cross-domain scientific solution transfer. Provisional gap: this review found no system that combines all three by training reusable, domain-general solution archetypes on explicit source-mechanism → unrelated-target adaptation trajectories and then demonstrating transfer under leave-domain-out or leave-archetype/domain-combination-out evaluation.
This is a narrow claim boundary, not a claim of historical priority. The literature is moving quickly, terminology is fragmented, and several 2026 items are preprints or workshop papers.
Benchmark and result table¶
| Source | Task and strongest relevant result | What the control supports | What it does not establish |
|---|---|---|---|
| MOOSE-Star (ICML 2026) | TOMATO-Star contains 108,717 paper decompositions across biology, chemistry, medicine, medical imaging, psychology, and cognitive science. Inspiration retrieval rose from 28.42% for the base 7B model to 54.37% after task training; the unified model’s hypothesis-composition M3 score rose from 4.05/12 to 5.02/12. | October 2025 papers were excluded from both post-training and the stated base-model pretraining period. With 14 hard/random negatives, all-hard accuracy was 53.42%; a misleading decoy cluster produced 77.65%, close to the all-random 78.58%, arguing against a simple cluster shortcut. | No held-out scientific domain, archetype, or source–target pair. Ground truth is reconstructed from future papers and citations; prospective novelty and independent utility are not tested. |
| Unlocking LLM Creativity in Science through Analogical Reasoning (2026 preprint) | Explicit relational mapping and cross-domain solution transfer improved solution-diversity measures by 90–173%, generated 3.5× as many unique domains, and exceeded 50% judged novelty for all three tested models. Human reviewers preferred AR novelty 78% of the time and judged 67% of sampled proposals reasonable. Four biomedical implementations produced quantitative gains, including nearly 13× on one distributional metric and state of the art on two oligonucleotide datasets. | Comparison with cross-domain-without-mapping and no-domain baselines isolates the value of structural analogy better than an unconstrained ideation comparison. Literature search and human review partially check novelty. | Prompting only; 50 sampled problems; biomedical targets; no training split or leave-domain-out test; novelty relies substantially on search plus model judging; no systematic archetype × domain matrix or negative terminal outcomes. |
| RLAD (ICLR 2026) | Jointly trains an abstraction proposer and abstraction-conditioned solver; abstractions are rewarded by downstream solution success. Reports an average 44% improvement over the cited long-CoT RL baseline on AIME 2025 and benefits from allocating test compute to abstraction diversity. | Decoupling abstraction search from solution search shows that a learned intermediate natural-language variable can improve hard-problem generalization. | Math only. Its “abstraction” is a procedural hint for the same problem, not a reusable cross-domain mechanism mapped from a source solution. No domain holdout or scientific novelty evaluation. |
| ResearchBench (Findings ACL 2026) | Decomposes 1,386 recent papers from 12 disciplines into inspiration retrieval, hypothesis composition, and ranking. GPT-4o found a ground-truth inspiration within the top 4% of candidates in 45.7% of cases. | Papers published in 2024 or later and a stricter post-July-2024 subset reduce contamination risk. Expert review found 91.9% of decompositions free of major errors. Mixed fake/model negatives and order reversal expose ranking weaknesses. | It is a benchmark, not a trained transfer curriculum. Temporal novelty is not cross-domain structural novelty; ranking suffers marked position bias. |
| MOOSE-Chem (ICLR 2025) | Reconstructs hypotheses from 51 chemistry papers published in 2024 using expert-annotated backgrounds and inspirations, after the tested models’ stated cutoff. | Strong temporal control and doctoral-level annotation make it useful for unseen-paper reconstruction. | One domain, small sample, paper inspirations rather than an ontology of reusable abstractions; not solution-to-problem transfer. |
| MOOSE-Chem2 (NeurIPS 2025) | Uses hierarchical heuristic search to turn a coarse research direction into fine-grained, actionable chemistry hypotheses evaluated against recent expert-annotated work. | Shows that decomposed search can make open-ended hypothesis generation more operational. | Inference/search rather than a domain-general transfer curriculum; chemistry only. |
| SCAR (Findings EMNLP 2023) | 400 scientific analogies from 13 fields require abducting the shared relational structure. ChatGPT and GPT-4 remained challenged. | Directly distinguishes structural explanation from surface word analogy. | Recognition/abduction, not source-solution adaptation or training; small curated benchmark. |
| ARN (TACL 2024) | Separates near and far narrative analogies and disanalogies. GPT-4 was below random in zero-shot far-analogy evaluation; few-shot chain-of-thought recovered only about half the human–random gap. | Far/near and disanalogy controls reveal that ordinary benchmark success does not imply robust remote structural mapping. | Narrative judgment, not scientific ideation or adaptation. |
| StoryAnalogy (EMNLP 2023) | 24,000 story pairs with structure-mapping annotations. ChatGPT achieved roughly 30% multiple-choice accuracy versus over 85% for people; fine-tuned Flan-T5-XXL was comparable to zero-shot ChatGPT. | Large supervised data can train analogy recognition, while the human gap remains large. | Identifies/generates story analogies; does not transfer source solutions or test remote scientific domains. |
| AnaloBench (EMNLP 2024) | 340 human-authored analogies; scaling yielded little improvement on long scenarios and analogy retrieval from long contexts remained difficult. | Human-authored long-form cases and retrieval settings reduce dependence on templated word analogies. | No adaptation or training curriculum; contamination cannot be eliminated for proprietary models. |
| ParallelPARC / ProPara-Logy (NAACL 2024) | LLM pipeline generates paragraph-level analogies between scientific processes plus easy and hard distractors; silver data improves trained models, but humans retain about a 13-point advantage after light supervision. | Human-validated gold data and structurally plausible hard negatives directly test shortcut resistance. | Analogy recognition, not problem formulation or solution transfer; generator/judge correlations remain possible in the silver set. |
| ANALOGYKB (ACL 2024) | More than one million analogies spanning 943 relations and 103 analogous relation pairs, built from Wikidata/ConceptNet with LLM filtering and human review; training improves recognition and generation. | Held-out relations/pairs can test some relational generalization, and KG provenance makes structures inspectable. | Mostly compact A:B::C:D relations, not multi-entity scientific mechanisms, adaptation plans, or empirical hypotheses. |
| Robustness of Analogical Reasoning (TMLR 2025) | Performance on letter strings, digit matrices, and story analogies drops sharply under counterfactual variants; answer order and paraphrase matter. | Counterfactual task families probe whether success survives departure from likely pretraining templates. | Does not show that every success is memorization; task mutations can also increase intrinsic difficulty. |
| Counterfactual tasks support emergent analogy (PNAS Nexus 2025) | GPT-4 with code use recovered substantial performance when familiar symbolic conventions such as alphabet order were changed. | A valuable contrary result: sufficiently capable models/tools can apply rules under some counterfactual remappings. | Success on compact formal tasks is not evidence of open-ended far transfer or novel science. |
| Emergent Analogical Reasoning in Transformers (LIT @ ICLR 2026) | On controlled synthetic categories, analogy emerges through learned relational geometry and functor-like mappings; results are sensitive to relational diversity, optimization, scale, and OOD ratio. | Clean held-out entities/relations and mechanistic analysis provide unusually direct evidence of learned structural transfer. | Toy synthetic worlds; no language-grounded scientific problem, novelty search, or real-domain adaptation. |
| Analogical Deep Research (2026 preprint/workshop version) | ADR-bench covers 15 historical/forward-looking events. CANA decomposes events into preconditions, temporal chains, mechanisms, and outcomes and improves historical analogy generation by up to 10%. | Mechanism alignment and cross-analogy confirmation explicitly target surface-match failures. | Very small benchmark; foresight rather than scientific solution transfer; no synthetic training. |
| ScienceAgentBench (ICLR 2025) | 102 executable research tasks from 44 papers in four disciplines, curated by nine experts. The best agent completed only 32.4% independently and 34.3% with expert-provided knowledge. | Executability and paper-grounded tasks test actual research workflows rather than prose plausibility. | Evaluates implementation agents, not analogy specifically. |
| DiscoveryBench (ICLR 2025) | 264 real-world and 903 synthetic data-driven discovery tasks; the best reported system solved only 25%. | Real/synthetic pairing and objective data-analysis outputs reveal a large gap between fluent hypotheses and reliable discovery. | Focuses on data-driven discoveries rather than cross-domain invention. |
| PreScience (2026 preprint) | Temporal forecasting over 98,000 papers; GPT-5 contribution generation scored 5.6/10, and a synthetic 12-month literature corpus was less diverse and novel than the human corpus. | Temporal prediction constrains retrospective leakage and compares AI output with what scientists later published. | Automated/literature-based judges remain imperfect; forecasted paper contributions are not experimental validation. |
Nearest systems¶
1. MOOSE-Star: nearest training precedent¶
MOOSE-Star formalizes a hypothesis as \(h=f(b,i_1,\ldots,i_k)\), where a research background is successively combined with retrieved inspirations. It separately trains inspiration retrieval and bounded hypothesis composition, and it represents each target increment with three fields that overlap strongly with an EoA trajectory:
- Motivation: why the inspiration addresses the current gap.
- Mechanism: why the transfer should work.
- Methodology: how it would be implemented.
This is not merely chain-of-thought distillation. Its 108,717 examples are derived from open-access papers and their historical citations, processed by DeepSeek-R1 and R1-Distilled-Qwen-32B, quality-checked for necessity, sufficiency, background disjointness, and nonredundancy, then used to train retrieval and composition models. Training includes one positive and 14 negatives per retrieval instance, including keyword- and embedding-similar hard negatives; composition examples include bounded semantic perturbations around the exact inspiration.
The decisive difference is the latent variable. An EoA prime is meant to be a named, reusable, domain-general solution structure with roles, mechanisms, constraints, and failure modes. MOOSE-Star’s inspiration is normally a cited paper or paper fragment selected because the later paper in fact used it. Motivation is generated per paper; it is not an independently curated archetype that recurs across unrelated source/target domains. Its temporal split supports unseen-literature reconstruction, but not systematic far transfer.
2. Shen–Druckmann–Zou: nearest cross-domain transfer procedure¶
This method most closely resembles the desired operation:
- Parse a research problem into objects and relations.
- Generate a problem in another domain with explicit object correspondences and shared relations.
- search the analogous domain for a candidate solution.
- Translate the solution back into the target domain.
Unlike generic “be creative” prompting, the approach makes relational mapping explicit and compares against both a generic cross-domain baseline and a no-domain baseline. Its four implementations are stronger evidence than judge scores alone. Yet it does not learn or reuse a curated abstraction ontology, does not train from trajectories, and generates one mapped solution per analogy rather than traversing a systematic abstraction-by-domain grid.
3. RLAD: nearest learned problem→abstraction→solution architecture¶
RLAD uses two policies: an abstraction generator \(\pi_{abs}(z\mid x)\) and a solution generator \(\pi_{sol}(y\mid x,z)\). Stronger models first produce warm-start abstractions by summarizing multiple candidate solutions; reinforcement learning then rewards an abstraction according to the conditional solver’s success. This is an important design precedent for treating abstractions as trained intermediate actions rather than decorative explanations.
But RLAD’s abstractions are variable-length mathematical hints—procedures or facts useful for the current problem. The reported generalization is to harder math problems, not to a new scientific domain, and there is no source-domain solution whose mechanism is explicitly aligned to a different target. It supports the architecture of EoA training, not the scientific transfer claim.
4. Ontology and knowledge-graph systems: structure without the full transfer loop¶
- SciAgents combines a large ontological knowledge graph with agents including an ontologist, scientists, and critics to generate and refine biologically inspired materials hypotheses. It offers inspectable knowledge structure and detailed cases, but only a handful of demonstrations and no controlled held-out-domain benchmark or post-training curriculum.
- KG-CoI grounds hypothesis generation in a scientific KG and adds a hallucination detector. Its masked-link evaluation is valuable for relation recovery, but recovering hidden KG edges is closer to link prediction than to proposing a new cross-domain solution.
- ANALOGYKB supplies a large relational analogy resource and training signal, but its atoms are compact relation pairs rather than causal, multi-role mechanisms with constraints and failure modes.
These systems show why “uses an ontology/KG” is too weak a differentiator. The relevant question is whether the ontology is the generative pivot for mapping, adaptation, rejection, and out-of-domain evaluation, not merely retrieval or grounding.
5. Automated research systems: breadth without an analogy-specific curriculum¶
The AI co-scientist uses generation, debate, evolution, ranking, proximity, and meta-review agents and reports initial experimental/genomic validation. MOOSE, MOOSE-Chem, and MOOSE-Chem2 progressively decompose literature-grounded hypothesis search. These are important comparators for proposal quality, refinement, and test-time scaling. They do not center reusable far-domain archetypes, and most gains come from inference orchestration, retrieval, feedback, or search rather than a synthetic cross-domain transfer curriculum.
Synthetic-curriculum precedents¶
The training literature supplies components of an EoA curriculum, but no discovered work combines them all.
| Precedent | Reusable lesson for EoA | Boundary |
|---|---|---|
| MOOSE-Star / TOMATO-Star | Distill long discovery records into supervised retrieval and composition transitions; represent why/why-it-works/how; train with noisy inspirations and hard negatives. | Paper hindsight and citations substitute for independently verified transfer; no domain/archetype holdout. |
| RLAD | Reward an abstraction by its causal usefulness to a downstream solver; train abstraction proposal and application separately; sample multiple abstractions. | Same-domain math reasoning; no source mechanism or scientific novelty. |
| Agentic Analogical Reasoning (ICLR 2026 submission) | Generate multi-turn analogical trajectories—query, analogical instance, contextualized insight—and reweight them during training; mix self-generated and externally retrieved analogies. | Seven standard reasoning datasets, not scientific discovery; no demonstrated held-out-domain transfer; submission status warrants caution. |
| ParallelPARC | Generate silver analogies plus easy and hard distractors, retain a human-validated gold set, and train recognition models. | Recognition rather than adaptation or proposal generation. |
| STaR | Iteratively generate rationales, use answers to rationalize failures, retain successful trajectories, and fine-tune. | Outcome-gated rationales can be post hoc and are not proof of faithful abstraction use. |
| Let’s Verify Step by Step / PRM800K | Supervise intermediate decisions rather than only final answers; reported 78.2% on the targeted MATH subset and 2.6× annotation efficiency from active learning. | Human-labeled mathematics; intermediate correctness is easier to define than scientific usefulness. |
| Can Models Learn Skill Composition from Examples? (NeurIPS 2024) | Training on synthetic two- and three-skill combinations improves four- and five-skill tasks and transfers to held-out skill categories: a useful compositional-split template. | Skills are benchmark operations, not causal scientific mechanisms or target domains. |
| CHIMERA (2026 preprint) | A model-generated taxonomy and cross-model validation can produce about 9,000 long scientific reasoning traces across eight disciplines and more than 1,000 topics. | Answers existing questions; automated generator/verifier dependence can preserve shared errors; no analogical adaptation. |
| Emergent Analogical Reasoning in Transformers | Control relational diversity, OOD ratio, entity overlap, and mapping structure in a synthetic world; inspect whether a reusable relational operator emerges. | Mechanistic clarity comes at the cost of ecological validity. |
The most promising synthesis is not to copy any one curriculum. It is to combine MOOSE-Star’s decomposed scientific transitions, RLAD’s downstream-utility reward, ParallelPARC’s hard structural distractors, PRM-style transition labels, and controlled compositional/domain splits from synthetic systematicity research.
Evidence against easy claims¶
“Strong scores on analogy tests show genuine far transfer.”¶
They do not. Webb, Holyoak, and Lu’s emergent analogy study found GPT-3 matched or exceeded people on several compact analogy tasks, but later tests show brittleness. Lewis and Mitchell report large degradation under counterfactual variations, while ARN finds GPT-4 below random on zero-shot far narrative analogies. AnaloBench and SCAR likewise show difficulty with long contexts and explicit scientific structure. The PNAS Nexus counterfactual follow-up is important counterevidence—GPT-4 with code can recover on some remapped formal tasks—so the conclusion should be conditional capacity plus substantial brittleness, not “mere memorization” or “human-level analogy.”
“An analogy prompt improves answers, therefore the model used the analogy.”¶
Relevant or Random shows that self-generated random examples can sometimes match or outperform relevant examples, including an approximately four-point GSM8K gain from random biological examples. Gains can arise from extra correct context, deliberation, format induction, or sampling diversity. An EoA experiment therefore needs mediation tests: corrupt the mapping while preserving length and topicality; replace the prime with a surface-similar but structurally wrong prime; and test whether performance follows structural correctness.
“Temporal holdout proves abstraction-mediated transfer.”¶
Temporal holdout is necessary for fast-moving literature tasks, but it tests only one axis. MOOSE-Star and MOOSE-Chem convincingly reduce direct paper contamination, yet the model may still reuse familiar domains, mechanisms, terminology, citation neighborhoods, and templates. Genuine EoA-style transfer calls for simultaneous controls over time, target domain, archetype, source–target pairing, and relation composition.
“A paper-derived trajectory is a faithful discovery trajectory.”¶
Published papers are selected successes written with hindsight. Citation choice is incomplete and rhetorical; decomposers can leak conclusions into motivations; many failed inspirations are absent. MOOSE-Star explicitly checks background disjointness and other quality criteria, which is a strong safeguard, but the trajectory is still a reconstruction. A useful curriculum needs failed mappings, duplications, infeasible proposals, misleading surface matches, and terminal classifications—not only paths that end at a publication.
“LLM judges and prior-art search establish novelty.”¶
They are screening mechanisms, not final evidence. Search coverage depends on queries and indexed sources; embedding rerankers can miss terminological distance; LLM judges exhibit position and self-preference biases. ResearchBench reports extreme order sensitivity in some rankers. Human implementation in Shen et al. and external experimental/genomic evidence in the co-scientist paper are stronger, but still narrow. Novelty, feasibility, adopter fit, and empirical validity should be reported separately.
“Fluent research agents are close to autonomous discovery.”¶
ScienceAgentBench’s 32.4% independent success, DiscoveryBench’s 25% best-system result, and PreScience’s 5.6/10 GPT-5 contribution score show a large execution and forecasting gap. Can LLMs Generate Novel Research Ideas? found LLM-generated NLP ideas rated more novel than human ideas in a blind study, but also weaker feasibility tendencies, poor self-evaluation, limited diversity, and no evaluation of execution. Claims should distinguish proposal fluency, judged novelty, reproducible implementation, and scientific discovery.
Implications for the Encyclopedia of Abstractions¶
-
Frame the contribution as a combination and evaluation regime. The components—structured analogies, abstraction-conditioned solving, literature-derived discovery traces, KGs, critics, and hard negatives—all have precedents. The plausible gap is their integration around a curated, reusable archetype ontology with systematic distant-domain traversal, explicit rejection states, prior-art scrutiny, and out-of-domain tests.
-
Define “prime” more strongly than “inspiration” or “hint.” A prime should have stable identity across instances: roles, causal mechanism, preconditions, invariants, adaptation operators, predictable failure modes, and positive and negative exemplars. This makes it testable whether the same abstraction was reused rather than regenerated as a convenient paraphrase.
-
Preserve directionality. Most nearby scientific systems go problem → inspiration → hypothesis. The EoA inverse-innovation program also asks solution structure → candidate problems → adapted solution. That solution-to-problem direction, systematically repeated, is a more defensible differentiator than generic cross-domain ideation.
-
Treat the ontology as an intervention. Compare no prime, retrieved prose, name-only prime, full prime, corrupted prime, and an equally long unrelated prime. If the full prime improves proposals and corrupted structure reverses the benefit, the case for abstraction mediation is much stronger.
-
Train on the full disposition space. Include
promising,duplicate/prior art,structurally invalid,mechanistically unsupported,infeasible,no plausible adopter, andinsufficient evidence. Published-success-only data will teach persuasive completion, not disciplined invention. -
Separate proposal metrics. Score structural fidelity, target-domain validity, prior-art distance, mechanism plausibility, implementation specificity, falsifiability, cost, adopter value, and eventual empirical outcome. Do not collapse them into one “novelty/quality” judge score.
-
Do not use “first.” A safer formulation is: “Within this cross-disciplinary search through 2026-08-03, we found close precedents for each major component but no evaluated system combining an explicit reusable solution-archetype ontology, synthetic cross-domain transfer training, systematic solution-to-problem traversal, and held-out-domain validation.”
Experimental recommendations¶
Minimum credible dataset unit¶
Store each trajectory as an inspectable graph rather than free-form chain of thought:
target problem → selected prime → source exemplar(s) → role alignment → invariant/mechanism → adaptation operators → proposal → prior-art evidence → critic objections → revision → disposition → falsifiable next test
Record provenance and timestamps for every external claim. Keep concise public rationales/structured state rather than relying on unverifiable hidden chain of thought.
Required split suite¶
- Chronological split: all target evidence and prior art after a frozen date; audit base-model cutoff where knowable.
- Leave-target-domain-out: no training trajectory from the evaluation target discipline.
- Leave-prime-out: test whether a model can infer or apply an unseen archetype from its formal definition and examples.
- Leave-prime × domain-pair-out: both parts may occur in training, but never together. This is the central compositional test.
- Leave-source–target pair-out: block repeated lexical/citation corridors between the same fields.
- Relation-combination-out: withhold particular multi-role mechanism compositions while retaining their individual relations.
- Entity-disjoint adversarial split: rename entities and remove distinctive surface terms while preserving relational structure.
- Prospective locked box: preregister prompts, candidates, and scoring before new literature or adopter outcomes appear.
Report in-domain, temporal-OOD, domain-OOD, and compositional-OOD separately. A single aggregate “unseen” score obscures the claim.
Baselines and ablations¶
- Base model with the same token/tool budget.
- Generic creativity prompt.
- Random-domain and random-prime prompts.
- Surface-similar retrieval without structural mapping.
- Nearest-neighbor source exemplar without a prime.
- Shen-style explicit analogical reasoning.
- MOOSE-style inspiration retrieval/composition where scientifically applicable.
- Full EoA prime, name-only prime, description-only prime, and corrupted-role prime.
- Oracle-prime versus learned-prime retrieval to separate retrieval from adaptation.
- No critic, no prior-art search, no adopter analysis, and no negative-data training.
Hard negatives and anti-memorization controls¶
Construct negatives that match vocabulary, domain, entities, and apparent goal while violating a causal role, precondition, direction, conservation constraint, or implementation requirement. Add decoy clusters modeled after MOOSE-Star. Test paraphrases, counterfactual conventions, and blinded entity renaming. Check whether the model selects the same prime under paraphrase and rejects it when one load-bearing relation is inverted.
Run nearest-neighbor and phrase-overlap audits across pretraining-accessible corpora where possible; search exact proposal phrases and mechanism combinations; report suspected contamination rather than silently deleting it. For generated training data, hold the generator family out of evaluation judging and manually adjudicate a stratified sample.
Supervision design¶
- Use process labels at graph transitions: correct mapping, unsupported leap, missing constraint, wrong direction, prior-art duplicate, and non-actionable adaptation.
- Reward abstractions for incremental downstream benefit, following RLAD, but penalize answer leakage and prime-specific boilerplate.
- Contrast a correct prime with topically similar wrong primes and structurally similar distant primes.
- Retain revisions and rejected branches; do not train only on final polished proposals.
- Balance primes and domains so high-frequency archetypes cannot dominate by prior probability.
Evaluation hierarchy¶
- Controlled synthetic: exact role mappings, relation combinations, and counterfactual distractors; objective scoring.
- Retrospective real-world: pre-cutoff corpus only, recover later documented transfers, with citation and phrase leakage audits.
- Expert blinded review: separate ratings for structure, mechanism, novelty, feasibility, and falsifiability; measure agreement.
- Prior-art replication: independent searcher attempts to disprove novelty.
- Adopter review: a practitioner states whether the problem and proposed intervention matter under realistic constraints.
- Prospective test: preregister and implement a tractable subset; report nulls and costs.
A convincing result would show that full primes outperform same-budget prose retrieval and random/corrupted primes on the leave-prime × domain-pair split, retain the advantage after prior-art filtering, and yield a higher rate of expert-endorsed falsifiable tests—not merely more diverse text.
Limitations of this field review¶
- This was a targeted cross-disciplinary scoping search, not an exhaustive systematic review or citation-index export. Terminology spans analogy, discovery, invention, creativity, abstraction learning, KGs, compositionality, and domain generalization, so relevant work can evade keyword searches.
- The cutoff is 2026-08-03. Several central items appeared in 2026; status and results may change. Preprints and workshop/submission items are labeled and should not be weighted like archival peer-reviewed studies.
- Some numerical results are author-reported and rely on model judges. This memo records their evaluation design but does not independently reproduce results.
- Proprietary-model training corpora are unknown; temporal publication dates reduce but do not eliminate contamination.
- “Cross-domain,” “inspiration,” “abstraction,” and “novelty” are not standardized. The claim boundary above uses the stricter EoA meanings: an explicit reusable mechanism, a semantically distant target, operational adaptation, and evaluation that rules out familiar domain/pair templates.
- Absence from this search is evidence only for a provisional gap. It is not evidence that no earlier prototype, patent system, unpublished industrial system, or differently named method exists.
Linked bibliography¶
Scientific ideation and discovery systems¶
- Yang, Z. and Bing, L. (2026). MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier. ICML 2026.
- Shen, A., Druckmann, S., and Zou, J. (2026). Unlocking LLM Creativity in Science through Analogical Reasoning. Preprint.
- Yang, Z. et al. (2025). MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses. ICLR 2025.
- Liu, Y. et al. (2025). MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery through Hierarchical Search. NeurIPS 2025.
- Yang, Z. et al. (2024). MOOSE: A Scalable, Open Domain Hypothesis Discovery System Using LLMs. Findings ACL 2024.
- Liu, Y. et al. (2026). ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decomposition. Findings ACL 2026.
- Gottweis, J. et al. (2026). Towards an AI co-scientist. Nature.
- Ghafarollahi, A. and Buehler, M. J. (2024). SciAgents: Automating Scientific Discovery through Multi-Agent Intelligent Graph Reasoning. Preprint.
- Qi, B. et al. (2024). Knowledge Graph Chain-of-Ideas for Scientific Hypothesis Generation. Preprint.
- Si, C. et al. (2025). Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers. ICLR 2025.
- Skouras, P. and Manikis, G. C. (2026). Cross-Domain Analogical Reasoning via Structural Logic Transfer in Multi-Agent Scientific Discovery Systems. AAAI 2026 Bridge workshop; single-case evidence.
Analogy and relational benchmarks¶
- Webb, T. W., Holyoak, K. J., and Lu, H. (2023). Emergent analogical reasoning in large language models. Nature Human Behaviour.
- Lewis, M. and Mitchell, M. (2025). Evaluating the Robustness of Analogical Reasoning in Large Language Models. TMLR.
- Webb, T. W. et al. (2025). Evidence from counterfactual tasks supports emergent analogical reasoning in large language models. PNAS Nexus.
- Ji, A.-A. et al. (2024). Analogical Reasoning on Narratives. TACL.
- Cheng, J. et al. (2023). StoryAnalogy: Deriving Story-Level Analogies from Large Language Models to Unlock Analogical Understanding. EMNLP 2023.
- Ye, X. et al. (2024). AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies. EMNLP 2024.
- Yuan, S. et al. (2023). Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction. Findings EMNLP 2023.
- Sultan, O. et al. (2024). ParallelPARC: A Scalable Pipeline for Generating Natural-Language Analogies. NAACL 2024.
- Yuan, S. et al. (2024). ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base. ACL 2024.
- Chen, Y. et al. (2026). Analogical Deep Research: Retrieving and Integrating Historical Analogies for Foresight Analysis. Preprint / Forecasting workshop version.
- Minegishi, G. et al. (2026). Emergent Analogical Reasoning in Transformers. LIT @ ICLR 2026.
- Qin, C. et al. (2025). Relevant or Random: Can LLMs Truly Perform Analogical Reasoning?. Findings ACL 2025.
- Zhu, L. et al. (2026). Human analogical guidance amplifies LLM performance in solving historical scientific problems. Nature Communications.
Synthetic reasoning and evaluation¶
- Qu, Y. et al. (2026). RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems. ICLR 2026.
- Ma, T. et al. (2026). Agentic Analogical Reasoning for Large Language Models. ICLR 2026 submission; review status should be checked before citation as archival work.
- Zelikman, E. et al. (2022). STaR: Bootstrapping Reasoning With Reasoning. NeurIPS 2022.
- Lightman, H. et al. (2023). Let’s Verify Step by Step. Preprint / OpenAI report.
- Zhang, Y. et al. (2024). Can Models Learn Skill Composition from Examples?. NeurIPS 2024.
- Zhu, X. et al. (2026). CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning. Preprint.
- Pelaez, E. A. et al. (2025). ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery. ICLR 2025.
- Majumder, B. P. et al. (2025). DiscoveryBench: Towards Data-Driven Discovery with Large Language Models. ICLR 2025.
- PreScience: Evaluating and Improving Scientific Idea Generation with Temporal Forecasting (2026). Preprint.