Combinatorial Meta-Analysis¶

Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research).

Version: v1 · 2026-09-28 · History
Domain-specific #: 8559
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomain: Meta Analysis → Experimental Design & Statistics

Core Idea¶

Combinatorial Meta-Analysis is treated here as the recurring formal models and representations identity summarized by this source-grounded definition: Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research).

Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research). In an article that develops the notion of "gravity" in the context of meta-analysis, Travis Gee proposed that the jackknife methods applied to meta-analysis in that article could be extended to examine all possible combinations of studies (where practical) or random subsets of studies (where the combinatorics of the situation made it computationally infeasible). A useful tool developed by Gee was the "PPES" plot (standing for "Probability of Positive Effect Size", assuming differences are scaled such that larger in a positive direction is desired).

For each subset of combinations, where studies are taken j = 1, 2, ... k − 1, k at a time, the proportion of results that show a positive effect size (either WMD or SMD will work) is taken, and this is plotted against j. In the original article, k objects (studies) are combined k-1 at a time (jackknife estimation), resulting in k estimates. It is observed that this is a special case of the more general approach of CMA which computes results for k studies taken 1, 2, 3 ... k − 1, k at a time.

For Combinatorial Meta-Analysis, the abstraction is narrower than the article's general subject matter: a positive case must preserve Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research). Retaining only the name, a familiar example, or a downstream effect is insufficient. The specialist roles and tests remain anchored in formal models and representations, which is why this identity is domain-specific rather than prime.

How would you explain it like I'm…

Trying Every Study Group

Scientists sometimes gather lots of studies that asked the same question. Combinatorial Meta-Analysis means picking different groups of those studies, one at a time, two at a time and so on, and checking the answer for each group. Then they see how often the answer comes out the same way, to tell whether it depends on which studies you pick.

Trying Every Group of Studies

A meta-analysis combines the results of many studies on the same question to get an overall answer. Combinatorial Meta-Analysis goes further: it looks at the results for many different combinations of those studies, taking them one at a time, two at a time, and so on up to all of them. If there are too many combinations, it uses random groups instead. One handy picture is the PPES plot, which shows, for each group size, what fraction of the combinations point to a positive effect. This helps show how much the overall conclusion depends on which studies happen to be included.

Subset-Combination Meta-Analysis

Combinatorial Meta-Analysis (CMA) studies how the statistical results of a meta-analysis behave across different combinations of the studies in its dataset, typically in social science research. Travis Gee proposed extending jackknife methods—where each study is left out in turn—to examine all possible combinations of studies, or random subsets when there are too many combinations to compute. For each subset size j, from 1 up to all k studies, CMA finds the proportion of combinations showing a positive effect size. Gee's PPES ('Probability of Positive Effect Size') plot graphs this proportion against j, assuming effects are scaled so that positive is the desired direction. The jackknife, which combines k studies k − 1 at a time, is a special case of this broader approach.

Combinatorial Meta-Analysis (CMA) is the study of how statistical properties of a meta-analytic result behave across combinations of the constituent studies, typically in social science research. Travis Gee, in work developing a notion of 'gravity' in meta-analysis, proposed generalizing jackknife methods to all possible combinations of studies where computationally practical, or to random subsets where the combinatorics make exhaustive enumeration infeasible. For each subset size j = 1, …, k, the proportion of combinations yielding a positive effect size (using either weighted mean difference or standardized mean difference) is computed and plotted against j in a PPES ('Probability of Positive Effect Size') plot, with effects signed so the positive direction is desired. The classical jackknife, combining k studies k − 1 at a time to produce k estimates, is thus a special case of CMA. The defining feature is systematic evaluation of meta-analytic statistics over combinations of studies.

Structural Signature¶

Sig role-phrases:

Defining carrier — This differs from the standard approach in meta-analysis of adopting a single method and computing a single result, and allows significant triangulation to occur, by computing different indices for each combination and examining whether they all tell the same story.
Constitutive relation — CMA can thus be used as a data mining method to identify the number of intercepts that may be present in the dataset by looking at which studies are included in the local minima that may be obtained through recombination.
Operating condition — A useful tool developed by Gee was the "PPES" plot (standing for "Probability of Positive Effect Size", assuming differences are scaled such that larger in a positive direction is desired).
Recognition evidence — It is also possible through CMA to examine the relationship of covariates with effect sizes.
Admissible variation — For example, if industry funding is suspected as a source of bias, then the proportion of studies in a given subset that were industry funded can be computed and plotted directly against the effect size estimate.
Characteristic consequence — More recent work by a software porting team at Brown University has implemented the concept in STATA.
Failure boundary — The most widely used implementation of this idea is the Graphical Display of Study Heterogeneity (GOSH) plot, introduced by Olkin, Dahabreh, and Trikalinos in 2012.

What It Is Not¶

Not the whole field of formal models and representations. The node requires the specific identity stated by Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research).
Not an over-broad reading. CMA makes it possible to study the relative behaviour of different statistics under combinatorial conditions.
Not an over-broad reading. This differs from the standard approach in meta-analysis of adopting a single method and computing a single result, and allows significant triangulation to occur, by computing different indices for each combination and examining whether they all tell the same story.
Not an over-broad reading. While this software was shared with colleagues informally, it was not published.
Not automatically Canonical correlation. Retrieval proximity does not establish equivalence; the two identities must be compared by carrier, operation, and failure boundary.

Scope of Application¶

Combinatorial Meta-Analysis applies literally inside formal models and representations wherever the source-defined carrier and relation can be established. Its documented habitats include:

Implications. Software support for this method is maintained in the widely used R package metafor, whose function fits equal-effects models to all (or a large random sample of) subsets of a fitted meta-analytic model, and companion packages such as dmetar extend the technique with unsupervised clustering algorithms (k-means, DBSCAN, and Gaussian mixture models) to automatically flag which studies drive the heterogeneity patterns observed in a GOSH plot.
Concept. This differs from the standard approach in meta-analysis of adopting a single method and computing a single result, and allows significant triangulation to occur, by computing different indices for each combination and examining whether they all tell the same story.
Implications. CMA can thus be used as a data mining method to identify the number of intercepts that may be present in the dataset by looking at which studies are included in the local minima that may be obtained through recombination.
Implications. Recent applications of combinatorial and bootstrap methods in meta-analysis.
Bootstrap methods. A study proposing a bootstrap resampling method for meta-analysis of magnetic resonance imaging (MRI) data collected across multiple scanners used individual participant data within each scanner and estimated the variance of the combined summary statistic via bootstrapping, comparing it against the DerSimonian–Laird and Hartung–Knapp random-effects methods as well as conventional pooling.
Bootstrap methods. The authors note that bootstrap methods offer a natural way to evaluate the sampling variability of diagnostic statistics and to construct data-adaptive reference values for identifying outliers and influential studies, and that similar bootstrap-calibrated approaches have previously been used in influence diagnostics for multicenter clinical trials, diagnostic test accuracy meta-analysis, and network meta-analysis; the package aims to fill a gap in user-friendly computational tools for applying these methods in routine pairwise meta-analysis.

Outside formal models and representations, the name should be retained only when these same operational conditions survive; otherwise the comparison belongs to the broader parent Theory or should be marked as analogy.

Clarity¶

A clear use of Combinatorial Meta-Analysis names the carrier, the operative relation, and the conditions under which the source treats the identity as present. The minimal definition is Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research). The strongest recognition evidence in the frozen account is: It is also possible through CMA to examine the relationship of covariates with effect sizes. A report should distinguish that evidence from a proxy, consequence, or common implementation. It should also state the qualification CMA makes it possible to study the relative behaviour of different statistics under combinatorial conditions. so that a reader can reproduce the classification rather than infer it from topical resemblance.

Manages Complexity¶

Combinatorial Meta-Analysis compresses multiple formal models and representations details into a stable diagnostic relation. The source shows both the central mechanism—cMA can thus be used as a data mining method to identify the number of intercepts that may be present in the dataset by looking at which studies are included in the local minima that may be obtained through recombination.—and the practical consequence—more recent work by a software porting team at Brown University has implemented the concept in STATA. This compression makes cases comparable while leaving parameters, conventions, exceptions, and evidential quality explicit. It is lossy by design: local history and implementation details may be omitted only when they do not alter the defining relation.

Abstract Reasoning¶

Type the carrier. Identify the formal models and representations entities to which the claim applies.
State the relation. Use the source-grounded identity: Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research).
Check operation and conditions. A useful tool developed by Gee was the "PPES" plot (standing for "Probability of Positive Effect Size", assuming differences are scaled such that larger in a positive direction is desired).
Demand recognition evidence. It is also possible through CMA to examine the relationship of covariates with effect sizes.
Test variation. Change an implementation or setting while preserving for example, if industry funding is suspected as a source of bias, then the proportion of studies in a given subset that were industry funded can be computed and plotted directly against the effect size estimate.
Run the collapse test. Remove the defining operation; if the label still seems equally apt, only a topic or correlate was retained.
Reduce cautiously. When the specialist conditions cannot be carried, route the residual comparison to Theory.

Knowledge Transfer¶

Within the home domain. Knowledge about Combinatorial Meta-Analysis transfers literally when a new case preserves the same carrier type, relation, and recognition test. Software support for this method is maintained in the widely used R package metafor, whose function fits equal-effects models to all (or a large random sample of) subsets of a fitted meta-analytic model, and companion packages such as dmetar extend the technique with unsupervised clustering algorithms (k-means, DBSCAN, and Gaussian mixture models) to automatically flag which studies drive the heterogeneity patterns observed in a GOSH plot. This differs from the standard approach in meta-analysis of adopting a single method and computing a single result, and allows significant triangulation to occur, by computing different indices for each combination and examining whether they all tell the same story.

Beyond the home domain. No canonical parent is asserted for Combinatorial Meta-Analysis. An outside case receives the specialist name only when the same typed roles and rejection conditions can be filled literally; otherwise the comparison remains an analogy pending later graph densification.

Examples¶

Canonical¶

It is observed that this is a special case of the more general approach of CMA which computes results for k studies taken 1, 2, 3 ... k − 1, k at a time. This case is canonical because it supplies a concrete carrier and lets the defining relation be checked rather than merely named.

Mapped back: carrier → the entities in the documented case; operation → Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research); recognition evidence → It is also possible through CMA to examine the relationship of covariates with effect sizes

Applied / In Practice¶

This can be adapted to a "PMES" plot (standing for "Probability of Minimal Effect Size"), where the proportion of studies exceeding some minimal effect size (e.g., SMD = 0.10) is taken for each value of j = 1, 2, ... k − 1, k. The applied case shows how the identity is used under a second setting or qualification while keeping the same operative relation.

Mapped back: changed setting → Implications; invariant → Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research); boundary → the case exits the class when cMA makes it possible to study the relative behaviour of different statistics under combinatorial conditions

Structural Tensions¶

T1 — Stable identity versus admissible variation. CMA makes it possible to study the relative behaviour of different statistics under combinatorial conditions. The tension matters because emphasizing only one side either dissolves the identity or overstates what the evidence and domain conventions warrant.

Diagnostic: Which changes preserve the defining relation, and which replace it?

T2 — Recognition versus proxy. This differs from the standard approach in meta-analysis of adopting a single method and computing a single result, and allows significant triangulation to occur, by computing different indices for each combination and examining whether they all tell the same story. The tension matters because emphasizing only one side either dissolves the identity or overstates what the evidence and domain conventions warrant.

Diagnostic: Does the cited evidence establish the identity or only a correlated sign?

T3 — Definition versus implementation. While this software was shared with colleagues informally, it was not published. The tension matters because emphasizing only one side either dissolves the identity or overstates what the evidence and domain conventions warrant.

Diagnostic: Is the observed implementation constitutive, optional, or merely common?

T4 — Scope versus overextension. A jackknife method was applied to meta-analytic data some years later but it does not appear that specialized software was developed for the task. The tension matters because emphasizing only one side either dissolves the identity or overstates what the evidence and domain conventions warrant.

Diagnostic: Can every claimed application fill the same typed roles without metaphor?

T5 — Transfer versus domain accent. This differs from the standard approach in meta-analysis of adopting a single method and computing a single result, and allows significant triangulation to occur, by computing different indices for each combination and examining whether they all tell the same story. The tension matters because emphasizing only one side either dissolves the identity or overstates what the evidence and domain conventions warrant.

Diagnostic: Does the receiving case instantiate Combinatorial Meta-Analysis literally, co-instantiate Theory, or only resemble it?

T6 — Autonomy versus reduction. CMA can thus be used as a data mining method to identify the number of intercepts that may be present in the dataset by looking at which studies are included in the local minima that may be obtained through recombination. The tension matters because emphasizing only one side either dissolves the identity or overstates what the evidence and domain conventions warrant.

Diagnostic: What does Combinatorial Meta-Analysis distinguish that the broader parent Theory leaves together?

Structural–Framed Character¶

Combinatorial Meta-Analysis is mixed or framed-leaning. Its structural side is the repeatable organization summarized by Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research). Its framed side is the formal models and representations vocabulary that fixes the carrier, evidence, exceptions, and admissible transformations.

Evaluative weight: the identity can be stated descriptively even when applications carry practical stakes. Human-practice dependence: the source-grounded carrier determines whether the relation exists independently or is constituted by a practice. Institutional origin: disciplinary conventions stabilize the name and test. Vocabulary portability: A useful tool developed by Gee was the "PPES" plot (standing for "Probability of Positive Effect Size", assuming differences are scaled such that larger in a positive direction is desired). Import versus recognition: literal transfer requires the same mechanism; shape alone is analogy.

Its portable skeleton is Theory. Its character: a recurring specialist identity whose thin organization can be abstracted, while its operational meaning remains domain-bound.

Structural Core vs. Domain Accent¶

What is skeletal. Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research). The stable skeleton is the typed relation expressed in that definition and the entry's recognition and collapse tests. The source identifies these operative conditions: This differs from the standard approach in meta-analysis of adopting a single method and computing a single result, and allows significant triangulation to occur, by computing different indices for each combination and examining whether they all tell the same story. CMA can thus be used as a data mining method to identify the number of intercepts that may be present in the dataset by looking at which studies are included in the local minima that may be obtained through recombination. It further constrains recognition and variation through: A useful tool developed by Gee was the "PPES" plot (standing for "Probability of Positive Effect Size", assuming differences are scaled such that larger in a positive direction is desired). It is also possible through CMA to examine the relationship of covariates with effect sizes.

What is domain-bound. formal models and representations supplies the operative entities, technical vocabulary, warrants, and exceptions that make Combinatorial Meta-Analysis literal. Its documented scope includes the condition that Software support for this method is maintained in the widely used R package metafor, whose function fits equal-effects models to all (or a large random sample of) subsets of a fitted meta-analytic model, and companion packages such as dmetar extend the technique with unsupervised clustering algorithms (k-means, DBSCAN, and Gaussian mixture models) to automatically flag which studies drive the heterogeneity patterns observed in a GOSH plot. Another bounded application condition is that This differs from the standard approach in meta-analysis of adopting a single method and computing a single result, and allows significant triangulation to occur, by computing different indices for each combination and examining whether they all tell the same story. These are not decorative examples; they determine which carrier and evidence can fill the abstraction's roles.

Why no parent is asserted. Removing those specialist details does not currently yield one live catalog node that is a necessary genus for every instance. The entry is therefore approved as unparented rather than attached by topical resemblance. Its collapse evidence remains specific—For example, if industry funding is suspected as a source of bias, then the proportion of studies in a given subset that were industry funded can be computed and plotted directly against the effect size estimate.—and future graph densification may discover a defensible relation only if it preserves that boundary.

Approved unparented node. No current live node supplies a defensible necessary genus or structural prerequisite for Combinatorial Meta-Analysis. The reviewed identity is: Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research). The accelerated suggestion was declined because topical or lexical similarity does not establish hierarchy; the node is admitted without a parent pending later graph densification.
Related reasoning operations. Evidence, representation, comparison, classification, transformation, or evaluation may participate in particular cases, but participation does not make any one of them a necessary parent of every instance.

Neighborhood in Abstraction Space¶

Combinatorial Meta-Analysis sits in a sparse region of the domain-specific corpus (75^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Clinical Trial & Research Methodology (20 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-08

Not to Be Confused With¶

Theory. The parent omits the specialist differentia. Tell: Can the case establish Combinatorial meta-analysis (CMA) is the study of the behaviour of statistical properties of combinations of studies from a meta-analytic dataset (typically in social science research)?
Canonical correlation. Way of inferring information from cross-covariance matrices. Tell: Which entry's carrier, operation, and failure condition are satisfied?
Combinatorial method (linguistics). A contextual method for inferring an undeciphered language’s word classes, meanings, and grammar without a known relative or large bilingual corpus. Tell: Which entry's carrier, operation, and failure condition are satisfied?
K-statistic. A symmetric unbiased estimator of a population cumulant constructed from sample power sums. Tell: Which entry's carrier, operation, and failure condition are satisfied?
A measurement, proxy, or consequence. Those may provide evidence without being the identity. Tell: Would Combinatorial Meta-Analysis remain present if the detector or downstream effect changed?
A metaphorical analogue. A similar shape outside formal models and representations lacks the specialist mechanism. Tell: Do the native roles transfer literally, or only the parent Theory?

References¶

Frozen Wikipedia discovery revision: https://en.wikipedia.org/wiki/Combinatorial_meta-analysis (revision 1368318121).
Preserved source candidate: https://www.acrjournal.com.au/resources/assets/journals/Volume-1-Issue-1-2005/V1_I1_Gee_52-75_7_05.pdf
Preserved source candidate: https://www.researchgate.net/publication/261534226
Preserved source candidate: https://wviechtb.github.io/metafor/reference/gosh.html
Preserved source candidate: https://dmetar.protectlab.org/reference/gosh.diagnostics.html

The frozen Wikipedia revision is discovery provenance. The retained source set was reviewed for identity, formal or operational relation, and scope. The encyclopedia's structural synthesis is bounded to those claims; a thin authority surface is recorded as a nonblocking source-strengthening repair rather than concealed.