(untitled)¶
(2002). (untitled).
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Domain-specific¶
- Hierarchical Dirichlet process
- … identity and a convenient observable transfers from Documents have different topic proportions while all select from one data-inferred global collection of topics. to A modeler checks sensitivity to concentration priors and likelihood misspecification and distinguishes inferred components from natural kinds..
This sourceWhat this means is that the data items come in multiple distinct groups. For example, in a topic model words are organized into documents, with each document formed by a bag (group) of words (data items). Indexing groups by j=1,...J , suppose each group consist of data items x_{j1},...x_{jn} . The HDP is parameterized by a base distribution H that governs the a priori distribution over data items, and a number of concentration parameters that govern the a priori number of clusters and amount of sharing across groups. The j th group is associated with a random probability measure G_j which has distribution given by a Dirichlet process: : \begin{align} G_j|G_0 &\sim \operatorname{DP}(\alpha_j,G_0) \end{align} where \alpha_j is the concentration parameter associated with the group, and G_0 is the base distribution shared across all groups. In turn, the common base distribution is Dirichlet process distributed: : \begin{align} G_0 &\sim \operatorname{DP}(\alpha_0,H) \end{align} with concentration parameter \alpha_0 and base distribution H . Finally, to relate the Dirichlet processes back with the observed data, each data item x_{ji} is associated with a latent parameter \theta_{ji} : : \begin{align} \theta_{ji}|G_j &\sim G_j \\ x_{ji}|\theta_{ji} &\sim F(\theta_{ji}) \end{align} The first line states that each parameter has a prior distribution given by G_j , while the second line states that each data item has a distribution F(\theta_{ji}) parameterized by its associated parameter. The resulting model above is called a HDP mixture model, with the HDP referring to the hierarchically linked set of Dirichlet processes, and the mixture model referring to the way the Dirichlet processes are related to the data items. To understand how the HDP implements a clustering model, and how clusters become shared across groups, recall that draws from a Dirichlet process are atomic probability measures with probability one. This means that the common base distribution G_0 has a form which can be written as: : \begin{align} G_0 &= \sum_{k=1}^\infty \pi_{0k}\delta_{\theta^_k} \end{align} where there are an infinite number of atoms, \theta^_k, k=1,2,... , assuming that the overall base distribution H has infinite support. Each atom is associated with a mass \pi_{0k} . The masses have to sum to one since G_0 is a probability measure. Since G_0 is itself the base distribution for the group specific Dirichlet processes, each G_j will have atoms given by the atoms of G_0 , and can itself be written in the form: : \begin{align} G_j &= \sum_{k=1}^\infty \pi_{jk}\delta_{\theta^_k} \end{align} Thus the set of atoms is shared across all groups, with each group having its own group-specific atom masses. Relating this representation back to the observed data, we see that each data item is described by a mixture model: : \begin{align} x_{ji}|G_j &\sim \sum_{k=1}^\infty \pi_{jk} F(\theta^_k) \end{align} where the atoms \theta^*_k play the role of the mixture component parameters, while the masses \pi_{jk} play the role of the mixing proportions. In conclusion, each group of data is modeled using a mixture model, with mixture components shared across all groups but mixing proportions being group-specific. In clustering terms, we can interpret each mixture component as modeling a cluster of data items, with clusters shared across all groups, and each group, having its own mixing proportions, composed of different combinations of clusters. ==Applications== The HDP mixture model is a natural nonparametric generalization of Latent Dirichlet allocation, where the number of topics can be unbounded and learnt from data. Here each group is a document consisting of a bag of words, each cluster is a topic, and each document is a mixture of topics. The HDP is also a core component of the infinite hidden Markov model, Beal, M.J., Ghahramani, Z. and Rasmussen, C.E. (2002). "The infinite hidden Markov model" (PDF). Advances in Neural Information Processing Systems 14:577–585. Cambridge, MA: MIT Press.
- … identity and a convenient observable transfers from Documents have different topic proportions while all select from one data-inferred global collection of topics. to A modeler checks sensitivity to concentration priors and likelihood misspecification and distinguishes inferred components from natural kinds..
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:81ca6245f5c4 · see in the full table