Unsupervised Cross-lingual Representation Learning at Scale.

Conneau, A., Khandelwal, Goyal, Chaudhary, Wenzek, Guzman, Grave, et al. (2020). Unsupervised Cross-lingual Representation Learning at Scale. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 8440-8451.

Type

Conference paper

Intellectual base

Primary research

Year

2020

Pages

8440-8451

Link

https://aclanthology.org/2020.acl-main.747/

Cited from

linguistics_semiotics

Cited by

2 citations across 2 artifacts.

Each citation links to the sentence it supports in the citing article.

Primes

• Code-Switching
  • Analogous structurally even if the user group is a single brain. Computational linguistics and NLP: Modern multilingual language models (XLM-R)

    This sourceIntroduces XLM-R, a multilingual masked-language model trained on 100 languages of CommonCrawl that implicitly ingests code-switched data.

    Supported in partVerified against the publisher's abstract

    Confirms XLM-R is a modern Transformer-based multilingual masked language model trained on 100 languages, but says nothing about code switching or a single-brain analogy.

    “We train a Transformer-based masked language model on one hundred languages, using more than two terabytes of filtered CommonCrawl data. Our model, dubbed XLM-R, significantly outperforms multilingual BERT (mBERT) on a variety of cross-lingual benchmarks”

    Read by an automated reader; not a human review. How support was checked

• Linguistic Universals
  • - Machine translation and multilingual NLP: Multilingual models (XLM, mBERT, XLM-R

    This sourceConneau XLM-R multilingual model cross-lingual transfer learning universals.

    Supported in partVerified against the publisher's abstract

    The abstract names XLM-R and mBERT as multilingual models and reports cross-lingual benchmark gains, backing the multilingual-NLP half but not the machine-translation label or XLM.

    “This paper shows that pretraining multilingual language models at scale leads to significant performance gains for a wide range of cross-lingual transfer tasks. We train a Transformer-based masked language model on one hundred languages, using more than two terabytes of filtered CommonCrawl data. Our model, dubbed XLM-R, significantly outperforms multilingual BERT (mBERT) on a variety of cross-lingual benchmarks, …”

    Read by an automated reader; not a human review. How support was checked

Verification

Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.

Does it back the claim? Read against the text for 2 of 2 citations: 2 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.

Was it audited? Yes. A second, independent pass read the citation against the article text and recorded a verdict.

Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.

See how references were verified.

Registry ID ref:fd245cd95642 · see in the full table