DeepSpeed-MoE

Samyam Rajbhandari, C. L., Yao, Z., Zhang, M., Aminabadi, R. Y., & Awan, A. A. (2022). DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale.

Type

Unclassified

Intellectual base

Unclassified

Year

2022

Link

no authoritative link yet

Cited by

1 citation across 1 artifact.

Domain-specific

• Neural scaling law
  • … evaluation set, scaling variables held fixed or co-varied, fitted functional form exponents and uncertainty, observed range and extrapolation limit and compute-optimal allocation rule are explicit. to An applied instance preserves the invariant under changed notation, scale, dataset, jurisdiction, or implementation..

Verification

This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.

Registry ID ref:f2cad7d260d6 · see in the full table