Gaussian Error Linear Units (GELUs)

Dan Hendrycks, K. G. (2016). Gaussian Error Linear Units (GELUs).

Type

Unclassified

Intellectual base

Unclassified

Year

2016

Link

no authoritative link yet

Cited by

1 citation across 1 artifact.

Domain-specific

• Swish function
  • … identity and a convenient observable transfers from At β=1 the activation is x times the logistic sigmoid of x; as β grows it approaches ReLU pointwise away from zero. to Model evaluation treats activation benefit as architecture- and optimization-dependent and records parameter training and numerical stability..

Verification

This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.

Registry ID ref:dd8203dad73e · see in the full table