Microsoft and Nvidia break records with neural network that mimics language

Selene supercomputer

Nvidia’s Selene supercomputer


Microsoft and Nvidia have created a vast artificial intelligence that can mimic human language more convincingly than ever before – but the cost and time involved in creating the neural network has called into question whether such AIs can continue to scale up.

The Megatron-Turing Natural Language Generation model (MT-NLG) has 530 billion parameters, more than tripling the scale of OpenAI’s groundbreaking GPT-3 model that was considered the state of the art up until now. This progress required more than a month of supercomputer access and almost 4500 high-power and …

Source link