attempto online - Research
23.08.2024
University of Tübingen and Google DeepMind Accelerate Neural Network Training Through Open-Source Competition
New Methods Significantly Cut Neural Network Training Time, Boosting AI Efficiency
Researchers from the Tübingen AI Center at the University of Tübingen, in partnership with Google DeepMind and other academic and industry labs, have carried out a competition that found new algorithms which greatly speed up the training of neural networks. These innovations could reduce training time by up to one-third, marking a major advancement in machine learning.
AlgoPerf: A Public Competition to Find the Best Training Algorithms
As part of their mission to enhance artificial intelligence, the University of Tübingen, Google DeepMind and collaborators launched a public benchmark competition called AlgoPerf: Training Algorithms Benchmark. This open-source competition, managed by the MLCommons Algorithms Working Group, sought to find the most efficient mathematical methods (called algorithms) to significantly shorten the time it takes to train neural networks.
The competition's rules ensured that participants could only modify the training methods, not the hardware or software used, making sure that faster training was achieved through better algorithms rather than more powerful computers. Google provided the necessary computing resources for the competition, and the results are available to everyone under an Apache 2.0 license.
Global Participation from Leading AI Institutions
The competition drew 18 submissions from 10 teams representing top AI research institutions worldwide, including ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems, UCLA, the University of Cambridge, Meta AI, Samsung AI and the Vector Institute.
Participants were tasked with creating algorithms that could speed up neural network training for various real-world tasks. Some algorithms delivered impressive results, improving training speeds by as much as 28% compared to current state-of-the-art methods.