Japans Fugaku Supercomputer Follow-up Adds Nvidia GPUs

Japanese research institution RIKEN has decided it needs GPUs for its next generation “FugakuNEXT” supercomputer and has signed Nvidia to supply them and design the systems needed to get them working.

RIKEN is home to Fugaku, a machine that from mid-2020 spent two years atop the TOP500 list of Earth’s mightiest supercomputers. The machine is still in seventh place, but RIKEN wants an upgrade and has already awarded a contract to Fujitsu to build its successor and the custom Arm-based CPU called “MONAKA-X” that will power it.

On Friday, the institution announced a change to its plan: Adding Nvidia GPUs, and contracting the AI hardware giant to design systems that incorporate its accelerators into the supercomputer.

RIKEN said Nvidia’s design “will explore the adoption of cutting-edge connection technologies between the CPU and GPU, while also considering the incorporation of advanced memory technologies.”

While Nvidia and RIKEN are both chuffed about the decision to use GPUs, neither has said which model accelerator FugakuNEXT will use, how many will be present, and whether adding GPUs means the supercomputer’s design will include Nvidia networks.

But whatever ends up inside FugakuNEXT, it will be very fast. RIKEN’s announcement of its Nvidia collaboration states it aims for the machine to exceed 600 exaFLOPS (EFLOPS) in FP8 precision (sparse), and expects the machine “to become the world's first ‘zetta-scale’ system.” The research outfit added that involving Nvidia means it’s set a target of “more than a fivefold improvement in hardware performance over Fugaku,” and it will pursue “a hundredfold increase in application performance” compared to its current supercomputer.

• Jupiter, Europe’s most powerful super, takes maiden flight - but misses exascale orbit
• Japan's Fugaku supercomputer released in virtual version that runs in AWS
• IBM quantum system elbows into Arm-powered Fugaku supercomputer
• Fujitsu, AMD lay groundwork to pair Monaka CPUs with Instinct GPUs

RIKEN’s announcement states that FugakuNEXT “will deliver exceptional performance in AI training and GPU-optimized HPC applications.” The reference to “training” could indicate large language models, as Japanese researchers have already used the current Fugaku machine to train the 13-billion parameter Fugaku-LLM.

Fujitsu is yet to deliver the MONAKA-X, and RIKEN wants FugakuNEXT to come online “around 2030”. Let’s see if adding GPUs to the machine helps or hinders that goal. ®