Boffins Rejoice: US Energy Department's Research Network Gets A 400G Upgrade

The fiber network connecting the US Department of Energy's (DoE) national laboratories received its first major upgrade in more than a decade this week, boosting its bandwidth capacity to a total of 46Tbit/sec.

Established in 1986, ESnet has grown to encompass 15,000 miles of fiber optic cables that transport scientific data between the numerous supercomputers and thousands of researchers housed at the DoE's 17 national labs.

"As scientific instruments grow in complexity and supercomputers simulate scientific phenomena at higher resolution, the scientific community is facing a growing challenge: data volumes increasing exponentially," said Barbara Helland, associate director of the DoE's Advanced Scientific Computing program, in a statement.

According to the DoE, in 2021, ESnet carried more than 1.1 exabytes of scientific data. The agency expects that to increase by a factor of ten every four years.

ESnet6 boosts the capacity of the network by an order of magnitude, with link bandwidth now ranging from 400 to 1,000Gbit/sec, up from the 100Gbit/sec that ESnet5 offered at its 2011 launch.

During a launch event at ESnet's headquarters at the Lawrence Berkeley National Laboratory Tuesday, the Department demonstrated how the network could transfer a multi-terabyte earthquake simulation nearly 2,500 miles to Oak Ridge National Laboratory in a matter of minutes.

The DoE is hardly the only scientific institution struggling with massive data flows generated by scientific research.

In 2019, researchers responsible for producing the first visual evidence of a supermassive blackhole generated roughly 14 petabytes of data over five days, which had to be shipped in hard drives by plane. At 400Gbit/sec, ESnet6 could transfer an equivalent dataset in a little over 87 hours.

The upgrades announced this week aren't limited to higher bandwidths. According to the DoE the upgrade also introduced greater degrees of programmability, automation, and security. Eventually, the DoE hopes researchers will be able to request network services using an API.

The upgraded network uses optical cable, switching, routing, and packet processing tech from Lumen Technologies (formerly CenturyLink), Ciena, Infinera, Nokia, and AMD.

"These new capabilities make it faster, easier, and more efficient for scientists around the world to conduct and collaborate on ground-breaking research," ESnet Executive Director Inder Monga, declared in a statement.

The updates come as the DoE invests heavily in new supercomputing hardware, and less than six months after the Oak Ridge National Laboratory's 1.1 exaflop Frontier supercomputer claimed the number one spot on the Top 500. ®