The Metaverse might not fully exist yet (and we don’t even know when it will) - but Meta is developing the world’s fastest AI supercomputer, which is slated to be finished in mid-2022.

We’ve all heard about the Metaverse in the last several months: a network of 3D virtual worlds focused on social connection, accessed by VR or AR goggles. In 2021 Facebook renamed itself “Meta Platforms” and declared itself devoted to developing the Metaverse. It’s thought that this virtual reality will be the next iteration of the internet. Though when, specifically, is a mystery.

Meta actually started its AI research ten years ago, with the Facebook AI Research lab. The lab developed chatbot design, AI systems to forget unnecessary information, and even synthetic skin that gives robots the ability to have a sense of touch. In 2017, Meta launched its first AI supercomputer, which leveraged open source and publicly available data sets. The new supercomputer, named AI Research SuperCluster - or RSC - will use its powerful hardware to train large computer vision and natural language processing models. Real time voice translation will be one of the main highlights for RSC, so that people all over the world will be able to chat in the Metaverse in real time, all speaking different languages and seamlessly communicating with one another.

In a blog post, Meta explains what the AI can already do, which includes translating languages and identifying harmful content. Upon completion, RSC should be able to accomplish building entirely new AI systems to power real time voice translation for huge groups of people, combining computer vision, natural language processing, and speech recognition. According to Mark Zuckerberg, RSC is already the fifth fastest computer in the world. Built from thousands of processors and currently hiding away in an undisclosed location, it is already operational, but will be launched later this year. The current computational infrastructure will need to improve a thousandfold to power the metaverse.

It makes sense that in order to fuel the Metaverse, RSC will require an immense amount of rapid computational power. There’s a ton of different ways to describe the computational power at play here - quintillions of operations per second, petaflops (one thousand teraflops) of computing in less than a millisecond, 5 exaflops of mixed precision computing at its peak, trillions of parameters in the neural networks. The natural language processor GPT-3 has 175 billion parameters alone. The current limit to RSC’s growth is the time it takes to train a neural network, which can take weeks of computing for large networks. New neural networks need to be built quickly in order to accomplish real time voice translations at the desired scale for the Metaverse.

The old system used 22,000 Nvidia V100 GPUs, and currently uses 6,080 Nvidia A100 GPUs. By later this year, when RSC is ready to be launched, it will be using 16,000 Nvidia A100 GPUs. RSC will train models with more than a trillion parameters on data sets as large as an exabyte, or 36,000 years of high-quality video. By connecting to 16,000 GPUs, the cache and storage will have a capacity of 1 exabyte, or 1 billion billion bytes, serving 16 terabytes per second of data to the system.

With this impressive computational power, RSC will enable new AI models that can learn from trillions of examples. But where, exactly, will these examples come from? Unlike its predecessor, RSC will train machine learning models on data sourced from the social media owned by Meta - Facebook, Instagram, WhatsApp, and others. And this might make you raise your eyebrows. What about security, and data privacy? Well, according to Meta, RSC has been designed from its infancy with privacy and security in mind, with the supercomputer being isolated from the internet, and having no inbound or outbound connections. Traffic will flow only from Meta’s production data centers and the entire data path is encrypted.

The COVID-19 pandemic has caused some setbacks on the project, just as it has for all industries. Supply chain constraints and other issues made it difficult to get necessary materials to build RSC, like chips and GPUs, and even basic construction materials. But if all goes according to plan, 2022 will be a big year for AI becoming faster, smarter, and more powerful than ever.