Nvidia says it has found a way speed up a computation-limited step in the chipmaking process so it happens 40 times as fast as today’s standard. Called inverse lithography, it’s a key tool to allow chipmakers to print nanometer-scale features using light with a longer wavelength than the size of those features. Inverse lithography’s use has been limited by the massive size of the needed computation. Nvidia’s answer, cuLitho, is a set of algorithms designed for use with GPUs, that turns what has been two weeks of work into an overnight job.  

The technology “will allow fabs to increase throughput, reduce their carbon footprint, and set the foundations for 2-nanometer \[manufacturing processes\] and beyond,” said [Nvidia CEO Jensen Huang at the Nvidia GTC developer](https://www.nvidia.com/gtc/keynote/) on Tuesday.

Leading logic chip foundry [TSMC](https://www.tsmc.com/english) will be qualifying cuLitho’s use in production starting in June, said Huang. Design automation software firm [Synopsys](https://blogs.synopsys.com/from-silicon-to-software/2023/03/21/nvidia-computational-lithography/) plans to integrate software, too, and lithography equipment maker [ASML](https://www.asml.com/en) plans to support cuLitho in its products as well.

Photolithography is basically the first step in the chipmaking process. It involves bouncing light off of a pattern called a photomask to project the forms of transistor structures and interconnects onto the wafer. (More mature technology uses transmissive photomasks instead of reflective ones, but the idea is the same.) It takes 98 photomasks to make an H100 GPU, Jensen said. The features projected from the photomask are smaller than the wavelength of light used—193 nanometers for the relatively large features and [13.5 nanometers](https://spectrum.ieee.org/euv-lithography-finally-ready-for-chip-manufacturing) for the finer bits. So, without the aid of tricks and design rules collectively called optical proximity correction, you’d only get a blurry mess projected on the wafer. With optical proximity correction, the designs on the photomask only vaguely resemble the pattern of light at the chip.

With the need for finer and finer features, the corrected shapes on the photomask have become more and more elaborate and difficult to come up with. It would be much better to start with the pattern you want on the wafer and then calculate what pattern on the photomask would produce them. Such a scheme is called inverse lithography. Simple as it sounds, it’s quite difficult to compute, often taking weeks to compile.

In fact, it’s such a slog that it’s often reserved for use on only a few critical layers of leading-edge chips or just particularly thorny bits of them, according to data from the [E-Beam Initiative](https://www.ebeam.org/home), which periodically surveys the industry.

![Images on a timeline start as a simple cross and become more complicated as time goes on.](https://spectrum.ieee.org/media-library/images-on-a-timeline-start-as-a-simple-cross-and-become-more-complicated-as-time-goes-on.jpg?id=33335579&width=980) As chipmaking required finer and finer features, engineers had to produce more and more complex designs to project those features onto the silicon. Inverse lithography (ILT) is the latest development.Nvidia

The long computation time for lithography slows the development and improvement of chip technology Even a change to the thickness of a material can lead to the need for a new set of photomasks, notes [Vivek K. Singh](https://www.linkedin.com/in/vivek-singh-9943891b8/), vice president in the advanced technology group working on silicon manufacturing at Nvidia. Computing masks “has been a long pole in chip development,” he says. “If inverse lithography technology were sped up 40 times, would more companies use it on more layers? Surely.”

Part of the computation is an image problem that’s a natural fit for GPUs, says Singh. But at most, that can only cut the computation time in half. The rest is not so easy to make parallel. But over the past four years, with development partners including TSMC, Nvidia engineers have come up with a collection of algorithms for making the remaining work parallel and have packaged it as a software library for use with GPUs.

According to Nvidia, using the cuLitho lets 500 Nvidia DGX [H100](https://spectrum.ieee.org/nvidias-next-gpu-shows-that-transformers-are-transforming-ai) computers do the work of 40,000 CPU systems. It can produce 3 to 5 times as many photomasks per day, drawing only 5 megawatts instead of 35 MW.

What’s more, it may deliver better results, according to Singh. cuLitho produces otherwise hard to calculate curvy polygons on the mask that result in a greater depth of focus for the pattern cast onto the wafer. That depth of focus should lead to less variation across the wafer and therefore a better yield of working chips per wafer, he says. In future, it also could mean fewer photomasks are needed, because what must now be done using a double pattern might work with only one using inverse lithography.

Nvidia is not the first to look to GPUs to accelerate inverse lithography technology. Silicon Valley-based [D2S](https://design2silicon.com/products/truemask-ilt) announced a [GPU-based computer](https://spectrum.ieee.org/custom-computer-makes-inverse-lithography-practical-for-first-time) custom built for the problem in 2019. _IEEE Spectrum_ reached out to D2S for comment, but the company did not reply before press time.

Nvidia Speeds Key Chipmaking Computation by 40x


There’s a handful of robotics companies currently working on what could be called general-purpose humanoid robots. That is, human-size, human-shaped robots with legs for mobility and arms for manipulation that can (or, may one day be able to) perform useful tasks in environments designed primarily for humans. The value proposition is obvious—drop-in replacement of humans for dull, dirty, or dangerous tasks. This sounds a little ominous, but the fact is that people don’t _want_ to be doing the jobs that these robots are intended to do in the short term, and there just aren’t enough people to do these jobs as it is.

We tend to look at claims of commercializable general-purpose humanoid robots with some skepticism, because humanoids are really, really hard. They’re still really hard in a research context, which is usually where things have to get easier before anyone starts thinking about commercialization. There are certainly companies out there doing some amazing work toward practical legged systems, but at this point, “practical” is more about not falling over than it is about performance or cost effectiveness. The overall approach toward solving humanoids in this way tends to be to build something complex and expensive that does what you want, with the goal of cost reduction over time to get it to a point where it’s affordable enough to be a practical solution to a real problem.

[Apptronik](https://apptronik.com/our-work/), based in Austin, Texas, is the latest company to attempt to figure out how to make a practical general-purpose robot. Its approach is to focus on things like cost and reliability from the start, developing (for example) its own actuators from scratch in a way that it can be sure will be cost effective and supply-chain friendly. Apptronik’s goal is to develop a platform that costs well under US $100,000 of which it hopes to be able to deliver a million by 2030, although the plan is to demonstrate a prototype early this year. Based on what we’ve seen of commercial humanoid robots recently, this seems like a huge challenge. And in part two of this story (to be posted tomorrow), we will be talking in depth to Apptronik’s cofounders to learn more about how they’re going to make general-purpose humanoids happen.

* * *

First, though, some company history. Apptronik spun out from the [Human Centered Robotics Lab at the University of Texas at Austin](https://sites.utexas.edu/hcrl/) in 2016, but the company [traces its robotics history](https://apptronik.com/our-work/) back a little farther, to 2015’s [DARPA Robotics Challenge](https://spectrum.ieee.org/darpa-robotics-challenge-amazing-moments-lessons-learned-whats-next). Apptronik’s CTO and cofounder, [Nick Paine](https://www.linkedin.com/in/nipaine/), was on the [NASA-JSC Valkyrie DRC team](https://spectrum.ieee.org/nasa-jsc-unveils-valkyrie-drc-robot), and Apptronik’s first contract was to work on next-gen actuation and controls for NASA. Since then, the company has been working on robotics projects for a variety of large companies. In particular, Apptronik developed Astra, a humanoid upper body for dexterous bimanual manipulation that’s currently being tested for supply-chain use.

But Apptronik has by no means abandoned its NASA roots. In 2019, [NASA had plans for what was essentially going to be a Valkyrie 2](https://spectrum.ieee.org/nasa-hiring-engineers-to-develop-next-generation-humanoid-robot), which was to be a ground-up redesign of the Valkyrie platform. As with many of the coolest NASA projects, the potential new humanoid didn’t survive budget prioritization for very long, but even at the time [it wasn’t clear to us](https://spectrum.ieee.org/nasa-hiring-engineers-to-develop-next-generation-humanoid-robot) why NASA wanted to build its own humanoid rather than asking someone else to build one for it considering how much progress we’ve seen with humanoid robots over the last decade. Ultimately, NASA decided to move forward with more of a partnership model, [which is where Apptronik fits in](https://www.prweb.com/releases/austin_based_apptronik_inks_partnership_with_nasa_for_humanoid_robots/prweb18907816.htm)—a partnership between Apptronik and NASA will help accelerate commercialization of Apollo.

“We recognize that Apptronik is building a production robot that’s designed for terrestrial use,” says NASA’s [Shaun Azimi](https://www.linkedin.com/in/shaun-azimi-90b0b2179/), who leads the Dexterous Robotics Team at [NASA’s Johnson Space Center](https://www.nasa.gov/centers/johnson/home/index.html). “From NASA’s perspective, what we’re aiming to do with this partnership is to encourage the development of technology and talent that will sustain us through the Artemis program and looking forward to Mars.”

Apptronik is positioning Apollo as a high-performance, easy-to-use, and versatile system. It is imagining an “iPhone of robots.”

“Apollo is the robot that we always wanted to build,” says [Jeff Cardenas](https://www.linkedin.com/in/jeffrey-cardenas/), Apptronik cofounder and CEO. This new humanoid is the culmination of an astonishing amount of R&D, all the way down to the actuator level. “As a company, we’ve built more than 30 unique electric actuators,” Cardenas explains. “You name it, we’ve tried it. Liquid cooling, cable driven, series elastic, parallel elastic, quasi-direct drive…. And we’ve now honed our approach and are applying it to commercial humanoids.”

Apptronik’s emphasis on commercialization gives it a much different perspective on robotics development than you get when focusing on pure research the way that NASA does. To build a commercial product rather than a handful of totally cool but extremely complex bespoke humanoids, you need to consider things like minimizing part count, maximizing maintainability and robustness, and keeping the overall cost manageable. “Our starting point was figuring out what the minimum viable humanoid robot looked like,” explains Apptronik CTO Nick Paine. “Iteration is then necessary to add complexity as needed to solve particular problems.”

This robot is called Astra. It’s only an upper body, and it’s Apptronik’s first product, but (not having any legs) it’s designed for manipulation rather than dynamic locomotion. Astra is force controlled, with series-elastic torque-controlled actuators, giving it the compliance necessary to work in dynamic environments (and particularly around humans). “Astra is pretty unique,” says Paine. “What we were trying to do with the system is to approach and achieve human-level capability in terms of manipulation workspace and payload. This robot taught us a lot about manipulation and actually doing useful work in the world, so that’s why it’s where we wanted to start.”

While Astra is currently out in the world doing pilot projects with clients (mostly in the logistics space), internally Apptronik has moved on to robots with legs. The following video, which Apptronik is sharing publicly for the first time, shows a robot that the company is calling its Quick Development Humanoid, or QDH:

[  
](https://www.youtube.com/watch?v=nRX6XvFUnls)

QDH builds on Astra by adding legs, along with a few extra degrees of freedom in the upper body to help with mobility and balance while simplifying the upper body for more basic manipulation capability. It uses only three different types of actuators, and everything (from structure to actuators to electronics to software) has been designed and built by Apptronik. “With QDH, we’re approaching minimum viable product from a usefulness standpoint,” says Paine, “and this is really what’s driving our development, both in software and hardware.”

“What people have done in humanoid robotics is to basically take the same sort of architectures that have been used in industrial robotics and apply those to building what is in essence a multi-degree-of-freedom industrial robot,” adds Cardenas. “We’re thinking of new ways to build these systems, leveraging mass manufacturing techniques to allow us to develop a high-degree-of-freedom robot that’s as affordable as many industrial robots that are out there today.”

Cardenas explains that a major driver for the cost of humanoid robots is the number of different parts, the precision machining of some specific parts, and the resulting time and effort it then takes to put these robots together. As an internal-controls test bed, QDH has helped Apptronik to explore how it can switch to less complex parts and lower the total part count. The plan for Apollo is to not use any high-precision or proprietary components at all, which mitigates many supply-chain issues and will help Apptronik reach its target price point for the robot.

Apollo will be a completely new robot, based around the lessons Apptronik has learned from QDH. It’ll be average human size: about 1.75 meters tall, weighing around 75 kilograms, with the ability to lift 25 kg. It’s designed to operate untethered, either indoors or outdoors. Broadly, Apptronik is positioning Apollo as a high-performance, easy-to-use, and versatile robot that can do a bunch of different things. It is imagining an “iPhone of robots,” where apps can be created for the robot to perform specific tasks. To extend the iPhone metaphor, Apptronik itself will make sure that Apollo can do all of the basics (such as locomotion and manipulation) so that it has fundamental value, but the company sees versatility as the way to get to large-scale deployments and the cost savings that come with them.

“I see the Apollo robot as a spiritual successor to Valkyrie. It’s not Valkyrie 2—Apollo is its own platform, but we’re working with Apptronik to adapt it as much as we can to space use cases.”  
—Shaun Azimi, NASA Johnson Space Center  

The challenge with this app approach is that there’s a critical mass that’s required to get it to work—after all, the primary motivation to develop an iPhone app is that there are a bajillion iPhones out there already. Apptronik is hoping that there are enough basic manipulation tasks in the supply-chain space that Apollo can leverage to scale to that critical-mass point. “This is a huge opportunity where the tasks that you need a robot to do are pretty straightforward,” Cardenas tells us. “Picking single items, moving things with two hands, and other manipulation tasks where industrial automation only gets you to a certain point. These companies have a huge labor challenge—they’re missing labor across every part of their business.”

While Apptronik’s goal is for Apollo to be autonomous, in the short to medium term, its approach will be hybrid autonomy, with a human overseeing first a few and eventually a lot of Apollos with the ability to step in and provide direct guidance through teleoperation when necessary. “That’s really where there’s a lot of business opportunity,” says Paine. Cardenas agrees. “I came into this thinking that we’d need to make Rosie the robot before we could have a successful commercial product. But I think the bar is much lower than that. There are fairly simple tasks that we can enter the market with, and then as we mature our controls and software, we can graduate to more complicated tasks.”

Apptronik is still keeping details about Apollo’s design under wraps, for now. We were shown renderings of the robot, but Apptronik is understandably hesitant to make those public, since the design of the robot may change. It does have a firm date for unveiling Apollo for the first time: SXSW, which takes place in Austin in March.

Apptronik Developing General-Purpose Humanoid Robot


When [Xiaomi announced its CyberOne humanoid robot](https://spectrum.ieee.org/humanoid-robot-xiaomi-cyberone) a couple of months back, it wasn’t entirely clear what the company was actually going to do with the robot. Our guess was that rather than pretending that CyberOne was going to have some sort of practical purpose, Xiaomi would use it as a way of exploring possibilities with technology that may have useful applications elsewhere, but there were no explicit suggestions that there would be any actual research to come out of it. In a nice surprise, Xiaomi roboticists have taught the robot to do something that is, if not exactly useful, at least loud: to play the drums.

* * *

The input for this performance is a MIDI file, which the robot is able to parse into drum beats. It then generates song-length sequences of coordinated whole-body trajectories which are synchronized to the music, which is tricky because the end effectors have to make sure to actuate the drums exactly on the beat. CyberOne does a pretty decent job even when it’s going back and forth across the drum kit. This is perhaps not super cutting-edge humanoid research, but it’s still interesting to see what a company like Xiaomi has been up to. And to that end, we asked [Zeyu Ren](https://www.linkedin.com/in/zeyu-ren-robotics/), a senior hardware engineer at the Xiaomi Robotics Lab, to answer a couple of questions for us.

**IEEE Spectrum: So why is Xiaomi working on a humanoid robot, anyway?**

**Zeyu Ren:** There are three reasons why Xiaomi is working on humanoid robots. The first reason is that we are seeing a huge decline in the labor force in China, and the world. We are working on replacing the human labor force with humanoid robots even though there is a long way to go. The second reason is that we believe humanoid robots are the most technically challenging of all robot forms. By working on humanoid robots, we can also use this technology to solve problems on other robot forms, such as quadruped robots, robotic arms, and even wheeled robots. The third reason is that Xiaomi wants to be the most technically advanced company in China, and humanoid robots are sexy.

**Why did you choose drumming to demonstrate your research?**

**Ren:** After the official release of Xiaomi CyberOne on August 11, we got a lot of feedback from the public who didn’t have a background in robotics. They are more interested in seeing humanoid robots doing things that humans cannot easily do. Honestly speaking, it’s pretty difficult to find such scenarios, since we know that the first prototype of CyberOne is far behind humans.

But one day, one of our engineers who had just begun to play drums suggested that drumming may be an exception. She thought that compared to rookie drummers, humanoid robots have more advantages in hand-foot coordinated motion and rhythmic control. We all thought it was a good idea, and drumming itself is super cool and interesting. So we choose drumming to demonstrate our research.

**What was the most challenging part of this research?**

**Ren:** The most challenging part of this research was that when receiving the long sequences of drum beats, CyberOne needs to assign sequences to each arm and leg and generate continuous collision-free whole-body trajectories within the hardware constraints. So, we extract the basic beats and build our drum beat motion trajectory library offline by optimization. Then, CyberOne can generate continuous trajectories consistent with any drum score. This approach gives more freedom to CyberOne playing drums, and is only limited by the robotics capability.

**What different things do you hope that this research will help your robot do in the future?**

**Ren:** Drumming requires CyberOne to coordinate whole-body motions to achieve a fast, accurate, and large range of movement. We first want to find the limit of our robot in terms of hardware and software to provide a reference for the next-generation design. Also, through this research, we have formed a complete set of automatic drumming methods for robots to perform different songs, and this experience also helps us to more quickly realize the development of other musical instruments to be played by robots.

**What are you working on next?**

**Ren:** We are working on the second generation of CyberOne, and hope to further improve its locomotion and manipulation ability. On the hardware level, we plan to add more degrees of freedom, integrate self-developed dexterous hands, and add more sensors. On the software level, more robust control algorithms for locomotion and vision will be developed.

Xiaomi’s Humanoid Drummer Beats Expectations

The machine learning consortium [MLCommons](https://mlcommons.org/en/) released the latest set of benchmark results last week, offering a glimpse at the capabilities of new chips and old as they tackled executing light-weight AI on the tiniest systems and training neural networks at both server and supercomputer scales. The benchmark tests saw the debut of new chips from [Intel](https://www.hpcwire.com/2022/09/27/intel-ships-sapphire-rapids-to-its-cloud/) and [Nvidia](https://www.nvidia.com/en-us/data-center/h100/) as well as speed boosts from software improvements and predictions that new software will play a role in speeding the new chips in the years after their debut.

* * *

Training servers
----------------

Training AI has been a problem that's driven billions of dollars in investment, and it seems to be paying off. “A few years ago we were talking about training these networks in days or weeks, now we’re talking about minutes,” says [Dave Salvator](https://www.linkedin.com/in/davesalvator), director of product marketing at Nvidia.

There are eight benchmarks in the MLPerf training suite, but here I'm just showing results from two—image classification and natural language processing—because although they don't give a complete picture, they're illustrative of what's happening. Not every company puts up benchmark results every time; in the past, systems from [Baidu](https://intl.cloud.baidu.com/), [Google](https://ai.google/), [Graphcore](https://www.graphcore.ai/), and [Qualcomm](https://www.qualcomm.com/research/artificial-intelligence) have made marks, but none of these were on the most recent list. And there are companies whose goal is to train the very biggest neural networks, such as [SambaNova](https://spectrum.ieee.org/sambanova-ceo-ai-interview) and [Cerebras](https://spectrum.ieee.org/cerebras-ai-computers), that have never participated.

Another note about the results I'm showing—they are incomplete. To keep the eye-glazing to a minimum I've only listed the fastest system of each configuration. There were already four categories in the main "closed" contest: cloud (self-evident), on-premises (systems you could buy and install in-house right now), preview (systems you can buy soon but not now), and R&D (interesting but odd, so I excluded them). I then listed the fastest training result for each category for each configuration—the number of accelerators in a computer. If you want to see the complete list, it's at the [MLCommons website](https://mlcommons.org/en/).

A casual glance shows that machine learning training is still very much Nvidia's house. It can bring a supercomputer-scale number of GPUs to the party to smash through training problems in mere seconds. Its [A100 GPUs](https://www.nvidia.com/en-us/data-center/a100/) have dominated the MLPerf list for several iterations now, and it powers Microsoft's Azure cloud AI offerings as well as systems large and small from partners including HPE, Dell, and Fujitsu. But even among the A100 gang there's real competition, particularly between Dell and HPE.

But perhaps more important was Azure's standing. On image classification, the cloud systems were essentially a match for the best A100 on-premises computers. The results strengthen Microsoft's case that renting resources in the cloud is as good as buying your own. And that case might might be even stronger soon. This week [Nvidia and Microsoft announced a multi-year collaboration](https://nvidianews.nvidia.com/news/nvidia-microsoft-accelerate-cloud-enterprise-ai) that would see the inclusion of Nvidia's upcoming GPU, the [H100](https://spectrum.ieee.org/nvidias-next-gpu-shows-that-transformers-are-transforming-ai), in the Azure cloud.

This was the first peak at training abilities for the H100. And Nivida's Dave Salvator emphasized how much progress happens—largely due to software improvements—in the years after a new chip comes out. On a per chip basis, the A100 delivers 2.5 times the average performance today versus its [first run at the MLPerf benchmarks](https://spectrum.ieee.org/new-records-for-ai-training) in 2020. Compared to A100's debut scores, H100 delivered 6.7 times the speed. But compared to A100 with today's software, the gain is only 2.6-fold.

In a way, H100 seems a bit over powered for the MLPerf benchmarks, tearing through most of them in minutes using a fraction of the A100 hardware needed to match it. And in truth, it is meant for bigger things. "H100 is our solution for the most advanced models where we get into the millions, even billions of hyperparameters," says Salvator.

Salvator says a lot of the gain is from the H100's "transformer engine." Essentially, it's the intelligent use of low-precision—efficient but less accurate—computations whenever possible. The scheme is particularly designed for neural networks called transformers, of which the natural language processing benchmark [BERT](https://en.wikipedia.org/wiki/BERT_(language_model)) is an example. Transformers are in the works for many other machine learning tasks. "Transformer-based networks have been literally transformative to AI," says Salvator. "It's a horrible pun."

Memory is a bottleneck for all sorts of AI, but it's particularly limiting in BERT and other transformer models. Such neural networks rely on a quality called "attention." You can think of it as how many words a language processor is aware of at once. It doesn't scale up well, largely because it leads to a huge increase in writing to system memory. Earlier this year [Hazy Research](https://hazyresearch.stanford.edu/) (the name for [Chris Re's](https://cs.stanford.edu/~chrismre/) lab at Stanford) deployed an algorithm to an Azure cloud system that shaved 10 percent of the training time off Microsoft's best effort. For this round, Azure and Hazy Research worked together to demonstrate the algorithm—called Flash Attention.

Both the image classification and natural language processing tables show Intel's competitive position. The company showed results for the [Habana Gaudi2](https://www.intel.com/content/www/us/en/newsroom/news/vision-2022-habana-gaudi2-greco.html#gs.igsnaq), its second generation AI accelerator, and the Sapphire Rapids Xeon CPU, which will be commercially available in the coming months. For the latter, the company was out to prove that you can do a lot of machine learning training without a GPU.

A setup with 32 CPUs landed well behind a Microsoft Azure cloud-based system with only 4 GPUs on object recognition, but it still finished in less than an hour and a half, and for natural language processing, it nearly matched that Azure system. In fact, none of the training took longer than 90 minutes, even on much more modest CPU-only computers.

“This is for customers for whom training is part of the workload, but it’s not _the_ workload,” says Jordan Plawner, senior director and AI product manager. Intel is reasoning that if a customer is only retraining once a week, whether the work takes 30 minutes or 5 minutes is of too little importance for them to spend on a GPU accelerator they don’t need for the rest of the week.

Habana Gaudi2 is a different story. As the company’s dedicated machine learning accelerator, the 7-nanometer chip goes up against Nvidia’s A100 (another 7-nm chip) and soon will face the 5-nanometer H100. In that light, it performed well on certain tests. On image classification, an 8-chip system landed only a couple of minutes behind an 8-chip H100. But the gap was much wider with the H100 at the natural language processing task, though it still narrowly bested an equal-sized and Hazy-Research-enhanced A100 system.

“We’re not done with Gaudi 2,” says Habana's Eitan Medina. Like others, Habana is hoping to speed learning by strategically using low-precision computations on certain layers of neural networks. The chip has 8-bit floating point capabilities, but so far the smallest precision the company has engaged on the chip for MLPerf training purposes is [bfloat 16](https://en.wikipedia.org/wiki/Bfloat16_floating-point_format).

  

Training Supercomputers
-----------------------

MLCommons released results for training high-performance computers—supercomputers and other big systems—at the same time as those for training servers. The HPC benchmarks are not as established and have fewer participants, but they still give a snapshot of how machine learning is done in the supercomputing space and what the goals are. There are three benchmarks: CosmoFlow estimates physical quantities from cosmological image data; DeepCAM spots hurricanes and atmospheric rivers in climate simulation data; and OpenCatalyst predicts the energy levels of molecular configurations.

There are two ways to measure systems on these benchmarks. One is run a number of instances of the same neural network on the supercomputer, and the other is to just throw a bunch of resources at a single instance of the problem and see how long it takes. The table below is the latter and just for CosmoFlow, because it's much simpler to read. (Again, feel free to view the whole schemozzle at MLCommons.)

The CosmoFlow results show four supercomputers powered by as many different types of CPU architectures and two types of GPU. Three of the four were accelerated by Nvidia GPUs, but Fugaku, the [second most powerful computer in the world](https://spectrum.ieee.org/frontier-exascale-top500-export-controls), used only its own custom-built processor, the Fujitsu A64FX.

The MLPerf HPC benchmarks came out only the week before [Supercomputing 2022,](https://sc22.supercomputing.org/) in Dallas, Tex., one of the two conferences at which new Top500 rankings of supercomputers are announced.

A separate benchmark for supercomputing AI has also been developed. Instead of training particular neural networks it solves "a system of linear equations using novel, mixed-precision algorithms that exploit modern hardware". Although results from the two benchmarks don't line up, there is overlap between the [HPL-MxP list](https://hpl-mxp.org/results.md) and the CosmoFlow results including: Nvidia's Selene, Riken's Fugaku, and Germany's JUWELS.

  

---

Tiny ML systems
---------------

The latest addition to the MLPerf effort is a suite of benchmarks designed to test the speed and energy efficiency of microcontrollers and other small chips that execute neural networks that do things like spotting keywords and other low-power, always-on tasks. MLPerf Tiny, as it's called, is too new for real trends to have emerged in the data. But the results released so far show a couple of standouts. The table here shows the fastest "visual wakewords" results for each type of processor, and shows that [Syntiant](https://www.syntiant.com/) and [Greenwave Technologies](https://greenwaves-technologies.com/) have an edge over the competition.

New Records for the Biggest and Smallest AI Computers

Metamaterials Could Solve One of 6G’s Big Problems

[Apple](https://www.apple.com/) once again surprised enthusiasts and analysts with [its announcement of the M1 Ultra](https://www.apple.com/newsroom/2022/03/apple-unveils-m1-ultra-the-worlds-most-powerful-chip-for-a-personal-computer/), a variant of the M1 Max that effectively fuses two chips into one. The result is a dual-chip design viewed by software as a single piece of silicon. [Nvidia](https://www.nvidia.com/en-us/) delivered similar news at [GPU Technology Conference 2022](https://www.nvidia.com/gtc/), where CEO [Jensen Huang](https://en.wikipedia.org/wiki/Jensen_Huang) announced the company will fuse two of the company’s new Grace CPU processors into a single “superchip.”

These announcements target different markets. Apple has its sights set on the consumer and professional workstation world while Nvidia intends to compete in high performance computing. Yet the divergence in purpose only underscores the broad challenges rapidly bringing the era of monolithic chip design to an end.

Multi-chip design isn’t new, but the idea has surged in popularity in the last five years. [AMD](https://www.amd.com/en), Apple, [Intel](https://www.intel.com/content/www/us/en/homepage.html), and Nvidia have all dabbled to varying degrees. [AMD has pursued chiplet design](https://ieeexplore.ieee.org/document/9499852) with its EPYC and RYZEN processors. [Intel plans to follow suit with Sapphire Rapids](https://youtu.be/3zyQNiJw82U?t=253), an upcoming architecture for the server market built on the use of chiplets it calls “tiles.” Now Apple and Nvidia have joined the party—though with designs that target significantly different markets.  

![A black board with two colorful side by side chips.](https://spectrum.ieee.org/media-library/a-black-board-with-two-colorful-side-by-side-chips.jpg?id=29633897&width=980) Nvidia's Grace CPU SuperchipNvidia

The shift towards multi-chip design is driven by the challenge of modern chip manufacturing. [Miniaturization of transistors has slowed](https://spectrum.ieee.org/transistors-could-stop-shrinking-in-2021), yet growth in transistor counts in leading-edge designs shows no sign of slowing.

Apple’s M1 Ultra has 114 billion transistors and a die area (or fabrication area) of roughly 860 square millimeters (an official figure for the M1 Ultra is unavailable, but a single M1 Max chip has a die area of 432 square millimeters). The transistor count of Nvidia’s Grace CPU is still under wraps, but the Hopper H100 GPU announced alongside the Grace CPU includes 80 billion transistors. For perspective, AMD’s 64-core EYPC Rome processor, released in 2019, [has 39.5 billion transistors](https://www.tomshardware.com/news/amd-64-core-epyc-cpu-die-design-architecture-ryzen-3000).

Transistor counts this high push modern chip production to its extremes, making multi-chip design more attractive. “Multi-chip module packaging has enabled the chip players to give better power efficiency and performance \[with regard\] to monolithic designs as the die size for chips becomes larger and wafer yield issues become more prominent,” [Akshara Bassi](https://www.counterpointresearch.com/author/akshara/), a research analyst at Counterpoint Research, said in an email. [Aside from Cerebras](https://spectrum.ieee.org/4-things-to-know-about-the-biggest-chip-ever-built), a startup attempting to build chips that span the entirety of a silicon wafer, the chip industry seems in agreement that monolithic design is becoming more trouble than it's worth.

This shift towards chiplets has occurred in step with support from manufacturers. [TSMC](https://www.tsmc.com/english) is a leader, offering [a suite of advanced packaging called 3DFabric](https://3dfabric.tsmc.com/english/dedicatedFoundry/technology/3DFabric.htm). Technologies that fall under the umbrella of 3DFabric are used by AMD in some EPYC and RYZEN processor designs and are almost certainly used by Apple for M1 Ultra (Apple has not confirmed this, but the M1 Ultra is produced by TSMC). Intel has its own packaging technologies, such as [EMIB](https://www.intel.com/content/www/us/en/silicon-innovations/6-pillars/emib.html) and [Foveros](https://www.intel.com/content/www/us/en/corporate/usa-chipmaking/news-and-resources/video-intel-foveros-technology-explained.html). Though originally meant for Intel’s own use, the company’s chip manufacturing technology is becoming relevant to the broader industry as [Intel Foundry Services](https://www.intel.com/content/www/us/en/foundry/intel-foundry-services.html) opens its doors.

“The ecosystem around the foundational semiconductor design, manufacturing, and packaging has progressed to the point of supporting the design nodes to economically and reliably produce chiplet-based solutions,” [Mark Nossokoff](https://hyperionresearch.com/hpc-industry-research-team/nossokoff-mark/), a senior analyst at Hyperion Research, said in an email. “The software design tools to seamlessly integrate the various chiplets’ functionality have also matured to optimize targeted solution performance.”

Chiplets are here to stay but, for the moment, it's a world of silos. AMD, Apple, Intel, and Nvidia are using their own interconnect designs meant for specific packaging technologies.

[Universal Chiplet Interconnection Express](https://www.uciexpress.org/_files/ugd/0c1418_c5970a68ab214ffc97fab16d11581449.pdf) hopes to bring the industry together. Announced on 2 March, 2022, this open standard offers a “standard” 2D package that targets “cost-effective performance” and an “advanced” package that targets leading-edge designs. UCIe also supports off-package connection through PCIe and CXL, opening up the potential for connecting multiple chips across multiple machines in a high-performance compute environment.

![Two diagrams for connecting chiplets together, the left labeled "Layering with UCIe" and the right labeled "Packaging Options: 2D and 2.5D"](https://spectrum.ieee.org/media-library/two-diagrams-for-connecting-chiplets-together-the-left-labeled-layering-with-ucie-and-the-right-labeled-packaging-options-2d-and-2-5d.jpg?id=29633917&width=980) An example of UCIe packaging options from the UCIe whitepaper.Universal Chiplet Interconnection Express

UCIe is a start, but the standard’s future remains to be seen. “The founding members of initial UCIe promoters represent an impressive list of contributors across a broad range of technology design and manufacturing areas, including the HPC ecosystem,” said Nossokoff, “but a number of major organizations have not as yet joined including Apple, AWS, Broadcom, IBM, NVIDIA, other silicon foundries, and memory vendors.”

Bassi points out Nvidia may be particularly reluctant to participate. [The company has opened its own NVLink-C2C interconnect up for custom silicon integration](https://nvidianews.nvidia.com/news/nvidia-opens-nvlink-for-custom-silicon-integration#:~:text=NVIDIA%20NVLink%2DC2C%20is%20the,level%20integration%20with%20NVIDIA%20technology.), making it a potential competitor for UCIe.

But while the fate of interconnects like UCIe and NVLink-C2C will determine the rules of the game, they’re unlikely to change the game being played. Apple’s M1 Ultra could be considered the canary in the coal mine. Multi-chip design is no longer reserved to data centers—it's coming to a home computer near you.

Single-Chip Processors Have Reached Their Limits

PCs have long come with the option to add more memory, giving extra-large applications and data-heavy work greater speed. Thanks to 3D chip stacking, AMD’s next generation CPU comes with that option, too. It’s not an aftermarket add on, of course, but if you’re looking to build a computer with some extra oomph, ordering up a processor with an extra-large cache memory could be the way to go.

The 3D stacking technique tripled the Zen 3 processor core’s cache memory, speeding its execution of games and electronic design automation. [John Wuu AMD senior fellow](https://www.linkedin.com/in/john-wuu-15b0b736/) design engineer detailed the previously announced technology for virtual attendees of the [IEEE International Solid State Circuits Conference](https://www.isscc.org/) in February.

Even though [Zen 2](https://www.amd.com/en/technologies/zen-core) and the new [Zen 3](https://www.amd.com/en/technologies/zen-core-3) processor cores are both made using the same TSMC manufacturing process (N7)—and therefore have the same size transistors, interconnects, and everything else—AMD made so many architectural alterations that even without the extra cache memory, Zen 3 provides 19 percent performance improvement on average. One of those architectural gems was the inclusion of a set of set of through-silicon vias (TSVs), vertical interconnects that burrow straight down through most of the silicon. The TSVs are built within Zen 3’s highest level cache, blocks of SRAM called L3, which is sits in the middle of the 81-square-millimeter compute chiplet and is shared across all eight of its cores.

In processors destined for data-heavy workloads, the Zen 3 wafer’s back side is thinned down until the TSVs are exposed. Then a 64-megabyte SRAM chiplet is bonded to those exposed TSVs using what’s called hybrid bonding—a process that’s like cold-welding the copper together. The result is a dense set of connections that can be as close together as 9 micrometers. Finally, for structural stability and heat conduction, two blank silicon chiplets are attached to cover the remainder of the Zen 3 CPU die.

Adding the extra memory by setting it beside the CPU die was not an option, because data would take too long to get to the processor cores. “Despite tripling the L3 \[cache\] size, 3D V-Cache only added four \[clock\] cycles of latency—something that could only be achieved through 3D stacking,” [John Wuu AMD senior fellow](https://www.linkedin.com/in/john-wuu-15b0b736/) design engineer told attendees of the [IEEE International Solid State Circuits Conference](https://www.isscc.org/).

The bigger cache made its mark in high end games. Using the desktop Ryzen CPU with 3D V-Cache sped games rendered at 1080p by an average of 15 percent. It was good for more serious work as well, shortening the runtime for a difficult semiconductor design computations by 66 percent.

The industry’s ability to shrink SRAM is slowing compared to how well it can shrink logic, Wuu pointed out. So you can probably expect future SRAM expansion packs to continue to be made using more established manufacturing processes while the compute chiplets are driven down to Moore’s Law’s bleeding edge.

AMD's 3D Memory-Extender Speeds Gaming

UK-based AI computer company [Graphcore](https://www.graphcore.ai/) made a significant boost to its computers' performance without changing much of anything about its specialized AI processor cores. The secret was to use [TSMC's wafer-on-wafer](https://3dfabric.tsmc.com/english/dedicatedFoundry/technology/SoIC.htm#SoIC_WoW) 3D integration technology during manufacture to attach a power-delivery chip to Graphcore's AI processor.

The new combined chip, called Bow for a district in London, is the first on the market to use wafer-on-wafer bonding, say Graphcore executives. The addition of the power-delivery silicon means Bow can run faster—1.85 gigahertz versus 1.35 GHz—and at lower voltage than its predecessor. That translates to computers that train neural nets up to 40 percent faster with as much as 16 percent less energy compared to its previous generation. Importantly, users get this improvement with no change to their software at all.

"We are entering an era of advanced packaging in which multiple silicon die are going to be assembled together to supplement the performance advantages we can get from increasing progress along an ever-slowing Moore's Law path," says [Simon Knowles](https://www.graphcore.ai/simon-knowles), Graphcore CTO and cofounder. Both Bow and its predecessor the Colossus MK2 were made using the same manufacturing technology, TSMC's N7.

![A bar chart with a black line through it.](https://spectrum.ieee.org/media-library/a-bar-chart-with-a-black-line-through-it.jpg?id=29453180&width=980) Compared to Graphcore's previous generation, the new computers can train key neural networks about 40 percent faster.Graphcore

![A bar chart.](https://spectrum.ieee.org/media-library/a-bar-chart.jpg?id=29453179&width=980) The new systems are up to 16 percent more efficient at training key neural networks.Graphcore

In other 3D chip stacking technology, such as [Intel's Foveros](https://spectrum.ieee.org/intel-s-exascale-supercomputer-chip-is-a-master-class-in-3d-integration), already excised chips are attached to other chips or to wafers. In [TSMC's SoIC WoW](https://3dfabric.tsmc.com/english/dedicatedFoundry/technology/SoIC_inDepth.htm) technology, two entire wafers of chips are bonded. The chips on each have copper pads that match up when the wafers are aligned. When the two wafers are pressed together, the pads fuse. "You can think of this as a kind of cold weld between the pads," says Knowles. The top wafer is then thinned down to just a few micrometers and the bonded wafer is diced up into chips.

In Graphcore's case, one wafer is full of the company's second generation AI processor (the company calls them IPUs for intelligence processing units) with 1472 IPU cores and 900 megabytes of on-chip memory. These processors were already in use in commercial systems and made a [good showing in the last round of MLPerf benchmark tests](https://spectrum.ieee.org/ai-training-mlperf). The other wafer had a corresponding set of power delivery chips. These chips carry no transistors or other active components. Instead they are packed with capacitors and vertical connections called through-silicon-vias. The latter make power and data connections that pass through the power chip to the processor die.

It's the capacitors that really make the difference. These components are formed in deep, narrow trenches in the silicon, exactly like the bit-storing capacitors in DRAM. By placing these reservoirs of charge so close to the transistors, power delivery is smoothed out, allowing the IPU cores to run faster at lower voltage. Without the power delivery chip, the IPU would have to increase its operating voltage above its nominal level to work at 1.85 GHz, consuming a lot more power. With the power chip, it can reach that clock-rate and consume less power, too.

Graphcore executives say wafer-on-wafer technology results in a higher density of connections between the chips than attaching individual chips to a wafer. However, one long-standing concern with this technique was the "known-good-die" problem. That is, there are always a few chips in a batch of wafers that are flawed. Bonding two wafers would then as much as double the resulting number of flawed chips. Graphcore's way around this is to let it happen, to a degree. Like some other new AI processors, the IPU is made up of many repeated, and therefore redundant, processor cores and other parts. Any duds can be cut-off from the rest of the IPU by means of built-in fuses, says [Nigel Toon](https://www.graphcore.ai/nigel-toon), Graphcore co-founder and CEO.

![A stack of rectangles is shot through with gaps of various sizes. The stack is topped with a series of circles.](https://spectrum.ieee.org/media-library/a-stack-of-rectangles-is-shot-through-with-gaps-of-various-sizes-the-stack-is-topped-with-a-series-of-circles.jpg?id=29453207&width=980) TSMC's wafer-on-wafer stacking results in a processor chip \[bottom\] connected to a power delivery chip \[top\] by copper pads \[yellow\]. Signals and power pass through the top chip from solder bumps \[grey\].Graphcore

Although the new product has no transistors on the power delivery chip, those might be coming. Using the technology only for power delivery "is just the first step for us," says Knowles. "It will go much further than that in the near future."

Graphcore revealed some plans for that near future, announcing that it will build supercomputers that can train "brain-scale" AIs—those having hundreds of trillions of parameters in a neural network. The "Good" computer, named in honor of British mathematician [I. J. "Jack" Good](https://en.wikipedia.org/wiki/I._J._Good), would be capable of more than 10 exaflops—10 billion billion floating point operations. Good would be made up of 512 systems with 8192 IPUs along with mass storage, CPUs, and networking. It will have 4 petabytes of memory and a bandwidth of more than 10 PB per second. Graphcore estimates each supercomputer will cost about $120 million and should be ready for delivery in 2024.

"When we started Graphcore... the idea has always been in the back of our mind to build an ultra-intelligent computer that would surpass the capability of a human brain," says Toon. "And that is what we are now working on."

Competitor [Cerebras Systems](https://spectrum.ieee.org/cerebras-giant-ai-chip-now-has-a-trillions-more-transistors) already planted its flag in the [quest for brain-scale AI](https://spectrum.ieee.org/cerebras-ai-computers). It developed an external memory system and a way to connect multiple computers that would allow its computers to train neural networks with hundreds of trillions of parameters.

Graphcore Uses TSMC 3D Chip Tech to Speed AI by 40%

First released in 2005, the Arduino microcontroller turbo charged the maker movement. While more advanced boards have joined the Arduino line since, the [basic Uno microcontroller](https://store-usa.arduino.cc/products/arduino-uno-rev3/) remains popular. Inexpensive, simple to program, and robust enough for beginners while still having enough capacity to do interesting things for more advanced users, it forms the basis of countless projects. Now one of [Arduino's original creators, Gianluca Martino](https://spectrum.ieee.org/the-making-of-arduino), has developed a new IoT-focused module that is designed as a drop-in replacement for the Uno's standard ATMega328P processor. Dubbed the Jolly, it integrates an [ESP8285](https://www.espressif.com/sites/default/files/documentation/0a-esp8285_datasheet_en.pdf) WiFi adapter and [ATMega328PB](https://www.microchip.com/en-us/product/ATMEGA328PB) in the same footprint as the original processor, while retaining complete backward software and hardware compatibility. Martino is [launching a Kickstarter campaign today](https://www.kickstarter.com/projects/1760898757/jolly-module) to bring it to market. _IEEE Spectrum_ spoke to Martino about what the new module bring to the world of Arduino.  
  
**What was the motivation behind creating the Jolly?**

**Gianluca Martino:** After I exited the Arduino \[company\], I was doing some consultancies for old friends. And something that people kept asking me was to have simple way to combine Arduino and the Internet of Things.

![A white man with grey hair and glasses sits in front of a workbench covered with electronics development equipment](https://spectrum.ieee.org/media-library/a-white-man-with-grey-hair-and-glasses-sits-in-front-of-a-workbench-covered-with-electronics-development-equipment.jpg?id=29452765&width=980) Gianluca MartinoJollu/Gianluca Martino

**There are add-on "shields" already available that let people add WiFi capabilities to Uno boards. What's special about the Jolly?**

**Martino:** This frees up the connectors a shield would use, and it saves space. It's a more integrated solution. It will allow you to use other shields without consuming the I/O and the other pins a shield would need, because the Jolly has all the communications between the ESP device and the processor be internal \[using SPI and UART interfaces\]. The ATMega variant we chose has four extra pins we use for that.

**Can the ESP processor be programmed independently?**

**Martino:** Yes, you can download your own boot \[firmware\]. As the Arduino ecosystem is for beginners and intermediate makers, we provide a restart from the beginning function if you make something that locks it up. The ESP chip is something for the advanced user, but there is documentation, there is firmware.

**The ESP has two megabytes of flash memory. Is that accessible from the ATMega side?**

**Martino:** In the basic version, no. We will think to make it transparent for some users, but we are trying to keep the two subsystems separate. An Arduino Uno is often for someone that needs something quick and dirty ... so we have to make it simple and not to make it super advanced with super features that will require a lot of maintenance.  
  
**With an integrated design that has to squeeze everything into the same footprint as the original ATMega processor, how did you approach designing the Jolly's radio antenna?**  
  
**Martino:** We had to find a compromise, because the position is not the best. Maybe in the future, we will add a connector to use an external antenna. But we have to remember this is not something for an industrial application. As usual, as we have in the past, if there is a customer that need a a special application customization, we will evaluate the possibility.

**Simplicity is not always easy with IoT projects, it can get complicated very quickly if you have to start registering for online services and so on. How are you making it easy for someone who wants to, say, blink an LED, but control it remotely?**

**Martino:** We would like to develop equivalent Arduino functions to make things transparent once connected, either peer to peer or over IP. We would like to hear from the community what they need.  

**We're in the middle of a [global semiconductor shortage](https://spectrum.ieee.org/search/?q=semiconductor+shortage). How has that affected the launch of the Jolly?**

**Martino****:** We were lucky that we started this project some time ago, and so we have in stock some parts. We are ready at least for the first 1,000 pieces to go on the market quickly. ... Clearly, we are living in a period of shortage ... the extra cost is something we will initially absorb because we don't want to transfer this to the customer. We know that we are approaching people \[with the pitch\] that it is an upgrade, and a small piece of extra hardware is not to be more expensive than the base hardware.

Arduino Co-Founder's Uno Upgrade

AT&T [shut down its 3G network](https://www.att.com/support/article/wireless/KM1324171/) in the U.S. on 22 February—a process called ‘sunsetting.’ It’s the first of the three major carriers in the country to do so. T-Mobile will be [shutting down its 3G network](https://www.t-mobile.com/support/coverage/t-mobile-network-evolution) on 1 July, while Verizon has announced it will be [doing the same](https://www.verizon.com/about/news/3g-cdma-network-shut-date-set-december-31-2022) on 31 December.

All three companies have cited the ongoing development of 4G and especially 5G networks as a major reason for the shutdowns. 5G in particular is much more spectrum-hungry than earlier generations of wireless. Shutting down 3G networks opens up valuable bandwidth to improve 5G coverage and performance.

Shutting down a nationwide wireless network isn’t as simple as flipping a big, Frankenstein-style lever from the ‘on’ to the ‘off’ position. [Sundeep Rangan](https://wireless.engineering.nyu.edu/sundeep-rangan/), the associate director of NYU Wireless, says that, generally speaking, sunsetting a wireless network requires going site by site to each cell tower to shut off any 3G equipment. Then that equipment can either be dismantled or repurposed for 4G and 5G networks.

Crucially, Rangan explains, that process doesn’t need to happen overnight. It might take days, weeks, or even longer for AT&T to make those network changes. In some locations, 3G service might even still be available on a site-by-site basis as AT&T workers make their way to each tower. The sunset date specifically indicates the day on which AT&T is no longer contractually obligated to offer nationwide 3G service. AT&T did not respond to a request for comment on the specifics of its sunsetting process.

AT&T rolled out its initial 3G networks in four cities [in June 2004](https://www.computerworld.com/article/2566481/at-t-wireless-launches-3g-in-four-cities.html)—nearly 18 years ago. That’s resulted in 3G ultimately having a shorter lifespan than its predecessor 2G networks that were deployed in the early 1990s. AT&T sunset its 2G network in 2017; T-Mobile’s is still active (the company will sunset its 2G network this December); and Verizon’s was operational until the end of 2020. All told, 2G networks in the United States were in use for nearly 30 years in some instances.

Companies are sunsetting their 3G networks after shorter yenures, speculates Rangan, because the bandwidth they use is extremely valuable and crucial to their plans for future 5G deployments. 5G continues to demonstrate that it will require an immense amount of spectrum for its data-intensive, low-latency applications. Any frequencies that AT&T and the other companies can free up from lesser-used 3G networks can be redirected toward those needs.

But “lesser used” doesn’t mean “not used at all.” AT&T itself estimated in September that [2.7 million customers were still reliant on its 3G networks](https://www.lightreading.com/5g/atandt-27-of-customers-will-be-affected-by-3g-shutdown/d/d-id/772115)—about 2.7 percent of its postpaid and prepaid customers.

The most obvious way in which customers—whether they use AT&T, Verizon, or T-Mobile—can still be reliant on 3G networks is to still own a 3G smartphone. However, that would only include people who haven’t bought a new phone since the first 4G phones debuted in 2008. Once 4G modems were available, it would have been rare for a company to manufacture a phone without one, and that would have been truer with each passing year. But inevitably, there are still people out there who have never needed or wanted to upgrade their phones.

To be clear, not all of those customers are carrying around 3G cellphones in their pockets. They very well may be driving around in cars using 3G-connected electronics. Plenty of cars—[even ones manufactured as late as the 2021 model year](https://www.thedrive.com/tech/43187/how-the-3g-shutdown-in-2022-could-screw-your-car)—rely on 3G networks for navigation and location data, emergency calls, remote lock functions, and more. Unless car manufacturers have taken it upon themselves to upgrade car systems ([some have](https://static.nhtsa.gov/odi/tsbs/2021/MC-10189606-0001.pdf)), cars that use 3G for these applications simply won’t have those features anymore.

Elsewhere, the American Association of School Administrators filed a petition to the U.S. Federal Communications Commission (FCC) to delay AT&T’s 3G sunset because [up to 10 percent of public school buses](https://ecfsapi.fcc.gov/file/1021578125962/NAPT%20AASA%20ATT%20Delay%2021-304%20021422.pdf) in the country rely on 3G connections for GPS and communications.

The Alarm Industry Communications Committee and American Association of Retired Persons also raised concerns to the FCC over the number of security systems and medical alert devices, respectively, that use 3G networks. They’ve argued that the ongoing covid pandemic and semiconductor shortages have made it difficult for people to upgrade or replace their devices. The 3G sunset, the groups say, will leave people in danger if security or alert devices are unable to send notifications when there’s trouble. In its response to the FCC, AT&T said that because the company first announced its 3G sunset three years earlier, companies and customers had ample time to make upgrades.

AT&T’s shutdown of its 3G network is being felt more keenly in these industries for the same reason the company wanted to shut down the network in the first place: Not many cellphone users needed it. 3G network speeds are capped at about 2 megabits per second, which is more than enough for the kinds of data connected cars, alarm systems, and medical alert devices send and receive. Using 3G networks made it cheaper for these devices to operate because they could avoid the pricier 4G networks. 2G’s sunsetting faced a similar problem, as hordes of IoT devices with low data rates still relied on those incumbent networks.

While 3G may not have lasted as long as 2G, it was a watershed moment for cellular technology. According to NYU’s Rangan, it heralded the shift from using cellular technologies strictly for voice to using them for data. With 3G, we all got our first glimpse at what we now take for granted: using our phones to browse the Internet, stream video, and more. But that also made working on 3G a tricky prospect, as it changed so much over its lifespan. “I think a lot of people will actually be okay, probably happy,” says Rangan. “Engineers did a fantastic job and put a lot of effort into it. It was a beast.”

3G's Sunset Takes Aging Cars and Medical Devices With It

On Monday, [Intel](https://www.intel.com/content/www/us/en/high-performance-computing/supercomputing/exascale-computing.html) unveiled new details of the processor that will power the [Aurora supercomputer](https://www.alcf.anl.gov/aurora), which is designed to become the one of the first U.S.-based machine to pierce the [exaflop](https://spectrum.ieee.org/exascale-supercomputing) barrier—a billion billion high-precision floating point calculating per second. Intel Fellow [Wilfred Gomes](https://www.linkedin.com/in/wilfredgomes/) told engineers virtually attending the [IEEE International Solid State Circuits Conference](https://www.isscc.org/) this week that the processor pushed [Intel's 2D and 3D chiplet integration technologies](https://spectrum.ieee.org/intel-shows-off-chip-packaging-powers) to the limits.

The processor, called Ponte Vecchio, is a package that combines multiple compute, cache, networking, and memory silicon tiles, or "chiplets". Each of the tiles in the package is made using different process technologies, in a stark example of a trend called [heterogenous integration](https://www.semi.org/en/communities/hir).  

Ponte Vecchio is, among other features, a master class in 3D integration.

The result is that Intel packed 3,100 square millimeters of silicon—nearly equal to four [Nvidia A100 GPU](https://www.nvidia.com/en-us/data-center/a100/)s—into a 2,330 mm2 footprint. That's more than 100 billion transistors across 47 pieces of silicon.

  

![An illustration shows groupings of labelled blue, dark grey, and tan tiles on the left and their positions on a green circuitboard on the right.](https://spectrum.ieee.org/media-library/an-illustration-shows-groupings-of-labelled-blue-dark-grey-and-tan-tiles-on-the-left-and-their-positions-on-a-green-circuitboard-on-the-right.jpg?id=29420969&width=980) Ponte Vecchio is made of multiple compute, cache, I/O, and memory tiles connected using 3D and 2D technology.Source: Intel

Ponte Vecchio is, among other features, a master class in 3D integration. Each Ponte Vecchio processor is really two mirror image sets of [chiplets](https://spectrum.ieee.org/tag/chiplets) tied together using Intel's 2D integration technology Co-EMIB. Co-EMIB forms a bridge of high-density interconnects between two 3D stacks of chiplets. The bridge itself is a small piece of silicon embedded in a package’s organic substrate. The interconnect lines on silicon can be made narrower than on the organic substrate. Ponte Vecchio's ordinary connections to the package substrate were 100 micrometers apart, whereas they were nearly twice as dense in the Co-EMIB chip. Co-EMIB dies also connect high-bandwidth memory (HBM) and the Xe Link I/O chiplet to the "base silicon", the largest chiplet, upon which others are stacked.

![An illustration shows an exploded view of the parts of a processor each represented by different colored rectangles.](https://spectrum.ieee.org/media-library/an-illustration-shows-an-exploded-view-of-the-parts-of-a-processor-each-represented-by-different-colored-rectangles.jpg?id=29421235&width=980) The parts of Ponte Vecchio.Source: Intel

Each set of eight compute tiles, four SRAM cache chiplets called RAMBO tiles, and eight blank "thermal" tiles meant to remove heat from the processor is connected vertically to a base tile. This base provides cache memory and a network that allows any compute tile to access any memory.

Notably, these tiles are made using different manufacturing technologies, according to what suited their performance requirements and yield. The latter term, the fraction of usable chips per wafer, is particularly important in a chiplet integration like Ponte Vecchio, because attaching bad tiles to good ones means you've wasted a lot of expensive silicon. The compute tiles needed top performance, so they were made using TSMC's N5 (often called a 5-nanometer) process. The RAMBO tile and the base tile both used Intel 7 (often called a 7-nanometer) process. HBM, a 3D stack of DRAM, uses a completely different process than the logic technology of the other chiplets, and the Xe Link tile was made using TSMC's N7 process.

![An illustration shows a cut-through of a processor with large blue, black, and grey rectangles representing the silicon parts and copper showing the connections.](https://spectrum.ieee.org/media-library/an-illustration-shows-a-cut-through-of-a-processor-with-large-blue-black-and-grey-rectangles-representing-the-silicon-parts-and-copper-showing-the-connections.jpg?id=29421255&width=980) The different parts of the processor are made using different manufacturing processes, such as Intel 7 and TSMC N5. Intel's FOVEROS technology creates the 3D interconnects and it's Co-EMIB makes horizontal connections.Source: Intel

The base die also used Intel's 3D stacking technology, called FOVEROS. The technology makes dense array of die-to-die vertical connections between two chips. The connections are just 36 micrometers apart and are made by connecting the chips "face to face", that is the top of one chip is bonded to the top of the other. Signals and power get into this stack by means of [through-silicon vias](https://en.wikipedia.org/wiki/Through-silicon_via), fairly wide vertical interconnects that cut right through the bulk of the silicon. The FOVEROS technology used on Ponte Vecchio is an improvement over the one used to make Intel's [Lakefield mobile processor](https://spectrum.ieee.org/3-ways-chiplets-are-remaking-processors), doubling the density of signal connections.

Expect the “zettascale” era of supercomputers to kick off sometime around 2028.

Needless to say, none of this was easy. It took innovations in yield, clock circuits, thermal regulation, and power delivery, Gomes said. In order to ramp performance up or down with need, each compute tile could run at a [different voltage](https://spectrum.ieee.org/a-circuit-to-boost-battery-life) and clock frequency. The clock signals originate in the base die but each compute tile can runs at its own rate. Providing the voltage was even more complicated. Intel engineers chose to supply the processor with a higher than normal voltage (1.8 volts) so they could simplify the package structure due to the lower current needs. Circuits in the base tile reduce the voltage to something closer to 0.7 volts for use on the compute tiles. So each compute tile had to have its own power domain in the base tile. Key to this were new high efficiency inductors called coaxial magnetic integrated inductors. Because these are built into the package substrate, the circuit actually snakes back and forth between the base tile and the package before supplying the voltage to the compute tile.

  

![A micrograph at left shows grey and white layers of the processor calling out parts that control the flow of heat. The same parts are illustrate on the right.](https://spectrum.ieee.org/media-library/a-micrograph-at-left-shows-grey-and-white-layers-of-the-processor-calling-out-parts-that-control-the-flow-of-heat-the-same-parts-are-illustrate-on-the-right.jpg?id=29421284&width=980) Getting the heat out of a complex 3D stack of chips was no easy feat.Source: Intel

  

Ponte Vecchio is meant to draw 600 watts, so making sure heat could be extracted from the 3D stack was always a high priority. Intel engineers used tiles that had no other function than to draw heat away from the active chiplets in the design. They also coated the top of the top of the entire chiplet agglomeration in heat-conducting metal, despite the various parts having different heights. Atop that was a solder-based thermal interface material (STIM), and an integrated heat spreader. The different tiles each have different operating temperature limits under liquid cooling and air cooling, yet this solution managed to keep them all in range, said Gomes.

"Ponte Vecchio started with a vision that we wanted democratize computing and bring petaflops to the mainstream," said Gomes. Each Ponte Vecchio system is capable of more than 45 trillion 32-floating point operations per second (teraflops). Four such systems fit together with two Sapphire Rapids CPUs in a complete compute system. These will be combined for a total exceeding 54,000 Ponte Vecchios and 18,000 Sapphire Rapids to form Aurora, a machine targeting 2 exaflops.

It's taken 14 years to go from the first petaflop supercomputers in 2008—capable of one million billion calculations per second—to exaflops today, Gomes pointed out. A 1000-fold increase in performance "is a really difficult task, and it's taken multiple innovations across many fields," he said. But with improvements in manufacturing processes, packaging, power delivery, memory, thermal control, and processor architecture, Gomes told engineers, the next 1000-fold increase could be accomplished in just six years rather than another 14.

Behind Intel’s HPC Chip that Will Break the Exascale Barrier

![](https://spectrum.ieee.org/media-library/risc-v-chip.jpg?id=29412668&width=1200&coordinates=0%2C106%2C0%2C106&height=800)  
  

The adoption of RISC-V, a free and open-source computer instruction-set architecture first introduced in 2010, is taking off like a rocket. And much of the fuel for this rocket is coming from demand for AI and machine learning. According to the research firm [Semico](https://semico.com/content/analyzing-risc-v-cpu-market-sip-socs-ai-and-design-starts), the number of chips that include at least some RISC-V technology will grow 73.6 percent per year to 2027 when there will be some 25 billion AI chips produced, accounting for $291-billion in revenue.  

The increase from what was still an upstart idea just a few years ago to now is impressive, but for AI it also represents something of a sea change, says [Dave Ditzel](https://www.linkedin.com/in/dave-ditzel-7aa72b3/), whose company [Esperanto Technologies](https://www.esperanto.ai/) has created the first high-performance RISC-V AI processor intended to compete against powerful GPUs in AI recommendation systems. According to Ditzel, during the early mania for machine learning and AI, people assumed general purpose computer architectures—x86 and Arm—would never keep up with GPUs and more purpose-built accelerator architectures.

“We set out to prove all those people wrong,” he says. “RISC-V seemed like an ideal base to solve a lot of the kinds of computation people wanted to do for artificial intelligence.” With the company’s first silicon—a 1092-core AI processor—in the hands of a set of early partners and a major development deal with Intel \[see sidebar\], he might soon be proved right.

Ditzel’s entire career has been defined by the theory behind RISC-V. RISC, as you may know, stands for reduced instruction set computer. It was the idea that you could make a smaller, lower power, but better-performing processor by slimming down the core set of instructions it can execute. [IEEE Fellow David Patterson](https://www.computer.org/profiles/david-patterson), coined the term in a [seminal paper in 1980](https://dl.acm.org/doi/10.1145/641914.641917). Ditzel, his student, was the coauthor. Ditzel went on to work on RISC processors at Bell Labs and [Sun Microsystems](https://standards.ieee.org/ieee/1754/2327/) before cofounding [Transmeta](https://spectrum.ieee.org/chip-hall-of-fame-transmeta-corp-crusoe-processor), which made a low-power processor meant to compete against Intel by translating x86 code for a RISC architecture.

With Esperanto, Ditzel saw RISC-V as a way to accelerate AI with relatively low power consumption. At a basic level, a more complex instruction set architecture means you need more transistors on the silicon to make up the processor, each one leaking a bit of current when off and consuming power when it switches states. “That was what was attractive about RISC-V,” he says. “It had a simple instruction set.”

### The Core

The core of RISC-V is a set of just 47 instructions. The actual [number of x86 instructions](https://www.unomaha.edu/college-of-information-science-and-technology/research-labs/_files/enumerating-x86-64-instructions.pdf) is oddly difficult to enumerate, but it's likely near 1000. Arm's instruction set is thought to be much smaller, but still considerably larger than RISC-V's. But simply using a slim set of instructions wouldn't be enough to achieve the computing power Esperanto was aiming for, says Ditzel. "Most of the RISC-V cores out there aren't that small or that energy efficient. So it's not just a question of us taking a RISC-V core and slapping 1,000 of them on a chip. We had to completely redesign the CPU so that it would fit into those very tough constraints."

Notably missing from the RISC-V instruction set at the time Ditzel and his colleagues started work were the “vector” instructions needed to efficiently do the math of machine learning, such as matrix multiplication. So Esperanto engineers came up with their own. As embodied in the architecture of the processor core, the ET-Minion, these included units that do 8-bit integer vectors and both 32- and 16-bit floating point vectors. There are also units that do a more complex “tensor” instructions, and systems related to the efficient movement of data and instructions related to the arrangement of ET-Minion cores on the chip.

The resulting system-on-chip, the ET-SoC-1 is made up of 1088 of the ET-Minion cores along with four cores called ET-Maxions, which help govern the Minions’ work. The chip’s 24-billion transistors take up 570 square millimeters. That puts it at around half the size of the popular AI accelerator [Nvidia A100](https://www.nvidia.com/en-us/data-center/a100/). Those two chips follow very different philosophies.

The ET-SoC-1 was designed to accelerate AI in power-constrained data centers as the heart of boards that fit into the PCIe slot of already installed servers. That meant the board had only 120 watts of power available, but it would have to provide at least 100 trillion operations per second to be worthwhile. Esperanto managed more than 800 trillion in that power envelope.

Most AI accelerators are built around a single chip that uses the bulk of the board’s power budget, Esperanto.ai principal architect Jayesh Iyer told technologists at the RISC-V Summit in December. “Esperanto’s approach is to use multiple, low-power chips, which still fits within the power budget,” he said.

Each chip consumes 20 watts when performing a recommender system benchmark neural network—less than 1/10th what the A100 can draw—and there are six on the board. That combination of power and performance was achieved by reducing the chips’ operating voltages without the expected sacrifice in performance. (Generally, a higher operating voltage means you can run the chip’s clock faster and get more computing done.) At 0.75 volts, the nominal voltage for the ET-SoC-1’s manufacturing process, a single chip would blow way past the board’s power budget. But dropping the voltage to about 0.4 volts, you can run six chips from the board’s 120 W and achieve better than a four-fold boost in recommender system performance over the single higher-voltage chip. At that voltage, each ET-Minion core is consuming only about 10 milliwatts.

“Low voltage operation is the key differentiator for Esperanto’s ET-minion \[core\] design,” said Iyer. It informed architectural and circuit-level decisions, he said. For instance, the core’s pipeline for the RISC-V integer instructions is made up of the fewest number of logic gates per clock cycle, allowing a higher clock rate at the reduced voltage. And when the core is performing long tensor computations, that pipeline is shut down to save energy.

### Other AI Processors

Other recently-developed AI processors have also turned to a combination of RISC-V and their own custom machine-learning acceleration. For example, [Ceremorphic](https://ceremorphic.com/), which recently came out of stealth with its Hierarchical Learning Processor, uses both a RISC-V and an Arm core along with its own custom machine learning and floating-point arithmetic units. And Intel’s upcoming [Mobileye EyeQ Ultra](https://www.intel.com/content/www/us/en/newsroom/news/mobileye-ces-2022-tech-news.html#gs.qdh7by) will have 12 RISC-V cores with its neural network accelerators in a chip meant to provide the intelligence for Level 4 autonomous driving.

### Intel's Big Bet on RISC-V

  

In perhaps its biggest move to jump start its manufacturing business, Intel said it would create a $1-billion fund to support startups and other potential customers of [Intel Foundry Services](https://www.intel.com/content/www/us/en/foundry/intel-foundry-services.html). Though the company didn’t say where the first funds will go, in announcing the fund it the company is planning “investments and offerings that will strengthen the ecosystem and help drive further adoption of RISC-V.”

Putting its weight behind RISC-V was a smart move, because it makes them seem more like a manufacturing partner than a competitor, according to Semico Research principal analyst [Rich Wawrzyniak](https://semico.com/content/rich-wawrzyniak). “All at once they not only validated the RISC-V architecture, they said they’re open for business,” he says. “They’re not going to restrict people in terms of the CPU designs they want to build. That, itself, is probably more important than anything else.”

Intel says it will help RISC-V companies innovate faster by prioritizing their manufacturing runs and working with them co-develop manufacturing technology that works for their designs, among other things. Intel, which is now a member of RISC-V International, singled out four major collaborations with RISC-V heavyweights that it expects will attract foundry customers:

[**Andes Technology**](http://www.andestech.com/en/homepage/): The San Jose-based company designs RISC-V processor cores for embedded systems. And through the new deal, Intel Foundry Services customers can integrate its low-power cores. Intel’s foundry capacity “ensures that SoC designs based on Andes RISC-V processor cores will achieve the production ramp and volume successfully,” said CEO Frankwell Lin, in a press release.  

[**Esperanto Technologies**](https://www.esperanto.ai/): The company’s current 1092-core chip is made using TSMC’s N7 process. Through a strategic partnership, “Esperanto is getting access to Intel's advanced technology nodes,” says CEO Dave Ditzel. The company could make the next generation of its SoC using the Intel 3 process, he says. “That's not one, but actually two generations ahead of its current design.” With that, Esperanto could increase the number of cores on its chips or make those chips smaller. The latter is more likely, he says. Different numbers of 1092-core chiplets could then be combined in a package using [Intel’s chiplet packaging technologies](https://spectrum.ieee.org/intel-shows-off-chip-packaging-powers) to tackle a variety of tasks.

[**Si-Five**](https://www.sifive.com/): The Silicon Valley designer of RISC-V cores and development tools partnered with Intel Foundry Services to produce SiFive’s P550 multicore processors in the foundry’s Intel 4 manufacturing process, which is set to come online later this year. The resulting chips will go to developer boards meant to drive growth in RISC-V. It’s worth noting that SiFive extended an existing partnership with Samsung last April to integrate custom AI accelerator blocks with SiFive’s RISC-V blocks using Samsung’s foundry technology.

[**Ventana Microsystems**](https://www.ventanamicro.com/): The startup has a deal to make its data-center-class tech available to Intel Foundry Services customers. Foundry customers will be able to integrate Ventana’s data-center-class cores into their SoCs using Intel’s “leading manufacturing process” or use its multicore chiplets to rapidly make custom systems.

Turning to RISC-V was both a business and technical move for embedded AI processor firm [Kneron](https://www.kneron.com/). The company has been selling chips and intellectual property using Arm CPU cores and its custom accelerator infrastructure. But last November Kneron released its first RISC-V based tech in the [KL530](https://www.kneron.com/cn/news/blog/143/), aimed at supporting autonomous driving with a relatively new type of neural network called a vision transformer. According to Kneron CEO [Albert Liu](https://www.kneron.com/about/), the RISC-V architecture makes it easier to preprocess neural network models so they run more efficiently. However, “it also made sense in light of the potential Arm acquisition by Nvidia last year to de-risk ourselves of any possible business decisions that could impact us," he says. That deal fell apart in February but would have put the provider of Kneron’s previous CPU core architecture in the hands of a competitor.

Future RISC-V processors will be able to tackle machine-learning-related operations using an open-source set of instructions agreed upon by the community. [RISC-V International](https://riscv.org/), the body that governs the codification of the core instruction set architecture and new extensions, ratified a set of just over 100 vector instructions in December 2021.

With the new vector instructions “somebody doing their own thing in AI doesn’t have to start from scratch,” says the organization’s CTO [Mark Himelstein](https://www.linkedin.com/in/heavenstone/). “They can use the instructions that other companies are using. They can use the tools that other companies are using. And then they can innovate in the implementation or power consumption, or performance, or whatever it is their added value is.”

Even with the vector extensions, promoting machine learning remains a top priority for the RISC-V community, says Himelstein. Most of the development of ML-related extensions to RISC-V is happening in the organization’s graphics special interest group, which merged with the machine learning group “because they wanted the same things,” he says. But other groups, such as those interested in high-performance and data-center computing are also focusing on ML-related extensions. It's Himelstein’s job to make sure the efforts converge where they can.

Despite RISC-V’s successes, Arm is the market leader in many of the markets where a lot of new AI functions are being added, and it is likely to still be so five years from now, with RISC-V capturing about 15 percent of total revenue in the market for CPU core designs, says Semico Research principal analyst, [Rich Wawrzyniak](https://semico.com/content/rich-wawrzyniak). “It’s not 50 percent, but it’s not 5 percent either. And if you think about how long RISC-V has been around, that’s pretty fast growth.”

RISC-V AI Chips Will Be Everywhere

**In December 2021,** just before CES, the world’s leading tech event, Dell introduced [Concept Luna](https://www.youtube.com/watch?v=WCcYsJREtjU&t=1s), a reimagining of the laptop PC that focuses on repairability.

Modern laptops are notoriously difficult ([and sometimes impossible](https://www.ifixit.com/Teardown/Microsoft+Surface+Laptop+Teardown/92915)) to repair, a problem Luna’s designers address by reducing the number of screws to four and avoiding permanent adhesives. Instead, many components lock into place. [Dell has set itself the goal of reusing or recycling one equivalent product](https://corporate.delltechnologies.com/en-us/social-impact/reporting/2030-goals.htm) for every product it sells. Luna, a laptop that can be disassembled and repurposed, could help achieve that goal.

* * *

Repairability dovetails with the larger goal of sustainability. Luna looks like Dell’s latest [XPS 13](https://www.dell.com/en-us/shop/dell-laptops/xps-13-laptop/spd/xps-13-9310-laptop), but clever tweaks have slimmed its carbon footprint. It’s designed for on-demand production to reduce waste from unsold inventory. The motherboard, among the most resource-intensive components, is shaved to a quarter of its typical size. The stamped aluminum is designed to minimize scrap and may be recycled at the laptop’s end of life.

Dell isn’t alone in its focus on [sustainability](https://www.lg.com/levant_en/oled-tvs/2021/sustainability). LG announced at CES 2022 that its new OLED TVs will use more recyclable materials and shave down packaging waste, and [Lenovo introduced](https://news.lenovo.com/pressroom/press-releases/new-yoga-pcs-blend-elevated-design-sustainability/) a Yoga laptop partially constructed from recycled plastics. These steps follow a trend set by Apple, [which uses recycled aluminum in several devices including the MacBook Air and iPad](https://www.apple.com/environment/).

These efforts strike a chord with the public. Among people in advanced economies, 72 percent [are concerned that global climate change will harm them](https://www.pewresearch.org/global/2021/09/14/in-response-to-climate-change-citizens-in-advanced-economies-are-willing-to-alter-how-they-live-and-work/), according to a Pew survey. Only 46 percent had confidence in efforts to reduce the effects of global climate change. And climate change isn’t the only problem worth worrying about. [Consumer electronics can cause deadly pollution by degrading into hazardous materials](https://www.epa.gov/international-cooperation/cleaning-electronic-waste-e-waste) (including lead, mercury, and arsenic) when left in landfills.

Modern electronics often remain relevant for at least five years; routers, displays, and high-end computers can be useful for a decade or more.

Sustainable, repairable devices may help consumers feel they’re contributing less to these long-term problems, but there’s also an immediate advantage: lifespan. Modern electronics often remain relevant for at least five years; routers, displays, and high-end computers can be useful for a decade or more.

Unfortunately, a device’s life is often cut short by a problem that should be (but isn’t) repairable, such as an aging battery or broken charging port. Luna would let users replace or upgrade components after purchasing, prolonging the life of the laptop. A stuck key or bulging battery would no longer be a death sentence.

Framework, a startup that released its first laptop in 2021, is already putting this idea into practice. [The company’s namesake laptop](https://frame.work/laptop) is designed for easy access to its internal hardware. Owners can replace the hard drive, battery, or Wi-Fi adapter, and expansion cards let owners replace or swap ports. Framework’s laptop is not as compact or modular as Dell’s concept, but it has the key advantage that you can buy it right now.

Apple, too, is preparing a Self Service Repair program that will sell parts for iPhones, iPads, and Macs directly to consumers. Owners will be able to fix their devices with new, official repair manuals. In the case of iPhone and iPad, [these will be the first official repair manuals ever released](https://www.ifixit.com/News/55370/apple-diy-repair-program-parts-tools-guides-software).

The cynic in me must point out that these steps, though welcome, are insignificant next to the deep, global change needed to stop the progress of climate change. Repairing the screen in an iPhone will not halt a heat dome.

Still, progress should not be overlooked. Consumer electronics won’t change overnight, but that doesn’t mean change is impossible. It’s important that Dell, and other companies in consumer electronics, know we want sustainable materials. We want modular design. We want devices we can fix and continue using for a decade or more.

These changes, small by themselves, are each just

one step in the marathon effort to curb climate change, but they take us in the right direction, all the same.

_This article appears in the March 2022 print issue as “Dell’s Proposed Laptop is Fit to be Fixed.”_

Dell’s Bold Idea: A Laptop You Can Actually Repair

Despite lurking between microwaves and infrared, terahertz radiation is notoriously difficult to tap into for applications because of its [peculiar physics](https://spectrum.ieee.org/the-truth-about-terahertz "Link: https://spectrum.ieee.org/the-truth-about-terahertz"). Over the past few decades, frequency tunable sources across this 0.3 to 3 terahertz (THz) range of the electromagnetic spectrum has remained elusive to researchers.  

“Electronic sources \[of radiation\] tend to get weaker and weaker as you go up in frequency,” says [Henry Everitt](https://fitzpatrick.duke.edu/faculty/henry-everitt) of the DEVCOM Army Research Lab, as their material properties limit the performance of devices as one approaches the terahertz region. “So, \[while\] the microwave reach is very mature…when you get up above 100 gigahertz \[0.1 THz\], power just falls off like crazy.” From the other end, optical sources of radiation (for example, infrared lasers), get worse as one goes down in frequency, again because of material properties. “The terahertz just happens to be the region where both of these techniques struggle.”

As a result, Everitt adds, tapping into the terahertz spectrum needs “to be something more than just electronics or just optics.” In search for a practical source for this range of the spectrum, Everitt, along with colleagues from the [Harvard John A. Paulson School of Engineering and Applied Sciences](https://www.seas.harvard.edu/) (SEAS), have [demonstrated](https://aip.scitation.org/doi/10.1063/5.0076310) a terahertz laser that is compact, operational at room temperature, and widely tunable—features that earlier sources have struggled with.

The researchers used a [quantum cascade laser](https://spectrum.ieee.org/acoustic-wave-modulate-quantum-cascade-laser) (QCL) pump with a methyl fluoride molecular laser. This compound reacts strongly with optical fields, absorbing infrared and emitting terahertz radiation. The QCL also works in a far more compact laser cavity than earlier carbon dioxide lasers. In their proof-of-concept tests, the team reported 120 individual frequencies between 0.25 and 1.3 THz.

Schematic of the experimental setup. A gold-plated silicon wafer was used as a beamsplitter to reflect a small portion of the pump beam into the reference gas cell, while the rest entered the THz cavity.

![Schematic of the experimental setup](https://spectrum.ieee.org/media-library/schematic-of-the-experimental-setup.jpg?id=29231870&width=980) A gold-plated silicon wafer was used as a beamsplitter to reflect a small portion of the pump beam into the reference gas cell, while the rest entered the THz cavity.Capasso Lab/Harvard SEAS

Everitt’s research interest in optically pumped terahertz lasers spans almost 40 years, and back in the 1980s, he worked on making compact molecular lasers. “But we still had this big CO2 laser that we had to pump it with, so it wasn’t portable. But now that we’ve replaced the CO2 laser with the QCL, both pieces are small…you could put the whole thing in a shoe box.”

This portability is also a game-changer, especially given that terahertz frequencies are highly sought after for their numerous potential applications. “I guarantee you that when the terahertz technology matures, there will be a lot of interest in exploiting it,” Everitt says. Applications include high-bandwidth communications and high-resolution radar. “The higher the carrier frequency, the more bandwidth you can put on it… \[With\] high-resolution radar…you can get very precise centimetre-scale or even finer resolution.”

There are possibilities of remote-sensing applications as well, Everitt says: “Molecules have very distinctive signatures in the terahertz region that allow you to discriminate species, toxins, chemical agents, things that might be a threat or just simply to monitor what's going on in a chemical plant.”

There are potential applications beyond Earth as well, such as for radio astronomers. “So I think the sky’s the limit,” Everitt says. “The point behind our \[research\] is to show the universality of this concept, and that almost any molecule that can exist in the gas phase and has a permanent electric dipole moment \[the net molecular polarity at either end of the dipole\] can be made to lase, including the gases that are being studied in the interstellar media. We could imagine making lasers of those gases and using those lasers as local oscillators for the receivers at these telescopes.”

To turn this proof-of-concept into a user-friendly technology or device that “doesn’t require a Ph.D. scientist from Harvard to operate,” as Everitt puts it, partners from the industry will have to step in. “The next steps for us are about engineering this into a turnkey system where you just type in the frequency you want and out comes the radiation that you’ve asked for…and the system automatically knows how to tune and optimize performance.”

Back in the lab, there are still a few things that need ironed out. Terahertz radiation can be limited in range because of atmospheric water vapour absorption. But Everitt feels this is feature, not a bug. For example, crosstalk or interference problems could be limited. “So if you want bubbles of communication, there are bubbles where your radar operates or bubbles where you’re doing remote sensing. terahertz is actually very favourable, because the atmospheric attenuation prevents the radiation from just going on forever.”

Terahertz Lasers Are About To Have a Moment

Last year, the US government got serious about snuffing out robocalls. Effective 30 June 2021, the FCC [mandated](https://www.fcc.gov/call-authentication) that phone providers verify all callers’ IDs with a cryptography-based protocol called [STIR/SHAKEN](https://transnexus.com/whitepapers/understanding-stir-shaken/).

Anyone expecting robocalls to vanish in a puff of regulation will be disappointed. But relief may be coming, gradually. The technology to block robocalls is improving, and STIR/SHAKEN is part of a trend: US phone users no longer have the sole responsibility to decline robocalls. Now, phone providers have to do their share.

Hard data on robocalls are collected by spam-call-blocking apps, using their users’ phones as a sample. [According to one such index](https://truecaller.blog/2021/12/17/top-20-countries-affected-by-spam-calls-in-2021/) by the app Truecaller, the world’s spam call nexus is Brazil. There, the average Truecaller user in 2021 received 33 spam calls per month—nearly double distantly second-placed Peru’s rate. (For Brazilians, that’s still a drastic decline from [a few years ago](https://www.washingtonpost.com/world/the_americas/from-cheap-loan-offers-to-fake-kidnapping-brazil-leads-the-world-in-telephone-harassment/2019/12/02/b4234e7a-0fb4-11ea-bf62-eadd5d11f559_story.html).)

Another app, YouMail, [has been tracking](https://robocallindex.com/) US robocalls since 2015. Its index shows the country’s robocall count steadily rising through much of the late 2010s, before plunging by nearly half in early 2020—likely thanks to COVID-19. It didn’t last long. By year’s end, robocalls had already returned to 2018 levels.

“Telemarketing is cruising along, just like it’s always been cruising along,” says YouMail CEO Alex Quilici.

If it’s up to apps to cull nuisance calls, then it’s fortunate that they’re getting smarter. Some are now employing AI. [One machine-learning-based system](https://ieeexplore.ieee.org/document/9474299/authors#authors) was created by a group including [Mustaque Ahamad](https://iisp.gatech.edu/mustaque-ahamad), a computer scientist at Georgia Tech, and Sharbani Pandit, then a graduate student.

Ahamad describes the system as a “virtual assistant” that screens callers with a few questions, like “whom do you want to speak to?” or “what is the weather like where you are calling from?” By judging the answers—or details like whether a caller interrupts the questions—the natural language processing system can make an educated guess as to whether a call is legitimate (or wanted).

It’s entirely possible that robocallers might deploy natural language processing systems of their own. But, Ahamad says, “it would require lots and lots of resources, if you’re going to make millions of calls, to be doing that for each and every target that you have.”

This system achieved a 97.8% success rate at pinpointing robocalls, and its creators hope to deploy it as an app. [Another machine-learning-based system](https://ieeexplore.ieee.org/document/8418596), created for Chinese mobile phones by researchers at Shanghai Jiao Tong University and the University of California, Berkeley, achieved a similar 90% success rate.

Apps can work well at the receiver’s end, one mobile phone at a time—which hints at a larger question. “Where do you stop these calls?” asks Ahamad.

Historically, that wasn’t US phone providers’ job. “A decade ago, a carrier had an obligation, a legal obligation, to deliver all calls without question,” says [Jim McEachern](https://riskandassurancegroup.org/jim-mceachern-atis/), an advisor to the [Alliance for Telecommunications Industry Solutions](https://www.atis.org/).

What robocall-culling methods existed, such as do not call lists, [were never particularly strong-willed](https://www.washingtonpost.com/lifestyle/magazine/how-robo-call-moguls-outwitted-the-government-and-completely-wrecked-the-do-not-call-list/2018/01/09/52c769b6-df7a-11e7-bbd0-9dfb2e37492a_story.html), and only impacted “legitimate” callers. Those lists grew even more toothless in the 2010s as robocallers adopted new methods such as VoIP and [falsifying caller IDs](https://www.fcc.gov/spoofing).

“Several years ago, the FCC changed the rules and said, ‘well, you can actually block \[a call\] if, for example, it’s not a real telephone number coming in,’” says McEachern.

Now, the STIR/SHAKEN mandate puts even more responsibility on providers to cull robocalls by fighting back against caller ID mischief. Smaller providers [were given](https://www.fcc.gov/document/fcc-moves-small-provider-stirshaken-start-date-combat-robocalls-0) an extra year, but by July 2022, that too will have passed.

It’s too soon to say how well the changes are working, but there are early reasons for cautious optimism. Quilici thinks robocall numbers have dropped, “maybe ten percent on average on a monthly basis.”

“I think it's making a dent in the numbers of unwanted spam robocalls getting through,” says [George Slover](https://www.consumerreports.org/cro/about-us/our-people/experts/george-slover/index.htm), a policy counsel at Consumer Reports. “It is certainly not catching them all. And the system will be revisited and updated as technology develops.”

Yet even as STIR/SHAKEN’s proponents work to export the protocol outside North America, [others argue](https://www.techdirt.com/articles/20220113/07280948274/why-us-robocall-hell-seemingly-never-ends.shtml) it doesn’t go far enough. The regulation, and much of the coverage surrounding it, is laden with language like “scammers” or “bad guys.” But not all robocalls are scams.

YouMail keeps track of America’s [top robocallers](https://robocallindex.com/), and all the top 15 (as of this writing) are legitimate callers. Some have obvious value—messages from school, for instance—but others are payment reminders, aimless customer calls, or simple telemarketing, nuisance calls that don’t need to play with caller ID. STIR/SHAKEN won’t stop them.

Fortunately, there’s some precedent for going further. Many jurisdictions already ban cold calls on weekends or holidays. In the EU, cold calling a private number without a recipient’s explicit permission [is generally illegal](https://voiptimecloud.com/blog/telemarketing-in-different-countries).

And in nuisance-call-infested Brazil, starting in 2022, all telemarketers [will be required](https://agenciabrasil.ebc.com.br/en/node/1418707) to use a number with a 0303 prefix. In theory, that should make it easier to identify those calls—and filter them out.

Similarly, US law favors quiet phones. “The [Telephone Consumer Protection Act](https://www.fcc.gov/sites/default/files/tcpa-rules.pdf) requires, with some limited exceptions, that robocalls be made to a consumer only with the consumer's prior consent,” says Slover.

It’s unlikely that the air will be permanently free of robocalls. But according to those fighting them, as the technology to block them evolves, and as the regulations keep changing, it’s possible that they’ll go the way of spam emails: present, malicious, but not clogging the tubes of an entire medium.