Co-designing hardware and software for a sustainable future of AI
Adoption of AI models is taking off at an unprecedented scale. However, even more widespread AI use in its current technological state could lead to an environmental disaster due to its substantial energy consumption. At the Advanced Engineering expo, on 15 and 16 May in Antwerp, Ben Stoffelen, Department Manager of AI at Imec, discusses how a co-design approach for hardware and software could address AI’s sustainability issues.
Auteur: Koen Vervloesem
In its annual Digimeter, research institute Imec surveys a representative portion of the Flemish population regarding their views on technological developments. The 2023 edition included some questions about artificial intelligence, and the responses were in line with expectations, shares Ben Stoffelen, the Department Manager of AI at Imec: “AI has made significant inroads into the general population: surveyed individuals replied that they understand it, use it, and believe that it makes them more efficient. This is primarily due to ChatGPT, which reached a high adoption rate in an incredibly short timeframe.”
Another domain where AI is gaining adoption is within autonomous driving, where vehicles use AI on the edge to process data from an array of sensors. “In both the Generative Pre-trained Transformer (GPT) instance and in the case of edge AI, the ongoing advances have been enabled by the introduction of high-powered GPU chips,” Stoffelen elaborates. “Yet, both training and inference of these AI models consumes copious amounts of energy, resulting in substantial environmental impact. As an illustration, researchers have estimated that training OpenAI’s GPT-3 large language model emits as much CO2 as driving a car to the Moon and back. If GPT models become mainstream and if autonomous driving achieves a breakthrough, we’ll have a huge sustainability problem.”
Co-designing hardware and software
It would be unfortunate if our society were unable to reap the benefits of these powerful AI models. Hence, researchers at Imec’s AI department are looking for methods to keep developing better AI models in a manner that is environmentally friendly. Stoffelen continues: “One method is using computational devices that are less generic. Given that a GPU and CPU are general compute solutions, they aren’t particularly energy-efficient: they are designed to execute a diverse set of generic processor instructions. But what if we designed a chip specifically tailored for an application, like a heart rate sensor, or a sensor in an autonomous vehicle?”
According to Stoffelen, achieving the significant improvements in energy efficiency that we need to keep improving AI applications necessitates a co-design approach for hardware and software: “AI application developers shouldn’t wait until Nvidia releases their next accelerator chip. And chip manufacturers shouldn’t solely aim to create more powerful AI chips. If we want to reach our sustainability goals and create better AI applications, we need to address these issues from the root. This means developing purpose-built hardware for the application. This will permit us to incorporate AI within all types of smart sensors.” Imec regards accelerating this co-design approach as part of its mission.
Breaking away from von Neumann
Furthermore, Stoffelen posits that to develop these application-specific, energy-efficient designs, we must break away from the conventional von Neumann architecture that we’ve been accustomed to since the 1940s. “There exist numerous alternative computer architectures, providing us more flexibility to engineer efficient chips. We’re examining neuromorphic chips, which closely mimic how the human brain works. Another promising approach involves analog processors: using physical characteristics of analog devices for computation. One of our ongoing research programs aims to build AI accelerators based on an analog computer architecture. By embracing new architectures, we’ll potentially reduce an AI model’s energy consumption down to nanojoules or picojoules per inference.”
If we manage to address this sustainability issue in AI, Stoffelen predicts the emergence of numerous new applications and breakthroughs in stagnant domains. “Progress in autonomous driving is currently slowing down. Although we have robotaxis, they only fare well because they are restricted to a geofenced and relatively controlled environment. For ordinary cars, the technology plateaued at adaptive cruise control, lane following, and similar features. However, cars eventually encounter unpredictable scenarios, such as harsh weather conditions or pedestrians dressed in dark clothing, leading to detection problems. Manufacturers are attempting to overcome this by adding more and more sensors, but this strategy isn’t feasible. Thus, we require energy-efficient AI to better use existing and novel sensor data.”
