Go back

The Future Runs on ARM: The Architecture That Will Reshape AI

0m 0s

The Future Runs on ARM: The Architecture That Will Reshape AI

The podcast discusses AI's significant environmental burden, focusing on the massive energy and water consumption of data centers. These facilities, consuming up to 1 GW each, strain power grids and require large amounts of water for cooling, often in arid regions. The hosts argue that ARM architecture offers a solution due to its superior energy efficiency—using about one-third the power of traditional Intel or AMD CPUs—and reduced heat generation, which lessens resource demands. Key industry players are adopting ARM: Amazon uses it in Graviton chips, Apple in M-series processors, and NVIDIA in the new RTX Spark system for local AI. ARM's RISC architecture simplifies instructions, completing one per clock cycle, unlike CISC's complex multi-cycle instructions, leading to lower power use and heat. Unified memory in ARM systems, like Apple's Mac Studio, further boosts AI performance by eliminating data transfer bottlenecks. The hosts note challenges, such as Microsoft's reliance on an emulator for ARM on Windows, which saps efficiency. They advocate for a native Windows rewrite for ARM to unlock full potential. Meta's recent contract for ARM silicon highlights growing industry recognition. The shift to ARM is seen as critical for reducing AI's environmental impact, with stock prices for ARM surging 340% in six months, reflecting market confidence. The future of AI hardware, they conclude, lies in energy-efficient ARM-based systems with unified memory architectures.

Transcription

7730 Words, 42228 Characters

English
The Gigawatt Problem: AI's Environmental Burden Welcome back. It's the Hackers Arise podcast. I'm collateral. And here I am with Master Otw. He is the founder of Hackers Arise. He's also the author of cybersecurity books and the person who has trained hackers for the US military and intelligence agencies for over 20 years now. Recently we spoke about the issue of resources, general models. They truly devour them. Companies like Open AI spend more on a single user than they get from that user, especially if those users are trying to get every cent out of their subscription, but armed with its efficiency can hopefully change that. So here we are to talk about this. Speaker 2 Thank you, Collateral. This is kind of an, I think an important subject area as we enter into the AI age. One of the issues that has become real key for the the the big boys, the open AI, the perplexity, the XAI, the anthropic is that they require an incredible amount of resources. We're talking about one GW and plus plants or data centers, right? This is like a medium sized city in the United States. That's how much power that they're using. And this becomes a, a burden to the, the power grid in the United States and wherever they place these data centers, there's so many of them popping up in the US And of course there's also the issue of cooling these systems. So you have incredible draw of power on system that really has no excess capacity. There's no spare capacity in the United States on the electrical power grid yet. We're putting up these plants one after another. And then because they produce so much heat, right, you have all this power going to all these GPU's and CPU's, they produce an incredible amount of heat. And just to keep these systems from cool from melting down and keep them cool, keep them melting down. These systems require large amounts of water. And so in some of these facilities are being built in desert areas where water is at a premium. It's it's something that you know, is necessary for life, for agriculture, for human beings. Yet millions of gallons being used to cool data centers. This enters why This is why we need ARM for AI. Amazon, Apple, NVIDIA Embrace ARM for Efficiency All of us are probably familiar with ARM because it's powering almost every cell phone in the world. This company has been around since 1990 and they immediately set out to build a risk architecture CPU that was very, very efficient on using up power electricity. That's why they dominate the cell phone and the tablet industry. So when the when Apple first built their first iPhone, they chose ARM as the processor of choice. When they built the first iPad, they chose ARM as a processor of choice, primarily because of its energy efficiency. The other advantage of energy efficiency is you don't have to have a separate, elaborate cooling system. And if you've ever opened up your, even, you know, your desktop machine, you'll see that the CPU, whether it be AMD or Intel, has a built in cooling system. Sometimes those cooling systems are pretty elaborate with the heat sink and fans. And if you run a GPU, you, you know how much power the GPU's can drop. So when we're start building these multi GW facilities, it makes sense to start thinking about power efficiency. And that's what ARM gives us. ARM gives us a powerful CPU without the power draw and without the excess heat, without as much heat being generated, meaning that it's not going to require those those draws on the local water resources and power resources, not as much, right? And so when we start thinking about these massive data centers, this seems like the obvious choice. And I'm happy to report that the obvious choice is now being selected by, for instance, when Amazon built their new, well, they're up to a Graviton 5, I think, and they have 192 ARM cores in that system. When Apple was it like 6-7 years ago, six years ago, they switched over to an ARM architecture for their system. So all of the new Apple products have the M1M2M3M4M5 chips. Those are all ARM. Those are all ARM architecture. And the beauty of what Apple did is that they not only switched over to ARM and ARM is a risk architecture. And we can talk more about that in a little bit. It's different, quite different than the SYSC architecture, the CISC architecture that's used by AMD and Intel. When they switched over, they rewrote their entire operating system, right? They didn't just try to put an emulator in there and run it. They put they rewrote it for ARM. I mean, some people are right listening to this go, No, no, Microsoft, I mean, Microsoft, Apple doesn't use ARM. They use Apple Silicon. Well, that Apple silicon is basically a, it's an ARM license, a license that architecture from ARM like a lot of people do. And then they are able to tailor it to their needs. So I mean, I've actually had people tell me no, no, no, Apple doesn't use ARM. They have their own, and in part we're both right. The core of that chip is ARM. Then Apple's designers are able to tailor it to their specific needs. So we have. And then last use case that you know is really important is that recently NVIDIA, Jensen Wong, founder president of NVIDIA, announced that they're coming out with a desktop system or agentic systems. So for systems that can run locally and it's going to be a system on a chip with a GPU from NVIDIA and ACPU from ARM. So we've seen all these major players in the industry, Amazon, Apple and NVIDIA all choosing ARM. And the reason is clear, I think, and that is that ARM offers the best combination of of power efficiency and computing power. You get both the computing power and without using all the energy. Understanding the Power of RISC Architecture for AI ARM CPUs use about 1/3 of the power of a Intel or an AM DCPU. So you can see when we start talking about GW plants, 1/3 the power becomes a significant, it's enough to run a small city, right? The difference and, and this is something that we, we need to keep in mind. You know, I've been an advocate for basically the risk architecture and ARM for 30 years. And I'm glad to see that finally the world is waking up to this superior architecture. It is a superior architecture that the ARM chips use. The the old Sisk architecture basically dates back to like the 60s and 70's. The ARM architecture actually dates back that far too. But we're talking about a very different world than when IBM developed the SYSC architecture. And so this is the time. This is the time for Risk RISC architecture and ARM. And in the last six months, the, the world has awakened to the superiority of these CPUs. And we're, for instance, the price of the, of stock in ARM has gone from like $100 to $440 in six months. So up 340% in just six months. And so this is a reflection of the industry finally go. Oh yeah, this is the way to go. This is the future is ARM. It's also important to note that in the Apple Mac Studio, the Ultra M3 and M4 and M5 coming out soon, that in those systems, what they've done is that one, they've built a system on a chip with shared unified memory. This is important because if you have memory on the CPU and you have memory on the GPU, the communication becomes the bottleneck between those two devices and it slows the whole system down. If you put everything on a single unified architecture, unified memory architecture, then what happens is that there's no, there's no pipeline that has to go between the two devices. They're both using the shit same memory space, which which eliminates the, the, the bottlenecks in the computations. And then last point is that Apple in their systems is actually using a ARM GPU, an ARM GPU. And there are some systems out there that are using ARM GPUs as well. And I think that this is also going to be what's going to happen in the future. And I give Apple a lot of credit for seeing the future and investing in it and putting out systems. Well, they're the Mac Studio is all ARM, it's both the GPU and the CPU are all ARM. And so I think this is what the future holds in terms of AI development. You know, there's we have these massive data centers that are just sucking up all this energy and they're built on very energy inefficient CPUs and GPUs. And I'm hoping these guys wake up as that there's they can do this. They can do exactly the same with an ARM CPU and even a GPUA graphic processing unit. Right? If you're not familiar, the reason that NVIDIA is the dominant chip in AI, like it's probably 80 or 90% of the AI systems, is that those GPU's are really, really good at pipelining data across many many many threads. Like thousands of threads and that's what you need to be able to do the inference necessary for AI well, the the new arm based GPU can do very similar things. Jensen Huang's Vision and Microsoft's Emulator Hurdle It's also kind of important to note that Jensen Wong when he produced the new, what they call the GTX Spark, right, was an RTX Spark, RTX Spark and he chose ARM. He could have chosen Intel, he could have chosen AMD, he could have chosen anybody. There's lots of manufacturers out there, but he chose ARM. That's an important this is this is the man who has revolutionized graphics and he's revolutionized AI and what does he choose? He chooses ARM. It's also important to note that about three or four years ago, Jensen Wong tried to buy ARM and he was rebuffed by intratrust considerations by the US government. And so he understands. He understands the superiority of the ARM architecture. If anybody should understand, he does. And he tried to purchase it to integrate it into NVIDIA. He was rebuffed. Then my Yossi son of Japan purchased it and then it was a private company up to this time. And then they spun it off as a public company over two years ago, 2 1/2 years ago. And so and since that time, the the management of the company has changed dramatically in that they're now aggressively going after these other markets. So most of us are familiar with ARM inside of our cell phones, ARM inside of our Raspberry pies, ARM inside of whatever tablet you're running. I don't care if it's an iPad or if it's a, if it's an Android tablet, it's got ARM inside of it. So that's where they dominate. But but we're now going to see them dominate in AI. That's my belief. And I'm beginning to see all of the telltale signs that the industry is waking up to ARM. Those of you who are part of our community at Hackers Rise will note that there's well, for over a year now, we've been publishing articles on ARM. And This is why, you know, we're, we're trying to make the industry see that this is a superior architecture for running AI. And for that matter, I would say just about anything the the hurdle that we have to get over in terms of our desktop systems use that Microsoft so far has not written their operating system for ARM. Apple has and Linux has, but Microsoft has not. Microsoft uses an emulator. That means it creates an additional layer of instructions between the operating system and the hardware that basically saps the strength of ARM. It's all you. You're doing twice as many operations, OK to do the same thing because you have an emulation layer in between. I hope that Microsoft sees the light like Apple did and then rewrites their entire operating system for R, but we'll see. I don't have a lot of confidence in Microsoft, but I basically just, we know that NVIDIA is coming out with, you know, these Windows based systems. But right now the only thing that Microsoft has is an emulator and that will certainly inhibit the growth of ARM within the desktop systems because it just, it doesn't mean it doesn't use the strengths of ARM. It basically slows down the ARM processors and it's software. So I'm hoping Microsoft, if you're listening, rewrite, rewrite your operating system for ARM. Everybody else has but you all right. And so I hope that they do, in which case then we'll all soon be running ARM on our desktops. So, so that's kind of why I, this is what I, why I want to talk about this, because I think it's going to have a lot of impact in the near future and, and it could have a dramatic impact upon the environmental effects of AI. Reducing AI's Environmental Impact with Local Systems And the environmental effects of AI are huge. The power draw means that stressing your, your power systems and of course generating power all has its own environmental effects. And whether it be natural gas, oil or whatever, you're going to create additional carbon dioxide into the atmosphere and exacerbate, you know, the global warming is taking place. So this is a way, this is what I would, I'm recommending. And I'm happy to see that I'm that more and more companies are coming to this view, but this is the way to go. Speaker 1 Yeah. And I would also like to add that I'm also happy to see that because just recently I found out that ARM has been already shipping it's silicon to Meta because the appetites that the AI has right now with the current hardware and the architecture, they're, you know, as you said, they're going to rob us of the most basic things like water and electricity, right? So I'm hoping exactly more company as well. Speaker 2 They do. Have you said Meta has already contracted with them for their silicon, for their data centers And that's, you know, give Meta some credit here. The it's hard to give Zuckerberg credit, but I will. He's seen the light. And he said this is the, this is the future. So they've got back orders already from Meta and other major players in the industry. So they're going to be producing a lot of silicon. I think the biggest limitation for them right now is they don't have a foundry. Almost everybody builds their chips through Taiwan Semiconductor. Taiwan Semiconductor makes almost everybody's chips. The only other major companies that have foundries, places you can make chips is Samsung and Intel. All right, so, so right now when everybody, there's such a large back order for chips in the microprocessor industry is growing dramatically because of AI. So we have some growth right now taking place at the data center and then we're switching now to the locally run AI, right. So and that's what Jensen Wong is anticipating. That's why he came out with the new RTX Spark chip that's designed to be run locally for your AI. And all of them, all of these new chips use the ARM risk architecture. The Fundamental Differences of RISC and CISC Architectures So Speaking of the ARM risk architecture and comparing it to the SYSC architecture, is the main advantage primarily about performance because ARM uses the simpler instruction or there's something else behind it? Speaker 2 Well, that's pretty much it. I mean it's the big difference between the two architectures is that risk is much simpler, very simple and and few instructions are not complex instructions. So in a SYSC processor, basically you're depending upon the processor to do a lot of the work, right? And, and that was fine in 1970s, right? Because in 1970s, you know, way back in the 70s, there really wasn't even RAM memory. And I used to use what was called core memory. It was very, very expensive. And you think that memory is expensive now, but it's way cheaper than it used to be. And so with the decline in the price of memory, this is one of the things that has changed the industry. And so now we can create, you know, 100 gig, 256 gig, 512 gigs of RAM on a system that wasn't possible just a few years ago and certainly not in the 1970s. So when they first developed SYSC and it was developed by IBM back in 1970s, IBM was 90% of the computer markets in the world. That's amazing. Like 90%. Imagine that one company. We're not, we're not talking this hardware, we're talking about hardware and software, 90%. And so they dominated. I mean, you go back and look at things that were developed, you know, in the computer industry, they almost all come from IBM before like 1990, almost everything they were, they were the company in computing. So at that time there was the whole idea was to create a chip that did a lot of the housekeeping of the operating system on the chip, right? Do all the housekeeping on the chip, making sure everything goes and you're putting multi complex instructions that take multiple CPU cycles. So CPU goes through a cycle or clock cycle and it will to do certain things and assist chip, you have to go through multiple CPU cycles. That's what raises, you know, the heat and electricity consumption. The risk strategy is very different. It says let's do everything in the software, let's do everything in the software. We'll keep create some very simple instructions and you do 1 instruction per clock cycle. And then you make the the CPU as efficient as possible. Because if you're going to do all of the work in software, you need a very, very efficient processor. So that's the whole strategy behind the risk architecture that ARM is building on, right. And so they are kind of one of The Pioneers in risk. I mean, risk has been around since the 1970s. In the 1980s, there was a a number of processes that were built on risk, including the MIPS and the Spark chips. I don't know, you may not have heard those, but those were really big CPUs and popular CPUs in the 90s, eighties, 90s, right up into the 21st century. And then everything went to Intel at that point. But nips and nips are still around, and I think Spark is still around. The old Sun Microsystems, those that were that powered a lot of big companies and the US military, they're all using Spark chips. OK, Unix and Spark, all right. And then the Silicon Graphics, which is the company who makes all the systems that are used in Hollywood to do graphics, right? Those are all risk processors too, right? Those are the NIPS processor, at least they used to be. I think that they, they are. And, and NIPS was using some of the, the most powerful. So MIPS is NIPS most powerful supercomputers. So we've, we've had a history with them that they've done really, really well. But for some reason also the Power PC, the IBM Power PC was also a risk architecture. So the Power PC was a IBM processor that was used, I had still using the Sony PlayStation. It's still used in there used to be the processor of choice of Apple products too. So Apple used to use that CPU. So Apple has some experience with risk processors. And I'll tell you, when I look at the architecture of the Mac Studio, I go, that's exactly how I would architect that system. That's like perfect. You guys nailed it, right? You've got an ARM processor and ARM GPU and unified memory. That's the way it should be done. And we're seeing that you like in the Graviton chips and a number of others that are coming out where Oh, oh, the RTX Spark from NVIDIA is the same thing, but they use a NVIDIA GPU. I would, if I had the choice, I would do it exactly the way that Apple is doing it, a ARM CPU and an ARM GPU and unified memory. That's, that is the structure of the future. That's the architecture of the future. Apple nailed it six years ago, right? And the rest of the world is just catching up. And Apple figured it out because they do so much in graphics. They needed, they needed the power of ARM, OK, without all of the, the overhead of the electricity use and the heat. And so they've come up with I is the best architecture out there. But I am seeing that same architecture now being replicated by other companies. Speaker 1 So you said that Intel and AMD are also using a lot of business and and data centers, right? Speaker 2 So. Why Intel and AMD Lag in Energy Efficiency Are they doing anything to fight back, you know, to stay relevant in in AI? Speaker 2 Well, they important yes yes both companies especially AMD is is coming out with some some very powerful processors and they're coming out with their own GPUs too. So, you know, AMD has always made their own GPUs. They, you probably have seen the Radeon brand on, you know, when you buy an AMD machine. So they've always had their own GPUs since, I don't know, 97 or something like that. They bought Radeon, but that company, Radeon has never been able to keep up with NVIDIA. At one time they were competitors to NVIDIA and then and AMD said, oh, we want our own GPUs, so we buy the company. But unfortunately they didn't put enough research and development into until recently. And now they're coming out with really powerful GPU's and CPU's on a chip just like we're talking about here, shared unified memory GPUCPU. The only problem is that they still use all the energy, so they're still going to be energy hogs. And we know what an energy hog the the NVIDIA GPU's are. If you have a system with an NVIDIA GPU, you know you have to have a really big power supply, right? Like 600 Watt power supply, whereas your desktop can manage just fine with 100 and 5200 Watt power supply. So the difference is all that GPU sucks so much energy. And so and many on these many of these systems that are built all on ARM are running on like 10 to 50 watts, right? We're talking about, OK, so the, the NVIDIA system is running 600 watts and some of these AMD ARM based systems are running on like 10 to 50 watts, right? So less than 1/10 of the energy. And then most of that is if you, if you get away from using NVIDIA GPUs, that's the big savings. I mean, obviously, you know, we're talking about the the CPU being about uses about 1/3 of the energy of say an AMD or an Intel. But if you use an ARM GPU, then you get the real energy savings. You get tremendous energy savings that, you know, could really transform, you know, what these data centers look like. And Popeye says that he's using 800 watts at best at least, right? That's gives you an idea. 800 watts. You know, I'm looking at my system right here. I think it has 150 Watt power supply on it, right? So that gives you an idea of how much those GPUs use and they obviously they're great. They're great GPUs and it's fine, you know, for your desktop use and your graphics, your gaming, what have you. But when you start putting like thousands and thousands and thousands together into a data center, that's where the problem comes up, right? Then you're using so much energy and generating so much heat that you need to use auxiliary cooling systems. That's where the water issue comes. So not only are they superior in architecture and in performance, but they also are superior in its impact upon society and the environment. Why hasn't anybody figured this out before? Why is it taking, why is it taking until 2026 for the world to figure this out? And I think the answer to that is pretty obvious and that there's a vested interest, the industry has a vested interest in the SYSC architecture, the Intel AMD architecture, right? They have a a vested interest in it and they're not going to kill their own goose that lays the golden egg. It takes somebody else to come along and kill it. And the dramatic change in the industry that AI represents is forcing them to finally realize that this is the superior architecture for computing. Speaker 1 Yeah. And I just wanted to ask you, as more companies are switching to ARM, will that make the Frontier AI more affordable for all of us? Like, do you think they will lower the prices because, you know, a couple of podcasts to go? Basically, we said that if things continue to go this way, the way they're going, right, with a lot of power being consumed, prices will just, you know, they'll be going up and up. But do you think they will actually lower them or they'll just, you know, try to compensate for all the financial losses they they had in in on the previous years? Speaker 2 Well, that's a that's a good question. The Financial Realities of AI and Local System Advantages So here's what I see happening in that you, you may have seen that Open AI is now contemplating lowering their prices because people are getting sticker shock from their cost of tokens, where they're using so many tokens and they're running up two to $5000 per person per month, right? And so Open eye hasn't done it yet, but they're contemplating lowering their prices because there's, there's some sticker shock taking place in the industry. So I think we will see lower prices, but lower prices will mean that those companies are just going to lose more money. They already have. They're all losing money. They're all losing money despite them. The fact that there's all of these, you know, all this mania about AI, they're all losing money. And but that's not unusual for new technologies, right? So when the Internet was new in the 90s, everybody lost money initially. And then even something like somebody like Amazon, Amazon lost money for years and years and years and years and years. I mean, they lost money for at least 10 years, maybe more if I remember correctly. And so that's not unusual to lose money. The problem comes in is that how long can you continue to lose money? You know, it's that's up to your investors. If the investors are willing and the banks are willing to lend you money. And these guys are sucking in a lot of capital and we're talking about hundreds of billions of dollars a year. That's what each one of these companies, the Facebook, the Amazon, the Google, you know, the XAI, the Anthropic, they're investing hundreds of billions of dollars a year. That's why they're putting so much strain on the microprocessor market because they're buying so many of them. They actually are lining up and pre ordering, say when you have that chip ready, we want it. That's what Facebook did with the ARM. When you have it, we will take, you know, $10 billion worth, over $100 billion worth. So initially, you know, they drop prices. That's probably what's going to happen to you want to get people, you know, with any new technology, you want to get people hooked. You don't want to put up the financial barrier for them to use it because, you know, you want to. You want people to use it and then find out how superior it is and then get hooked on it. And then you raise prices. Then you raise prices so you can make money. So one of the ways, though, that they can become more efficient is, of course, adopting these ARM processors so their electricity costs aren't as dramatic. And hopefully that helps them survive. But I really believe that the future of AI is on local systems. Agentic AI. Why? Why should we be paying, you know, the open AI, all this money where we can generate our own AI systems locally and use those? And I think that's really dangerous for Open AI and Anthropic, if everybody starts developing your own local AIS because, you know, their market is going to shrink. And so there is a possibility in the near future that we come to a point in time, I don't know, five years from now where these companies are not profitable, They're not profitable now, they're not profitable 5 years from now. That could be, that could be a serious problem because they have sucked in so much capital into their formation of these data centers. And nobody's questioning, nobody's questioning. Is this the right strategy? Right. Because it serves them well. It serves the big boys well. It serves the open AI and the anthropics and the Googles. It serves their interests. It doesn't necessarily serve our interests, but it serves their interests. And sometimes people can continue on a path that is has no future and not change until it smacks them in the face with major financial losses, which I think there's a possibility that that happens. They might they might adapt, they might change in the five years or so. It may not even be five years and maybe two or three years because both Open AI and Entropic will become public companies this fall. So public companies means that everybody can invest in it. Outsiders going to do right now, the only people can invest are, you know, the big players, people who have a position that they can put in a billion dollars or $10 billion. And then those companies will then sell their shares to the public come this fall. And then when that happens, we will all be able to look at their financials because public companies must show their financials to the public. And then that could bring about a reckoning for this industry. And so that could spell a period of dramatic change. Period of dramatic change now. But if these companies fail, then that could even have more repercussions, primarily financial. So the answer to your question, I don't think that they're going to use it as a way to lower cost. Hopefully they do, but it will certainly save them money. And I think as they become right now, nobody's thinking about costs, right? Nobody's thinking about there's like, build it as fast as we can build a build, build, build. And in the future they're going to have say, oh, this doesn't make any sense. This is this is going to lose US money. Everybody's in a race and nobody's thinking about the consequences of the race. Speaker 1 So you think we're just going to be far better off running our local systems, right? Speaker 2 I think so. And if you look, if you watch the developments that are taking place in AI, the open source agentic systems are, are right up there, right, in terms of their efficiency and the the precision of their answers as the big systems, the anthropics that mean they're not exceeding it, but they're really, really close. So really, really close is probably good enough, right? And you don't have to have something that, you know, it's going to cost you $5000 a month when you could build your own system, you know, and run it locally and not have to spend, you have to pay money for electricity and some hardware and some development. But once you invest that money, then you don't have to keep on paying for the system and you get results that are 99% of whatever Anthropic is putting out or 98%. And so I think that's the way to go. There's other advantages to running it locally too, and that you're not sharing your data with these companies. And I think that's a really important point because I don't want them to have my data, right? Because once they have your data, then all kinds of possibilities exist that none of which are good. All right? And of course, there's the cost. And you can also tailor the system for your needs, right, that you can't necessarily do with the big boys, including, you know, what data sets you want to use. So I got a question here. Papaya says, what if the big boys cut us all from open source? What if the big boys cut us all from open, cut us off? Is that what you mean by cut us off, like not allow us to use open source? Well, that's not a possibility, right? How can they do that? I mean, open source is, you know, it, it's been around for 35 years or so. I mean, open source is we can all just share between us that doesn't doesn't require their permission. All right? I don't understand how they could cut us off open source. Local AI systems are getting smaller and smaller, OK, and getting more and more efficient. So it's becoming pretty clear that you don't need open AI or anthropic, right? And so this could have major repercussions on the industry. Speaker 1 And you also mentioned Taiwan, right. The Geopolitical Risks of Global Chip Production So we all directly or indirectly rely on their production. And I mean, China has its own plans on Taiwan. the US administration has been motivating those companies to, you know, to relocate couple of factories are, you know, appearing in Europe and we also have them in the United States, but they're expensive and slow to start. Do you expect anything happening like before 20-30 that could essentially to to Taiwan or with Taiwan that could have a negative impact on the tech companies, the Western tech companies that rely on the production? Essentially ARM in in in a way because Apple, NVIDIA, they rely on the production, right? Speaker 2 Everybody. No, I thought this, those guys, the only ones who don't really depend upon it's the company's name is Taiwan Semiconductor. They, the government of Taiwan 20 years ago or so made a decision that they wanted to be the major player in the production, like a factory of producing chips and they did it. They now are producing over 90% of this, the processors in the world. And so this is kind of a, it's important point. I think that if Taiwan Semiconductor wasn't there, the US and the other Western countries wouldn't be that concerned about China coming in and taking back Taiwan. For those of you who don't know the history of China, that in 1949 Mao Zi Tong was a communist leader who led a revolution in China. China had a very oppressive government, more of a dictorial style government and Bao rejected that and had a revolution. He was successful in 1949. And the old government left, you know, they left into exile and they left to Taiwan, right? So that's the old government. And then the one that's in China now is the new government. And the the new government, the Communist government, says Taiwan is still ours. You know, it's still rightfully ours. It's ours. And if there weren't a Taiwan semiconductor, I don't think that the US would be that concerned. And I don't really know what Trump's position on Taiwan is at this point. But imagine if that China did take Taiwan, right? And the US is all busy with, you know, one invasion after another. And, and the Western countries are all busy with their own stuff, whether it be Ukraine or Iran, what have you. And China kind of sneaks in and like says, oh, well, this is our opportunity. Those guys are all busy doing other things. And they took Taiwan back. And then they said you can't export to the West. Boy, that would be devastating, right? All of a sudden, we'd have no CPUs. This would be serious. We do have boundaries in the United States. A few. Taiwan Semiconductor is making a foundry right now in Arizona. Intel has foundries and Samsung have founders. So those those would be outside of the realm of TSMC. But yeah, it would be devastating. Remember that during the pandemic? Remember, during the pandemic, we had a hard time getting CPU's during the pandemic because so many of them were overseas and some of those facilities were not producing. So for instance, there was a big problem with new car production because if you don't know it, cars, obviously cars use CPUs. They have all kinds of CPUs throughout them. And so this inhibited car production during the pandemic. There wasn't enough CPUs and drove the prices up. Can you imagine if that happened to the whole world right now? So and, and China has taken kind of an adversarial, the US has taken an adversarial approach to China and China has taken an adversarial approach to the US. So if they were to take Taiwan, I don't know that they would let Taiwan Semiconductor continue to export to us so that we can make smart weapons against them. And that's, you know, that's the US position of NVIDIA cannot sell its chips in China, right? They're top end chips. They cannot sell them to China because U.S. government says no, you cannot. Now ARM is a British company. It's not an American company. And so that same restriction doesn't apply to ARM, right? They can sell to China unless the British government does something to limit them. But right now that hasn't been the case. So it could be if this happens, I mean, if you get, if you get Taiwan taken over in Taiwan, some of course taken over by the Chinese government, it could be a devastating impact upon our IT systems, our military systems. And I think it might actually lead to, it might be, it's a possibility, it could lead to war between the US and China. And that's why it hasn't happened yet, because I think China recognizes that to do so, even though when Trump was there recently, they told him the most important issue for us, the most important issue for us is Taiwan. We want Taiwan. It's our land. It's, it's, it belongs to us. Just because some rogue government decided to move there doesn't mean that it's not ours. Still imagine if whatever country you're in, you had a rogue government take part of the country and set up their own government there. Probably wouldn't take too, wouldn't take that too kindly if if that happened. And that's that's kind of the Chinese position is that that's still ours. Taiwan is still ours. So I'm obviously concerned about China taking back Taiwan, but the same time you can kind of understand why they want it. They're like, this is our country, This is not a separate country, this is ours. So let's hope that that doesn't happen in the near future. And I think that the West really needs to build boundaries. The problem is that nobody can really build chips as well and as efficiently as Taiwan Semiconductor. That's why we don't have the found piece there. They were so good at what they do that they put everybody else out of business with the exception of Intel and Samsung. Well. From Acorn to AI: ARM's Path to Global Dominance Just wanted to, as we're talking about ARM, just wanted to add that, you know, recently I found out that ARM came up with ACCA, it's confidential compute architecture and the. Yeah, right. The problem that we have with Memory Pulse is I think they're not encrypted, is that correct? That's correct, right. So, you know, ARM is getting those re alms and you know, isolating the environments. So I think they're winning in in many directions, just their time, you know? Speaker 2 It's it's their time. It definitely is their time. They've been patiently, you know, one of the things that I think about ARM is that they're actually are kind of a gift to the world because ARM, Arm's been making these use microprocessor designs. They don't make microprocessor, they make designs. And basically they've been giving them to the world for like pennies on the dollar. I recently found out that those M1M2M3 chips that that the Apple makes, they pay a royalty of $0.30 a chip, OK, $0.30 a chip. That's incredibly low. And so and that's, that's probably going to continue that way. But now what? Now when ARM makes a deal for a chip design, they're charging 10's and maybe hundreds of dollars of royalty per chip, right? And Apple only pays $0.30 because they made that deal like 10 years ago and that, and so they've got a, they got a sweet deal there that isn't necessarily good for ARM, but it is good for Apple. But they have been doing this for 36 years and really not making a whole lot of money, right? And now, you know, things are changing. Some of you may well, you know that the the the Raspberry Pi was an ARM CPU, right? But there's been a lot of other like mini micro computers that I've been using ARM for quite a while. There's the name ARM, right? Is originally OK, Originally the name ARM stood for Acorn Risk Machine. OK, Acorn Wrist machine. What the hell is an acorn? Well, what it was is one of the early PCs that the folks at ARM had developed and it was kind of a competitor to the design of our now, you know, it's a Windows desktop. And so that's where they began. And then of course, they changed the name to Advanced wrist machines now, but that's where they have the beginnings of making really inexpensive micro computers. And of course the the the Raspberry Pi is outgrowth of their efforts as well. So they've been really a good company for the computing world. ARM has been around for a long time. They have adopted the risk architecture that has been around for a long time. They've adopted because it's it's superior architecture, but simply it's a superior architecture. At least it has been superior for at least 30 years. The problem with risk architecture is that the embedded players, the Microsofts and the Intel's and the others have resisted it for their own financial gain. And one of the point I wanted this just one last point talking about financial gain and, and, and risk architecture. There was in the 1990s, there was a, a risk processor called the Alpha that came out from digital equipment company. That's probably a company you ever heard of either because they went out of business in the 90s. They were at one time the second largest computer company in the world and they came out with the fastest and best risk any type of processor in the 90s. And it was way exceeded, way exceeded in the Intel or AMD CPUs. The problem that came up with it is that Microsoft would not rewrite their operating system for it. So even though this is the fastest CPU in the world, it, the operating system slowed it down. And then eventually, this is kind of an interesting story that not many people know, but eventually Intel bought the rights to that processor and killed it, right? They killed it because it was a threat to them. And so now essentially the same architecture is coming back 30 years later and people are like, Oh yeah, this, this is a better way of processing data on our computers. It's a much more powerful, much more elegant design than the 6th design. The sys design was great for 1970s computing, 1980s computing, but the Risk design is superior for the type of processing we need in the 21st century. So watch Risk, watch ARM, watch Risk and ARM, because you know those, they're basically integrated. This is going to change AI, it's going to change the way we run AI, and it's going to change eventually your desktop too. Your desktop is going to be running a risk processor probably in two or three years, a risk arm processor. So Risk is the architecture, ARM is the company who makes it right. And there are other risk processors out there, but ARM dominates this field right now. They are the company and they are I, I think they have a tremendous future and we should continue to look for them to appear in, in, in lots of new places. You know, they're, they dominate your phone, they dominate the tablets, they dominate your IoT. Those are all ARM processors. Why? Because they're so efficient. And soon we're going to see it take over the AI industry as well as the desktop. And with that, I'll say thank you all for coming. Speaker 1 Thank you. Speaker 2 See you next week.

Podcast Summary

Key Points:

  1. AI data centers require massive energy (up to 1 GW per facility, equivalent to a medium-sized city) and water for cooling, straining power grids and resources.
  2. ARM architecture offers superior energy efficiency (using about 1/3 the power of Intel/AMD CPUs) and generates less heat, reducing environmental impact.
  3. Major companies like Amazon (Graviton chips), Apple (M-series chips), and NVIDIA (RTX Spark) are adopting ARM for AI due to its power efficiency and performance.
  4. ARM's RISC architecture uses simple instructions per clock cycle, contrasting with CISC's complex instructions, which consume more power and generate more heat.
  5. Unified memory in ARM systems (e.g., Apple Mac Studio) eliminates bottlenecks between CPU and GPU, enhancing AI computational efficiency.
  6. Microsoft's use of an emulator for ARM on Windows hinders performance; a native OS rewrite is needed for full ARM benefits.
  7. Meta has already contracted ARM for data center silicon, signaling industry shift toward energy-efficient AI hardware.

Summary:

The podcast discusses AI's significant environmental burden, focusing on the massive energy and water consumption of data centers. These facilities, consuming up to 1 GW each, strain power grids and require large amounts of water for cooling, often in arid regions. The hosts argue that ARM architecture offers a solution due to its superior energy efficiency—using about one-third the power of traditional Intel or AMD CPUs—and reduced heat generation, which lessens resource demands.

Key industry players are adopting ARM: Amazon uses it in Graviton chips, Apple in M-series processors, and NVIDIA in the new RTX Spark system for local AI. ARM's RISC architecture simplifies instructions, completing one per clock cycle, unlike CISC's complex multi-cycle instructions, leading to lower power use and heat. Unified memory in ARM systems, like Apple's Mac Studio, further boosts AI performance by eliminating data transfer bottlenecks.

The hosts note challenges, such as Microsoft's reliance on an emulator for ARM on Windows, which saps efficiency. They advocate for a native Windows rewrite for ARM to unlock full potential. Meta's recent contract for ARM silicon highlights growing industry recognition. The shift to ARM is seen as critical for reducing AI's environmental impact, with stock prices for ARM surging 340% in six months, reflecting market confidence. The future of AI hardware, they conclude, lies in energy-efficient ARM-based systems with unified memory architectures.

FAQs

AI data centers can use millions of gallons of water for cooling, often in desert areas where water is scarce. This strains local water resources needed for agriculture and human consumption.

NVIDIA's CEO Jensen Huang chose ARM for its superior combination of power efficiency and computing power, believing it's the optimal architecture for AI. He previously tried to acquire ARM due to its advantages.

RISC uses simple instructions executed one per clock cycle, relying on software for complexity, which reduces power draw and heat. CISC uses complex multi-cycle instructions, increasing electricity use and heat generation.

Unified memory allows the ARM CPU and GPU to share the same memory space, eliminating communication bottlenecks between separate CPU and GPU memory. This speeds up computations and enhances efficiency.

Microsoft uses an emulator to run Windows on ARM, which adds an extra instruction layer that slows performance and negates ARM's efficiency benefits. Apple and Linux have rewritten their OS for ARM, but Microsoft has not.

Yes, ARM GPUs can handle AI inference by pipelining data across many threads, similar to NVIDIA's approach. Apple's Mac Studio uses an ARM GPU, and NVIDIA's RTX Spark pairs an ARM CPU with its own GPU, showing hybrid potential.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.