ep42 - inControl guide to ... the Nyquist criterion

Hello and welcome to InControl, the first podcast on control theory. Here we discuss the science of feedback, precision making, artificial intelligence and much more. Picture a copper wire stretching from New York to San Francisco. Three thousand miles of it strung on wooden poles across the Appalachians, threaded through the cornfields of Ohio, slung over the Rockies, humming in the wind above the Mojave. It is 1927, you are an engineer at the American Telephone and Telegraph Company, commonly known as AT&T, and your job, your life's problem, is to make a human voice travel that wire and arrive at the other end still sounding like a human voice. But copper is not kind. Every mile eats the signal, a little more attenuation, a little more distortion. So you add amplifiers, dozens of them, hundreds, eventually on a single transcontinental line. Each one takes the weakened signal and pushes it back up, and here is where the trouble begins. Because each amplifier also adds a little distortion of its own, a little noise, a little drift. By the time the signal has passed through 100 amplifiers in series, your colleague down the hole doubts whether anyone would even care to listen to the sound after the signal had gone in succession through several dozen or even hundreds of the finest amplifiers. And then one morning, on a ferry crossing the Hudson River, another young engineer named Harold Black has a revelation, and that will change the world. But his revelation comes with a catch, a dangerous, beautiful, mathematically subtle catch, and it will take another man, a quiet swede, who once worked at a stable boy in Minneapolis to untangle it. This is a story of that catch. This is the story of the Nyquist stability criterion. Welcome to the Incontrol Guide to the Nyquist criterion. I'm your host Alberto Paduan, live from a recording studio in Vancouver. Quick thanks to our long-term sponsor, and CCR Automation. Today we open with a new episode of the Incontrol Guide series, one I've been looking forward to for a long time. Our subject is the Nyquist stability criterion, one of the most powerful ideas in all of control theory. First off, a warning. The Nyquist stability criterion is by common consensus one of the hardest lectures to give in any classroom at any level. It is famously the lecture that makes even the most experienced instructors break into a cold sweat. Even seasoned control engineers sometimes get tangled in the details. So today I'm going to attempt something slightly risky, explaining it without visual aids. No diagrams, no curves to trace in the complex plane, just words, images, and a few analogies. That's the experiment. I hope it works. But just in case it doesn't, I'll also include links in the description to some excellent explanations by Brian Douglas, who does a tremendous job walking through the Nyquist criterion step by step. Before we get into the mathematics though, I'd like you to meet the man behind the name. One needs to understand the world he lived in and worked in because great ideas rarely come from nowhere and this one is no exception. The Nyquist stability criterion is a result that was born in the telephone laboratories of the 1930s New York, but rooted in the pure mathematics of the 19th century Paris and that remains nearly a century later one of the most important tools for understanding whether a feedback system will behave. We're going to talk about Harry Nyquist, of course, the man, the immigrant, the quiet genius who held 138 patents and described his life's work as a few technical papers. We're going to talk about Bell Labs, the most extraordinary research institution of the world, what the world has ever seen. And we're going to talk about the mathematics and applications related to Nyquist's criterion, from Koshy's argument principle and winding numbers to encircumence of a point in the complex plane. And hopefully by the end it will all make sense, even without a diagram. Harry Theodor Nyquist was born on the 7th of February 1889 in a village called Nilsby in Varnland, Sweden. His parents, Lars Johnson Nyquist and Catalina Erickstokter were not wealthy people. Lars was a shoemaker and a small farmer who had bought a place called Darsor, literally the south one in the hamlet of Tomholt, about 40 km north of Carlsstad. The family were Baptists in a Lutheran country, which already marked them as fairly independent minded for the time. Harry was the 4th of 8 children. Now here's a small but telling detail. The family name was originally Johnson, Lars Johnson. But just a hundred meters down the road, lived another Lars Johnson and the post office kept mixing up their mail. But one family agreed to change its name as was common practice in ruled Sweden. Harry's father became Lars Nyquist. And later when Harry had to cross the Atlantic it would change the spelling one more time to Nyquist, to make it easier for English speakers to actually spell his name. The 8 children were sent to school, which was not a given for a poor farming family in late 1800s, Sweden. And Harry attended 3 different school houses. The old one burned down in 1899 for a while classes were held in the Nilspie Mission Hall. One of his teachers, in particular, a man named Morden, recognized something unusual in the boy. The teacher put great confidence in young Harry and even learnt in books, something that was not that common in those days. Two of Morden's own songs had already emigrated to America and when Harry was about 14 the teacher made a suggestion that would change the course of 20th century technology. Go to America you'll find more opportunities there. But the family as I mentioned was poor, so for the next four years Harry worked at a sulfate factory in the nearby town of Dij. Saving money and meeting the immigration requirements. You need a $10 and a guaranteed job. In 1907 at the age 18, he boarded a ship bound for Boston. From Boston he made his way to Minneapolis and found work first in a library stable and then in a farm. And then he did something extraordinary, enrolled in the southern Minnesota Normal College in Austin, Minneapolis where he met Antonio Wachlin and proposed to her. At the time they did not know that it would take about eight years before they could marry. After one year he interrupted his studies and took a teaching job. But he was persuaded to continue his studies to get a diploma for teaching in high school. He finished in 1911 as the valedictorian in his class, namely the top student. He taught at high school, originally then entered University of North Dakota in Grand Forks, where he earned a bachelor's degree in electrical engineering in 1914 and a master's degree in 1915. And it was a North Dakota that he befriended another young Swedish-born electrical engineer and physicist named John Bertram Johnson. A friendship that would a dozen years later produce one of the foundation of results of modern physics, but more on that later. From North Dakota, Nyquist went to Yale. There in a complete dispute in physics in 1917 with a thesis titled "On the Stark Effect in Helium and Neon". This was largely an experimental thesis studying how spectral lines split in elected fields. And then instead of seeing an academia, it did something that turned out to be one of the best career decisions in the history of engineering. As I mentioned, he joined AT&T, the American Telephone and Telegraph Company. To understand what Nyquist walked into, you first have to understand what AT&T and its research arm Bell Labs were. That's what they did, but what they were. Bell Telephone laboratories was in a way an accident of monopoly. AT&T held a government-mandated monopoly on telephone service in the United States, and this monopoly produced enormous guaranteed revenue. So AT&T's leadership, particularly a visionary named Mervyn Kelly, decided to pour that revenue into research. Not just applied research, but fundamental research. Physics, chemistry, mathematics, metallurgy, anything that might someday in some way improve the telephone network. The result was the most extraordinary concentration of scientific talent, perhaps in recent human history. The output speaks for itself. The transistor, radio astronomy, the first solar cell, the charge-capulet device, the laser, unix, the C-programming language, cloud-shannon's information theory, and Walter Schuert's Statistical Quality Control, and by last count, at least ten noble prices. One may justifiably ask what made it work. Well, several things, according to John Kertner, who wrote the definitive history of Bell Labs, titled "The Idea Factory." There were a few ingredients that really mattered. First, the funding was patient. You would work on a problem for years without anyone asking for a quarterly deliverable. John Pierce, one of Bell Labs' most brilliant scientists, put it this way. And I quote, "Researchers didn't have to raise funds. "Work on a topic could be supported for years. "And research could be terminated "without done in the researcher." The second ingredient was perhaps the environment, which was designed for intellectual and quite literally physical collision. When the director, Marvin Kelly, built the Mara Hill campus in New Jersey, he made the corridors longer than two football fields. And he put labs and offices in different wings. So you had to walk past other people's work to get to your own. Cloud Shannon famously used to ride a unicycle down those corridors, juggling bowls and not-in-ed colleagues. Doors were kept open. Theoreticians took to experimentalists, engineers to chemists, physicists to mathematicians. The third ingredient, and this is a crucial one, was that there was a mission. Not a vague aspiration, but a concrete, compelling, engineering challenge. Make long-distance telephony work reliably and affordably. Every fundamental discovery of bad labs, no matter our abstract, was connected, however, distantly to that mission. Penzias and Wilson discovered the cosmic microwave back-around radiation, the echo of the Big Bang, because they were trying to eliminate noise from a telephone antenna. The transistor was famously invented in 1947 by physicist John Burdine, Walter Bratain and William Schockley, because vacuum tubes were too fragile, too hot and too expensive to scale the telephone network. And into this extraordinary environment in 1917, walked Harry Nyquist, the formal stable boy from Barman. He moved from the AT&T engineering department to the Department of Development and Research in 1919, where he would spend the next three and a half decades working on the telegraphy, telephony, voice, and picture transmission, as well as the mathematics of communication. When the department fully emerged, well-telephone laboratories around 1934, Nyquist continued without interruption. He would eventually rise to assistant director of system studies in 1952 and retired in 1954. But here is perhaps the most revealing detail about Harry Nyquist at Bell Labs. Some years after his time there, patent lawyers in the legal department conducted a study. They wanted to understand why Bell Labs employees were so much more productive than others. They analyzed the data, looking for a common factors, education, background, area of expertise. And as reported in the idea factory, the book I mentioned earlier, they found one important detail. The workers with the most patents often shared breakfast or lunch with Harry Nyquist. It wasn't that Nyquist gave them specific ideas. As one colleague recalled and I quote, "He drew people out, God then thinking." More than anything, it seems like Nyquist asked very good questions. So if you've been following the podcast, you may remember from our incontrol guide to feedback episode that we already talked about Harold Black, one of Nyquist's colleagues, and his famous fairy, Rye, across the Hudson River, where the invention of the negative feedback amplifier took place. Just in case you didn't, let me briefly take you to August 2nd, 1927. Harold Black was a 29-year-old electrical engineer. He had been at Bell Labs for about six years working on a problem that aimed to reduce amplifier distortion. Every day, he wrote the lack of one a fairy from New Jersey across the Hudson River to Manhattan, where Bell Labs occupied a building on West Street. In that particular morning, somewhere on the water, he had one of the most famous Urika moments in engineering history. The idea is this, what if instead of trying to build a perfect amplifier, one that introduces zero distortion, you would build an amplifier with a high gain, far more than you need, and then feed it back to the input in so-called reverse space. By doing this, one would deliberately throw away most of the gain, but in doing so, the distortion would be reduced by the same factor. Thus, if your amplifier has a gain of 100, and you apply enough feedback to reduce the effective gain to one, the distortion is also reduced by roughly a factor 100. And this is the negative feedback amplifier, one of the most important inventions in the 20th century. Black famously felt the urge to write, but he had nothing to write on, so he picked up his morning newspaper and by what he later called sheer coincidence, one page was blank. It was a copy of The New York Times. On that blank page, Black scribbled his idea, signed it, and dated it. And before the fair, he docked, the essential concept was on paper. Now, the rest of the world was not as quick to understand. Black filed his patent in August 1928, but it took nine years to be granted. Finally issued it as a US patent in December 1937. The reason for this delay was that the patent office didn't believe it would work. The concept of delivery, reducing gain to improve performance was so counterintuitive that the examiner's thought it was pure nonsense. And Black's mathematical ideas did little to persuade them. Ben Lab's mathematics department had thought on fry, reportedly called it, and I quote, "beneth contempt." But there was a catch, one that even Black initially overlooked. His negative feedback amplifier could become unstable, or as engineers said at the time, it could oscillate. In technical terms, if the phase shift around the loop caused the return signal to arrive in phase with the input, turning negative feedback into positive feedback, the amplifier would erupt into self-sustaining oscillations. Telephone engineers had a simple word for this phenomenon, singing. This is because to a listener on the line, it sounded like a continuous whistle or tone, very much like a note being sung. Singing was not a theoretical concern, it was a practical one. Every long-distance telephone line in America ran through long chains of amplifiers. Each one a potential source of oscillation. To quote Black's own paper inventing the negative feedback amplifier that will be a link in the description, don't worry, this meant that if I had a string of 1000 amplifiers, the cumulative voltage distortion would be about 60 decibels. In other words, a voltage signal would be distorted by a factor of about a thousand, enough to make a voice on the telephone barely recognizable. Black's colleagues, Fris and Jensen at Bell Labs, had already documented the problem in a 1924 technical report, titled "High Frequency Amplifiers". They called the problem "regioneration", which made, and I quote, the total amplification varies irregularly in a very undesirable manner, and also makes the set sing at certain frequencies. So Black's invention was brilliant, but without a theory, it was useless. When would a given amplifier sing and when would it behave? That is the question Harry Nyquist set out to answer. Before we get into the stability criterion, let me briefly set the stage by mentioning Nyquist's other two landmark contributions, because all three of them came from the same mind in the same building within about 10 years of each other. The first is the so-called Jensen Nyquist noise. Remember JB Jensen, Nyquist's friend from the University of North Dakota. By the late 1920s, Jensen was at Bell Labs too, and he had been making extremely precise measurements of a strange phenomenon, a tiny random voltage that appeared across any electrical resistor, even with no signal applied. Jensen measured it carefully, and then walked down the hole and told Nyquist about it. Jensen later recalled in a quote, "The results were discussed with Dr. Nyquist, "who in a matter of a month or so, "came up with the famous formula." Nyquist showed that the power of this thermal noise is four times the resistance, times Boltzmann's constant, times the absolute temperature, times the bandwidth. This result beautifully connects thermodynamics to electrical engineering in four symbols. Both papers appeared back to back in the physical review in 1928, and of course, I'll put a link in the description, worth checking out. The second contribution is the celebrated Nyquist sampling theorem. In his 1928 paper, titled "Certain Topics in Telegraph Transmission Theory," Nyquist established that to faithfully represent a signal, one must sample it at a rate of at least twice its highest frequency component. This minimum rate is now called the Nyquist rate. and sampling below it leads to aliasing, a phenomenon in which different frequency components overlap in the spectrum of the sample signal, making the original signal impossible to reconstruct correctly. Claude Shannon, the father of information theory, also at the labs, later gave the theorem its full mathematical proof in 1949, and it became the foundation of all digital audio, video, communications in general, essentially everything digital. Every time you listen to a podcast, including this one, the Nyquist sampling theorem is quietly at work. The third contribution is the subject of today's episode, and that's what we'll discuss next. In January 1932, the belt system technical journal published a 21-page paper by Harry Nyquist title "Regeneration Theory". Few papers have had a great impact on the field of control engineering or, generally speaking, engineering as a whole. Let me tell you what the paper does and why it was revolutionary, and this is important, how it differs also from what you find in a modern textbook. Because the modern derivation of the Nyquist criterion is not actually Nyquists derivation, but more on that in a moment. The paper opens with a beautifully clean statement of the problem, and I quote, "Regeneration or feedback is of considerable importance in many applications of vacuum tubes". The most obvious example is that a vacuum tube oscillators, but feedback is carried beyond the singing point. So this was his problem. Now, a brief word about vacuum tubes since they're central to this story. A vacuum tube or a thermionic valve is an electronic device sealed inside a glass envelope from which the air has been removed. Inside, a heated filament boils off electrons, which drift through the vacuum towards a positively charged electron. Their flow can be controlled by a voltage applied to a metal grid placed between them. And so the controllable flow of electrons is what makes amplification possible. A small signal applied at the grid can control a much larger current, giving rise to a gain. Vacuum tubes were essentially the transistors of the earlier 20th century. The fundamental building blocks of every radio, every amplifier, and every telephone repeater. But they had their drawbacks. As we mentioned, they were hot, they were fragile, they were typically short-lived, and they were expensive. And this is why for long-distance telephony, you need hundreds of them in series. And so Nyquist, in his regeneration theory paper, considers a feedback system where an amplifier's output is connected to its input through a transducer, essentially a sensor, and defines a complex quantity equals a j of j omega, where j here stands for the imaginary unit. This is what we now call today the loop gain, the ratio by which the amplifier and the feedback circuit modify a sinusoidal signal in one complete round trip through the loop. The magnitude gives rise to the gain ratio, the angle gives the phase shift. But more on that later too. Now here is the conventional wisdom that Nyquist was about to demolish. Engineers of the time believe that a feedback amplifier was stable, even only if the round trip gain was less than one at every frequency. And that is, if the loop gain had magnitude less than one everywhere. If the gain exceeded one at any frequency, then they believed that the signal would grow on each round trip and the system would sink. Nyquist showed that this was essentially wrong, or rather incomplete. He proved that the system could have a round trip gain greater than one at certain frequencies and still be perfectly stable, provided that the phase relationships were right, and conversely a system could satisfy the naive gain condition and still oscillate. This was a phenomenon of conditional stability, deeply counterintuitive, and exactly what the telephone engineers were wrestling with in practice. Nyquist's insight was to look not at the gain at any single frequency, but the entire curve, traced by a j of j omega, the loop gain in the complex plane. As the frequency sweeps from minus infinity to plus infinity. If this curve, what we now call the Nyquist plot, encircles a certain critical point, then the system is unstable. If it does not, then the system is stable, and that's in essence the criterion. Now here's the historical twist. In his original paper Nyquist did not use the so-called Koshii argument principle, we'll come back to what that means shortly. However, the modern textbook derivation of the Nyquist criterion relies heavily on tools from complex analysis, contour integration, wind the numbers, and the residue theorem. But that was not Nyquist's approach. Instead, as noted by Carl Johan Ostrom, Nyquist and a quote, did a direct calculation based on his insight into the propagation of sinusoidal signals through systems. It did not use results from the theory of complex functions. His original proof was, by all accounts, fairly complicated. Indeed it was Leroy McCall, another Bell Labs colleague who showed in 1945 that a short proof can be given by using the principle of variation of the argument, that is Koshii's argument principle. And it was Henrik Bodhi, also at Bell Labs, who shifted the convention from Nyquist's original critical point at plus one to the modern critical point at minus one, by absorbing the negative sign of the feedback loop into the loop transfer function. So the elegant compact derivation you find in every control textbook today is really the work of three people. Nyquist supplied the insight, McCall supplied the mathematical shortcut, and Bodhi supplied the notation. There is also another historical footnote worth posing on. The German electrical engineer, Felix Strecker, working at Siemens in Berlin, independently arrived at a similar graphical stability criterion in 1930, two years before Nyquist's publication. Strecker published his results in the German journal Erek Tricche Nachrichtentechnik. His approach was also based on the polar plot of the loop frequency response, and he identified the same in circumand condition, though expressed in somewhat different notation. The result is sometimes called the Strecker Nyquist criterion in German language textbooks. Strecker went on to contribute to television technology and electronic filter theory. His work is rarely mentioned outside Germany, though, and which is a small injustice of history, while Nyquist's name as Stuck, both because his mathematical treatment was more complete, and because Bell Labs' reach was simply greater. Before we continue with the Nyquist criterion and the corresponding diagram, it's worth posing to introduce another concept on which the whole story seems to rest. The frequency response and why Nyquist criterion proved so useful. The frequency response is one of those ideas so fundamental, so pervasive in engineering that it's worth slowing down for, in case you haven't seen it before. Imagine you have a system, say a loudspeaker, a suspension bridge, the steering column of a car. Now imagine you poke it with a periodic perturbation, say a sine wave, a perfectly smooth, oscillating signal at a single frequency, like the steady ticking of a metronome. What comes out of the other end? Well, the remarkable fact is that if your system is time invariant and linear, meaning its behavior does not change over time and its response to multiple inputs is just as some of the individual responses, each scaled according to the magnitude of the inputs, then the corresponding output is also sine wave, at exactly the same frequency. In other words, the system cannot generate sinusoidal oscillations at new frequencies from a single sinusoidal input. All it can do is make the waves intensity larger or smaller and shift it forward or backwards in time. Engineers call the relevant quantities that change, amplitude and phase, something that I also mentioned earlier, two numbers per frequency. So the frequency response is nothing but the complete catalog of those two numbers for every possible input frequency. From the slowest imaginable oscillation, all the way up to infinitely fast weakling. It is in a deep sense the system's fingerprint. The practical importance of the frequency response and of Nyquist's stability criterion which relies on it is that engineers can measure it directly by performing a so-called swept sine test. Engineers often use simpler tricks to probe systems like this. If you want to understand how a bridge, how a building or a mechanical structure vibrates, you can literally hit it with a calibrated hammer and measure the resulting oscillations. From the vibration that comes back, you can reconstruct how the structure responds across different frequencies. In automotive engineering, for example, a similar idea appears in a more controlled form. When testing the lateral dynamics of a passenger car, engineers program the test bench to add a sinusoidal signal to the steering angle, then they measure the urate and the side-slip of the vehicle at each frequency. The results allow them to build a model of how the car responds to steering inputs and from that design a controller that keeps the vehicle centered in its lane. You never really need to write down a differential equation. You just measure and then plot. In electrical system for example one can measure the frequency response of say an RC circuit with an oscilloscope and a signal generator and then feed those measurements directly into the Nyquist Stability Test. No model required, just data. And as a representation I should name here before we go further. Nyquists colleague as I mentioned Handrich Bodi also joined Bell Labs in 1926 and developed one of the most widely used tools in frequency domain design. Bodi's insight was to display these frequency responses in two separate plots, both with logarithmic frequency axes. One showing the amplitude gain, in decibels and the other one showing the phase shift in degrees. You may justifiably ask why logarithms? Because multiplication of gains becomes addition in decibels which makes it easy to see how cascaded components combine. And because practical frequencies span many orders of magnitude, a log scale fits them all on one sheet of paper. There's beautiful connections here else to music theory but I don't want to digress further. The Nyquist diagram and the Bodi plots are two views of the same and the line information. The same frequency response displayed differently. The Bodi plot is often more convenient for design because its logarithmic scales make it easy to sketch and see how controller adjustments shift the loop shape. The Nyquist diagram is more powerful though for stability analysis because the encirclement criterion has no direct analog on a body plot. In practice, control engineers move fluidly between both views. I suppose the Bodi plot itself deserves its own episode so we'll come back to it hopefully soon. Now to understand the modern motivation of the Nyquist criterion we need to take a detour through 19th century Paris. We need to talk about Augustine Louis Koshy. Koshy was born in Paris on August 21st in 1789 about one month after the storming of the Bastille. His father was a Parisian police official of last disposition in the revolution. The family fled to the village of Arquay during the reign of Terror where the young Augustine was chronically minorised, a condition that would later affect his health for life. But the family had extraordinary neighbors Laplace and Bertue. In Lagrange, a friend of the family reportedly looked at the boys early work and told his father. And here I quote "do not let him touch a mathematical book till he's 17. Not because he liked talent but because Lagrange wanted him to have a broad classical education first. I'm not sure whether Koshy did not listen or simply was advised differently by someone else, but clearly he became by any measure one of the most prolific mathematicians who ever lived. Writing approximately 800 papers covering complex analysis group theory elasticity optics and much else. Mathematician Hans Freudenthal wrote that and I quote "more concepts and theorems have been named for Koshy than any other mathematician. It was also by all accounts quite difficult as a person. It was deeply religious, fiercely loyalist and politically stubborn enough to refuse an oath of allegiance to the French King and spent years in self-imposed exile. Norwegian mathematician Nils Enrich Abel called him and I quote again a bigoted catholic and quote again "mad and there's nothing that can be down about him." But it was also recognized as a towering figure in mathematics. Among Koshy's many contributions is the argument principle which he first presented to the Royal Academy of Sciences in Turin on November 27, 1831, during his self-imposed exile in the capital of the Kingdom of Piedmont, Serdina. The idea is this. Suppose you have a function called it F that behaves nicely inside and the longer-close curve in the complex plane. Technically we should say that this is a metamorphic function. Such a function may have zeros so points where the function equals zero and poles. Points where the function blows up to infinity. The argument principle says the following. If you travel once around the boundary curve and keep track of how the argument, that is the angle of the value of F changes as you go. The total change in that angle is equal to 2 pi times the difference between the number of zeros and the number of poles of the function F inside the region enclosed by the curve. Let me say that differently. Map every point on the boundary curve through a function F. You get a new curve, the image curve in the complex plane. Now you may ask how many times does the image curve wind around the origin? The answer is exactly the number of zeros minus the number of poles inside the region enclosed by the original curve. Each zero contributes one counterclockwise winding and each pole contributes one clockwise winding. Provided we follow the usual convention of traversing the original curve counterclockwise. That's the argument principle. It's a topological result. It counts the difference between the number of zeros and poles by checking how a curve wraps around the point, not by the particular shape of the curve. And as we shall see in a moment, it is the key to the Nyquist criterion. All right, here we go. This is the heart of the episode and I'm going to try to make it as clear as I possibly can without a single formula. Here I'm going to ask you to picture a classical feedback loop. You have a plant, so a physical system you want to control, say a motor, a chemical reactor, an aircraft, and you have a controller. The algorithm that drives it. The controller looks at the error between where the system is and where you want it to be. Computes a corrective action and sends it back to the plant. The plant responds and that response feeds back to the controller. Round and round it goes. Now the question is, will this loop be stable? At its core, this is a question that asks what happens to signals as they travel all the way around the loop. If you inject a sinusoidal signal at some point in the loop and let it travel once around, through the plant, through the controller, and back to where it's starting, what is it that comes back? Is it samplitude larger or smaller? Is it in phase or shift in time? The loop transfer function is the mathematical object that answers this question. At each frequency, it simply returns a complex number. Its magnitude tells you how much the amplitude is scaled in one round trip through the loop, and its angle tells you how much the signal is shifted in phase. Stability in these terms means that the sinusoidal disturbances die away rather than grow. If possible, worst case scenario is this. A signal that travels around the loop and comes back with exactly the same amplitude, nothing lost, nothing gained, and a phase shift of exactly 180 degrees. At that point, what was negative feedback has become positive feedback. The signal reinforces itself on every round trip. The system sings, as they used to say. So the danger point, geometrically, is the complex number that has magnitude one, and phase 180 degrees. And guess what? That number is minus one. Mark it clearly on the complex plane, in a bright danger color. That is the point we need to watch. Now let's walk through the Nyquist criterion step by step. Step one, draw the portrait. So we take our loop transfer function, plant times controller, and evaluate it at every frequency, from the very slowest to the very fastest. At each frequency, you get a complex number, a point in the plane. As you sweep through all frequencies, those points trace out the curve, and this curve is the Nyquist plot. It is a portrait of the open loop, drawn in the complex plane before you have closed the loop at all. Step two, ask this question. Does the curve encircle the critical point minus one? Does it wrap around it the way a ribbon wraps around the post? Step three, count. If the open loop system is already stable, which is the usual starting point, the answer is clean. Zero encircumence means the closed loop is stable. Each clockwise encirclement introduces one additional unstable mode, when the loop is closed. Each counterclockwise encirclement cancels one out. The arithmetic is exact, and it comes directly from Koshi's argument principle applied to the loop transfer function. That's it. No algebra, no eigenvalues, no characteristic polynomial. You measure the open loop frequency response, something you can do with an experiment, without even knowing the mathematical model of your plant, you plot the resulting curve, and you look at how it will be. behaves around one special point. The geometry of the curve tells you whether the closed loop will be stable. This is what Nyquist's paper of 1932 gave to the world. He turned a question about closed loop dynamics into a geometric question about an open loop curve. This is the criterion. And here's what makes it more than just an engineering recipe. The thing we're counting, how many times a curve winds around a given point, is a so-called topological invariant. This means that it cannot change unless the curve actually crosses that critical point. You can perturb the plant model, introduce small measurement errors, vary parameters slightly, even vary the curve. But the encirclement count stays exactly the same, until the curve is pushed through minus one. This is the deep reason stability is a robust property. It just doesn't happen to survive all small perturbations. It must, by the logic of topology. Similar mathematical ideas appear in fluid dynamics as the circulation of a vortex, incondensed matter physics as the classification of defects in superconductors, and in signal processing as the phenomenon of phase wrapping. When we draw a Nyquist plot, encountering encirclements, we are, whether we know it or not, participating in one of the deepest threads in all of mathematics. As mentioned earlier in the episode, the Nyquist criterion has a bit of a bad reputation. In student service, it is consistently ranked as the most difficult topic in an introductory control systems course. Harder than route, or with criterion, harder than brute locus, harder even than body plots. Which naturally raises the question, why is it that this is so hard? I believe there are several reasons. The first is that the criterion really sits at the intersection of complex analysis, topology, and frequency domain control theory. Three subjects that most engineering students have studied only likely when they first encounter this. You need to understand what a winding number is, what a conformal mapping does, ideally, and what it means for a curve to encircle a point. Second, the criterion is inherently visual. To really see it, you need to watch a point move along the imaginary axis while another point traces out the Nyquist curve in the complex plane. You need to feel the curve sweeping around, approaching minus one, maybe wrapping around it, maybe not. Without a dynamic picture in mind, the explanation feels a bit like describing a dance over the telephone. The third point is that, and this is the practical curse, is that awkward details like sign conventions turn the whole result into a minefield. Clockwise or anti-clockwise. Plus one or minus one. Every textbook seems to choose slightly different conventions and a student switching between references can easily get turned around. Let me say a word about two complications that trip up almost everyone in this context. The first is Paul's on the imaginary axis. If the loop transfer function has a pole on the imaginary axis and this is extremely common, because an integrator, for example, has a pole right at the origin, then the Nyquist contour passes through a singularity. And that is illegal for the argument principle. You cannot evaluate a function at a point where it is infinite. The standard fix is to indent the contour. You take a small semicircular detour of radius, epsilon around the offending pole, keeping the detour either just outside or just inside the right hand plane. By convention, the indentation usually goes to the right half plane, so the pole is excluded from the enclosed region. Then you let epsilon shrink to zero. These little indentations contribute large semicircular arcs to the Nyquist plot, often sweeping through angles of 180 degrees or more. And this is where mistakes frequently occur. If you generate a Nyquist plot with a computer, it only shows you the frequency response along the imaginary axis. Those arcs disappear entirely. Matlab's Nyquist command, for example, does not explicitly draw them. You have to account for them yourself. Forgetting to do so is one of the most common errors in applying the criteria. A second source of confusion is sign conventions. Different textbooks count in circlements differently. Some count counterclockwise in circlements as positive, some count clockwise in circlements as positive. The formula changes accordingly. The number of zeros equals the number of in circlements plus the number of poles or the reverse depending on the convention. The only safe rule in this context is the following. Always state your convention explicitly and always test it on a known example. Work through a simple system, say a first order model, where you already know the answer, confirm that your convention produces the correct count, and only then apply it to the system you actually care about. Knowing the system is stable, however, is often not enough. The real question is typically how stable, how much can the world change, a component drift, a delay creeping in, the plan behaving slightly differently than the model predicted, before the Nyquist curve finally reaches the critical point and the system tips over. This is the question of robustness and it is where the Nyquist plot becomes not just a test but a measuring instrument. Two classical measures emerged directly from the geometry of the Nyquist plot. The first is the so-called gain margin. As you sweep through frequencies there will be a particular frequency where the Nyquist curve crosses the negative real axis, where the phase shift of the round trip has reached exactly 180 degrees. Back crossing, look at how far the curve is from the critical point. The gain margin tells you by what factor could you increase the overall loop gain before the curve reaches minus 1. If the gain margin is 6 decibels, you can double the gain and still remain stable. It is a measure of how much headroom you have in the horizontal direction, so to speak. The second is the phase margin. There is also a frequency where the Nyquist curve crosses the unit circle, where the round trip gain is exactly 1, meaning the signal comes back at the same amplitude it left. The phase margin tells you how much additional phase lag could the loop accumulate, say from an unmodeled delay or a sensor that is slightly slower than you thought, before the crossing rotates all the way around to hit minus 1. So the phase margin is a measure of angular headroom. These two numbers gain margin and phase margin are the bread and batter of classical control design. They are the numbers you report to your manager, the numbers you check before you ship. But here's the trap. Both margins alone can look very healthy while the system is still dangerously close to instability. How? Well, if the Nyquist curve swings close to the critical point at some frequency that is neither the real axis crossing nor the unit circle crossing, if it sneaks up on minus 1 from a oblique angle, the individual margins look fine, but the actual distance to the danger point is small. This is why the sensitivity peak is often a better single robustness metric. It is simply the inverse of the shortest distance from the Nyquist curve to the point minus 1. The closest the curve ever gets regardless of any direction. A typical design target keeps the closest approach no near and then about a half unit. If the curve comes within half a unit, then any perturbation that shifts the curve by more than that amount could destabilize the system. This geometric picture, the Nyquist curve dancing around minus 1, a keep-out circle drawn around the critical point, is the foundation of modern robust control. Loop shaping, meonalities, and many other things. The art of designing the controller so that the open loop curve as the right overall shape, high gain at low frequencies for disturbance rejection, adequate roll-off and high frequencies for noise rejection and enough clearance from minus 1 for robustness. Body reformatted the same information into gain and phase plotted against frequency. The body plots as we mentioned before, which are more intuitive for design. But underneath all of these is really the same Nyquist geometry. Now, let me tell you what this means in practice and what this meant at the time, because this is where the story really comes alive. Before the paper became known, control design was to put it bluntly guesswork. In Karl Ostrom's word, run from those years when he worked at ASEA, the Swedish electrical company that is now ABB engineers in 1940s described the state of the art with painful honesty. Design was done by trial and error, computing roots of characteristic polynomials manually with poor computational tools, and no idea of how to change a system to make it stable. And when they learned about Nyquist's paper, the change was immediate. Again, quoting Ostrom, "For the first time, we could determine how to change the controller to stabilize the system. Knowledge about Nyquist's paper." change control from trial and error to systematic design. This is the key inside that separates the frequency domain of first from all earlier stability methods. Ralph and Hervitz could tell you yes or no is the system stable but they gave no guidance on what to do if it wasn't. This approach because it is graphical because you can see the shape of the frequency response tells you exactly where the problem is and what to change. Too much gain at the crossover frequency reduce it. Not enough phase margin well add a lead compensator to inject phase at the right frequency. The plot turns stability from a mathematical verdict into a design tool and during World War II these ideas became militarily critical. Fire control systems for anti-aircraft guns rather tracking server mechanisms and auto pilots all relied on feedback loops and had to be stable and responsive under extreme conditions. Body and McCall published the first textbook covering frequency domain stability methods in 1945 and these ideas spread rapidly from telephone engineering to all of automatic control. At this point you might wonder why in 2026 are we still talking about a result from 1932. The answer is that it aged extraordinarily well in particular because it is rooted in geometry and topology rather than in the specifics of any particular technology. Consider time delays. A pure time delay the kind of delays you would find anywhere signals take non-negligible time to propagate is a deeply awkward object mathematically. It cannot be described by a transfer function that is defined by a finite polynomial. Older algebraic methods are essentially helpless against it but the frequency domain approach handles it effortlessly. The delay simply causes the phase of the round-trip signal to keep winding as frequency increases while the amplitude stays constant. The Nycrist curve spirals inward and you count and circlements as usual. The geometry doesn't care that the underlying mathematics is infinite dimensional. At a deeper level the entire apparatus of modern robust control is fundamentally about keeping the open-loop curve away from the critical point under uncertainty. Uncertainty in the plant is represented as a perturbation that shifts the curve and that the question becomes how large a perturbation can the curve absorb before it is pushed through the danger point. The modern framework gives you more sophisticated ways to quantify and bound that perturbation for richer and richer classes of uncertainty. But the picture is geometric and is always the same. Keep the curve away from minus one. That is the unifying principle. Everything else is really elaboration. Before we go any further let me say one word about the man himself. His daughter Phoebe left us a portrait that is worth quoting at length. She said and I quote the following. I think one of the ways he succeeded in accomplishing so much is that he was always very disciplined. The alarm always rang at 6.45 and our house. He always got upright. While mother fixed his breakfast he got dressed and was out the door punctually at 7.30. You could set your watch by it but he never heard. He walked a mile to the train station and rode into New York City. His return home was just as regular so we could plan on eating at 6.15. Some days he took the ferry across the Hudson River instead of the tubes under the water because the air was fresher. He did like being outdoors. I have a picture of him stretched out on the lawn taking a nap. He always reserved Saturday for household chores and Sunday was church. A good dinner and time to read or think. He frequently had a legal size yellow notebook on his lap and started figuring those equations. I suppose he couldn't let go of a problem until he got it solved. There it is. The yellow notebook on his lap, the equations that wouldn't let him rest. And yet when he was asked about life, this man with 138 patents, this man who had fundamentally reshaped communications, noise theory, something theory and feedback stability, his answers were astonishingly modest. On impressions as an immigrant, in a quote, he said, "I don't think it was any different from what I would have been if I had gone to Calstad or Stockholm." On the awards, he mentioned, "I have received honors for technical work." On his publications, he mentioned, "I have published a few technical papers. On his inventions, I have been granted a number of patents." In 2005, the American Society for Mechanical Engineering ASME established a lecture in his honor. The inaugural talk was given by Carl Ostrom himself in Orlando, Florida, under the title, "NiQuist and his Seminal Papers," which I highly recommend checking. Ostrom in his tribute observed that he was, in a quote, "a much better mathematician that most men untackled the problems of telegraphy, and he had remained a clear, original, and philosophical thinker concerning communication." He tackled the problems of telegraphy with powerful methods and with clear insight. "NiQuist received the Stuart Valentine medal from the Franklin Institute in 1960, the IRE Medal of Honor that same year, the Founder's Medal of the National Academy of Engineering in 1969, and he was only the first person to receive it." And the Rufus Olderberg Medal of the ASME in 1975 shared fittingly with Enrich Bodie. He retired from Bell Labs in 1954, spent his later years consulting for the Department of Defense and died on April 4, 1966 in Halingen, Texas, a D.A. of '87. Now a word about what came next. After NyQuist, the natural question was, "Well, can we do this for no-linear systems?" As already mentioned in the show, the setup is the so-called Luriae problem after the Soviet mathematician Anatolia Luria, who formulated it with his colleague Postnikov in 1944. You have a linear term invariant plant in the forward path and the static nullinarity in the feedback path. The nullinarity is unknown, and static here means that there is no memory, so it's not a dynamic element. You don't know it's exact shape, but you know it lines within a sector. The ratio of the output to input is always between some minimum slope, call it K1, and some maximum slope, call it K2. The question is, "For all nullinities in this sector is the closed-loop system globally stable." This is called absolute stability. In 1949, the Soviet control theorist Mark Eiserman posed a seductive conjecture about this problem. The conjecture is the following. If the closed-loop system is stable for every linear gain in the sector, replace the nullinarity with a constant, and the system is stable for all constants in the relevant range, then surely it should be the case that the system is stable for all nullinar function in that sector too, right? The answer beautifully and frustratingly is no. Counter-examples exist and also similar conjectures exist. For example, the Kalman conjecture. A system can be linearly stable across the entire sector and yet harbor a hidden limit cycle for certain linearities. This failure was enormously productive because it motivated the search for valid sufficient conditions. And the first great success was the circle criterion. The idea is that a direct generalization of everything we have talked about today is essentially very simple geometrically. Instead of a single critical point to avoid, you now have a critical disk, a circle in the complex plane whose diameter is determined by the bounds of the sector. If the open-loop frequency response curve stays entirely outside the disk, the system is guaranteed stable for every nullinarity in the sector, regardless of its exact shape. The critical point has become a region in the complex plane. The geometry is the same, the scope has widened. Then in 1961 the Romanian mathematician Vasyl Mejai Popov found something even sharper. By a clever transformation of the frequency response plot, tilting it in effect, you obtain a less conservative condition that could be checked with a straight line rather than a circle. The result is remarkable. Stability of an linear system guaranteed by a simple geometric condition on a modified plot. The intuition is the same as before. The curve must stay on the right-hand side of a line, but you're still fundamentally keeping a curve away from a dangerous region. It turned out later that Popov's work connected beautifully to Jan Wilhelm's Disciptivity Theory in 1972, the idea that stability can be understood through energy and storage and dissipation, and to desalibrate it Kalman-Yakobovic Popov-Lemmer, which bridges frequency-domain conditions and leopon of functions. The intellectual line runs from 1932 through Popov, 1961 to Wilhelm's 1970s. And the combination perhaps came in 1997 when Alexander Magrowski and MIT and Anders Ranzer at Lund University published the IQC framework, integral quadratic constraints. The idea is beautifully general. You may remember that from our episode with Anders Ranzer, if not, check it out. I'll summarize it here briefly. Essentially, any uncertain Lundiner component in a feedback loop can be described by a quadratic inequality on its input output signals, and stability of the interconnection can be checked via frequency domain condition. The classical stability criterion, the circle criterion, the pop-up criterion, and the small gain theorem even are all special cases. One framework may results all rooted in the same geometry of frequency response. The IQC framework is in a sense the combination of the single loop story. And now you should justifiably ask, what happens when the loop as many inputs and many outputs simultaneously? Well, for systems with multiple inputs and multiple outputs, so-called MIMO systems, a single curve in the complex plane is no longer enough. In 1977, MacFarlane and Possilwhite at Cambridge published the generalized criterion for this case. The idea is very elegant. Instead of a single complex number at each frequency, you compute the eigenvalues of the loop transfer matrix. Each eigenvalue traces its own curve in the complex plane as frequency varies. These are called the so-called characteristic lock-eye or eigenlock-eye. Stability is determined by the total number of uncircumence of -1 by all lock-eye together. This is the natural matrix generalization in this mathematical foundation for modern multi-baraboo robust design. And this is something that we have discussed a large also in our episode with John Doyle, where we discussed new synthesis. Really the robust version of what I just described. Please check out that episode 2, it's really fun. I want to close this story by mentioning that in the last few years something new and unexpected has happened. The trio of researchers, Ernest Riu, Robert Hanna and Wotau Yin, introduced in 2019 what they called "scale relative graphs" or SRGs. Originally, these were tools for analyzing large scale optimization algorithms with no connection to control whatsoever. A scale relative graph represents an operator as a region in the complex plane, based on how it scales and rotates inputs relative to outputs. Notice that the domain of this operator can be in principle any Hilbert space. And so in this context, my past colleagues, Thomas Chaffey, for the Forny and my past supervisor, World of Supporter, realized that for a linear system, the scale relative graph is intimately connected to the frequency response curve that we have been discussing throughout this episode. And in particular, for nonlinear operators, it generalizes that curve to a full graphical representation, providing us with a sort of nonlinear version of the Nyquist stability criteria. Stability of a feedback interconnection can be dust checked by asking whether the SRGs of the two components are separated in the complex plane, a condition that reduces again in the linear case to the classical and circumvent condition. Today, this is a lively research area, as recently as 2024 and 2025. Researchers have extended SRG analysis to obtain generalized circle criteria and have developed SRG-based notions of nonlinear bandwidth and nonlinear body diagrams. The diagram born in 1932 to prevent telephone amplifiers from singing is being reinvented as a universal language for nonlinear systems analysis. Geometry is really the same, but the scope has grown to encompass much broader classes of systems. Before we close, I'd like to spend a few words about the common pitfalls and misconceptions about the Nyquist diagram. Every practice in control engineers fallen into at least one of these traps and most of us are fallen into several. 1. Confusing the stability criterion with the sampling theorem. Same person, completely different results, the sampling theorem is about how fast you need to measure a signal, the stability criterion is about whether a feedback system oscillates, the only connection is the manual produced in both. 2. Sign conventions. Always always state your convention. Clockwise and circummence positive or negative. If you have a point at minus 1 or plus 1, check on a known example before trusting your count. 3. Forgetting the indentation arcs. If your system is an integrator, the contour has to detour around the pole at the origin. This detour produces a large semicircular arc in the plot that sweeps through 180 degrees or more for repeated poles. If you rely only on a computer generated curve without adding these arcs, by hand, it will miscount and circumference. 4. Trusting the two margins blindy. Both gain margin and phase margin can be large while the curve still raises the critical prime form of the oblique angle. The sensitivity peak, the inverse of the minimum distance to minus 1, is a much better single metric for robustness. 5. Conditionally stable systems. These are systems that are stable only for a range of gains. Increase the gain and the system destabilizes. But decrease it too far and it also destabilizes. The plot shows this clearly. The curve encircles the critical point at low gain, clears it and moderate gain, and encircles it again and high gain. Single margin analysis which considers only one crossing can completely miss this double edge structure. 6. Systems with time delay. The curve for a delay system spirals infinitely. The phase keeps winding and if you sample the frequency grid too costly, you will miss tight loops near the critical point. Dance frequency sampling is essential here. 7. Non-minimum phase zeros. Those in the right half plane constrain the achievable loop performance in fundamental ways. They impose a lower bound on the sensitivity peak, limiting how robust the system can be. The plot makes these constraints visible, but they are easy to miss if you're not looking. 9. Now let me end where Nyquist began. With a copper wire stretching across a continent in a question. Will this thing sing? Would those 21 pages in 1932 gave us was not just an answer to that question. It was a way of seeing. A way of looking at the feedback system from the outside, tracing its frequency response, watching the curve it draws in the complex plane, and reading its deepest property, stability from the geometry of that curve. No eigenvalues, no polynomial root finding, just a curve, a point in the count. It is a standing example of what using BigNerd call, in a quote, the unreasonable effectiveness of mathematics. A topological argument developed by Koshy in the 1830s in the context of pure complex analysis turned out to be exactly the tool needed to save long distance leftony, enable the feedback amplifier, and lay the foundations for control engineering. And not just control engineering. The same geometric thinking now underlies robust control, non-linear stability theory, modern optimization, and the analysis of algorithms that power machine learning. Good mathematics doesn't expire. Nyquist would probably shrug at all this. He would probably say, with that characteristic modesty, I'd be granted a number of patents. He would go home, eat dinner at 6.15, read on Sunday, and hope he is yellow notebook. And we get to stand back and see the full picture. A poor fanboy from Varmaland, Sweden, coached by a rural school teacher crossing an ocean with $10 in his pocket, landing in a stable in Minneapolis, working his way through college and grad school, walking into the most extraordinary laboratory in human history, and producing quietly, methodically, with powerful methods and clear insight, ideas that the world is still building on a century later. That is the story of feedback, stability, and the winding number. Thank you for listening. I hope you liked this show today. If you enjoy the podcast, please consider giving us five stars on Apple podcasts, follow us on Spotify, support on Patreon, or Paypal, and connect with us on social media platforms. See you next time.