26 - Artificial Intelligence and Aviation

[Music] Altitude is a monthly podcast brought to you by Nats, the UK's leading air traffic control company. In the show, we talk about current and prominent aviation topics. In this month's episode, we're exploring the use of artificial intelligence in aviation. Enjoy the show and find out more at naz.airro/altitude. Hello and welcome to altitude. I'm your host Russell Portron. Today we'll be talking about artificial intelligence, creating art, writing stories and sometimes even driving cars. AI is all around us. But what does it mean for the aviation industry? We're joined by three experts to answer that question. Marco Rukert, Vice President of Technology at Syridge Technologies. Hello Marco. Emily Price, Director of Analytics and Nats. Hello Emily. And Ben Carvell, AI Design Lead at Project Bluebird. Hello Ben. Hi Russell. Just a quick reminder for those of you watching we are live so you can post your questions at any time. So please do so and I'll do my best to make sure that our panel gets to them. But let's start with some introductions. Emily, could we start with you please? Of course, as Russell said, I'm Emily Price. I'm the Director of Analytics at Nats. And our analytics team essentially convert data and information into recommendations and insight. So that's for customers. Excuse me within that. But also for customers around the world. And so clearly providing that and optimized decision making for them. AI can be really useful. See looking forward to talking about that this morning. Thanks Emily and welcome to altitude. And Marco. Yeah, I Russell. Thanks for introducing me. My name is Marco Rukard. I'm the Vice President of Technology at Syridge Technologies. So my my really really sees me oversee the software development and anything technology that goes into operational systems. And our research and development and innovation systems with our customers worldwide. Focus mostly on digital towers, but we also have some airport systems. Thanks Marco and also thank you for taking the time to join us from across the Atlantic today. And Ben. Hi Russell. Yeah, hi, I'm Ben Carvelle. I'm one of the leads on project Bluebird. Project Bluebird's a really exciting collaboration between Nats and the Alan Turing Institute. An extra in Cambridge University is looking at applications of AI for tactical control. Fantastic. Welcome all to altitude. Great set of introductions there and I'm really excited about this episode. It's definitely a topic that's in the in the media at the moment. Marco, could we start with you please and could you tell us a little bit more about Syridge Technologies and then really what AI means in the context of your business? Yeah, for sure. So Syridge Technologies is a private company where we had quoted an Ottawa Canada. We are now 100% owned by Nats. So we are a very happy member of the Nats Group. What we mostly do is our core technologies for digital tower. So you can imagine digital towers as we install CCTV cameras at strategic locations around arrowed rooms. We stitched those cameras together and then we will provide either remote control of that arrowed room or we for larger airports. We also provide a hybrid digital tower where we bring video into the tower to enhance the visual line of sight. For our traffic controllers, we also apply that same technology to airport operations. For example, we have some installations in the US where we do virtual ramp control using the same technology. So in our context, AI really means machine learning. We use machine learning to train different models on the best amount of data set that we have access to with our customers. Predominantly, we use it in the visual domain. So we started using that about 2016 with the sort of coming up of ImageNet. We started looking at using AI to recognize objects, aircraft vehicles in images and then drive some intelligence and improve the visual tracking capabilities of our systems. Fantastic. Thank you, Marco. How much in digital towers there? How many around the world are using that type of technology? How many airports? Digital towers overall. I know we have about 50 customers globally. There are some other competing companies as well. So I would probably guesstimate around 100 installations between POCs and full operational systems. There's probably 100 digital towers around the world. I've also heard about something called Amy. Is Amy a personal or a technology? Can you tell me a little bit more? Yeah, so when we first started with AI, we wanted to come up with a unified brand for anything with AI. I mentioned a bit the visual spectrum that we use using AI for visual analysis. But we also have models that are trained on surveillance data. For example, we have a model that's trained on a year's worth of AC and GCS data to predict runway exits. We have the same on a voice. We actually did a collaboration together with NATS on understanding pilot to add goes speech and turning that into text and running an LP on it. So Amy is actually based on Amelia Earhart when we first started. So we are probably trying to humanize AI a little bit. But really what we were trying to do is create a unified brand for anything AI. So that when our customers understand that something is powered by AI in terms of our products, they understand that they can expect a certain level performance in it. A rigorous training that went into the AI. So it's more of a brand than trying to humanize the AI. I also love the link back to aviation there as well. It's really nice. What's the long term ambition them for Amy? Yeah, so our long term ambition is really we are putting more and more AI into more and more products. So we started really with the visual spectrum because we had a product that that visual tracking already. We're now getting really into surveillance. So optimizing the ground traffic. We have a product called traffic light automation system where we turn signals for service roads red or green to either let vehicles pass or not pass to cross the taxiways of aircraft. So we're trying to combine a lot of the different data domains. So doing that on a visual aspect lets you turn your life traffic lights on and off as you see it. But if you can also listen to the RT between the pilot and the at go, you can understand clearance is given before the aircraft is actually turning or not turning. So if we're supposed to turn it to tax away Charlie, but there's a clearance, the last minute change. We wouldn't want to turn that service road green. Sorry red to impede flow of traffic if we know that the aircraft is actually taking a different route. So it allows us to have a bit more of a time horizon on decision and really ensure not only safety, which is number one, but we also want to make sure that the big hub airports have the efficiency. So they can serve more customers and increase their capacity. So the long term ambition really to summarize this, we want to be able to help the aviation industry scale back to even beyond pre pandemic levels. So doing that in a way to allow the human operators to be more efficient and control more traffic and increase the capacity. Thanks Marco and as so as Amy's being used to help me out here a little bit then is Amy learning from the data that is kind of building up in the system. And I guess are you seeing better better decisions being made I guess is the outcome. So we have definitely seen that when we train Amy from from different sites that it is able that the AI model is really able to generalize better. So for example, if we started with aircraft only that have the air Canada library in our first few season Canada. Injecting some some different types of aircraft from Emirates from Lufthansa really allows the AI to to generalize more and not only learn the colors of air Canada. So we really seeing that the training data that will broader becomes the more diverse and they need to train data becomes the better the AI model becomes. And one big distinction I want to make sure is that the audience understands we don't have the AI actually learning during the operation. So what we do is we train the model we freeze the model we run certain tests of performance and we do our safety assurance on that. So we want to make sure that we don't have a situation where you have the input coming into the model and you can get two different results coming out. So we make sure that the model is frozen it doesn't learn during the operation because that would be quite a challenge to get that to the safety regulator. Thanks Marco. So talking of complex models Ben I've been reading about project Bluebird and its mission to create a digital twin of UK space. Can you tell us a little bit more about Bluebird please? Yeah, thanks Russell. Yeah, I certainly can. So as I mentioned before, so Bluebird is a big collaboration research project. It's a five years in duration. We're currently a couple years in with the partners being Nats as the industry partner and then the Alan Cheering Institute up in London being the National Institute for AI and Exeter University Cambridge University or working together on this problem of how you might start to apply artificial intelligence to the world. To the task of tactical air traffic control. So in short, the programs split into three primary pillars, or we call them themes. So we've got three research themes. The first of which is the construction of the digital twin that you mentioned. So the best way to describe that is it is like a it's a simulator, but it's a simulator that says close to the real world as we can get. So we take advantage of all our data embed that within it and try and get a simulator that says close to the real world as we can. So that's our first theme. The second one is the construction of. artificially intelligent agents to then go and control traffic within that simulated environment. So that's the theme too. That's what I'm leading on. And then the final theme is one which I think will touch on together quite a bit later, which is all about trust, explainability and how you start to embed those kind of values in these kinds of systems. So we're trying to make sure we've got all the basis covered essentially. So we've got the simulated environment, the agents within it, and then how we start to embed trust and explainability within them. Thanks Ben. And when you mention agents there, I can't help but think about the film I robot, which I know is obviously based on a book as well, but you know with human like robots working kind of hand in hand with human air traffic controllers, are you building robots? Not quite, although maybe that would help making them a bit more relatable. I don't know maybe we should look into that, but I think one of the big challenges with this is how you make these systems a bit more understandable and relatable and able to look into exactly what they're doing. So they're just computer programs at the moment, but we've put a lot of effort into the interfaces, which we can understand the processes that they're, you know, the things that they're doing, the way that they're controlling traffic in this environment essentially. So we've ran some simulations over the summer and we put a lot of effort into creating some interfaces that look like essentially what you'd see if you were to walk into operation. So you've got radar screens, you've got strip information displays that tell you about the aircraft that are being controlled, and then you've got other panels that link directly into the agents that we're building, that'll tell you what they're up to. So it'll give you a list of the actions they're performing, and you can also go and perhaps interrogate the plans that they're making for aircraft. So that's kind of the way we interact them with at the moment. It is through a computer, but we're trying to make it as easy as possible to understand what's happening. And how does their decision-making and planning compare to an actual air traffic control? Have you done that analysis? Have you verified that? Yeah, absolutely. So I mentioned the simulations we've run over the summer. That was kind of the key part of that was to start bringing some of that operational expertise into the loop of this process so that we're not just off doing research in isolation. We want to have operational expertise embedded right the way through the program. So we ran a human in the loop test. So we were running our, we had our simulator run up, we had a couple of sectors running. And on the one hand you've got an agent controlling one sector alongside a, but one of our real echoes controlling the other sector. And then we also had some great help from some of the training section to come and do essentially like an assessment of the agent as if it was a controller. So we're getting that kind of feedback, which is really helps with the development. So it's not just to see how we're doing the feedback we get from that gets folded back into the development and informs then how we develop some of these techniques. That sounds absolutely incredible. How have the real human control is reacted to this type of work? How are they, how they're finding it? Well, they've been essential, I think, is the first thing to say, you know, you really, you really can't do this kind of work without that kind of support. So we, as a project, we're incredibly grateful to get the support we did over the summer because it's a busy time. But I think we had quite a few people come through because of the way the watch system works. We had a lot of different people come in and get involved. It was great to see how willing people were to engage with the technology once they sort of understood the motivation behind what we were doing. Yeah, they were really, they were really helpful, the really keen to come and give their insights and help us with the development. So yeah, but it was, it was com really well. Nice. You mentioned the Alan Turing Institute, Exeter University, Cambridge University. That's a real powerhouse of knowledge. What's it like or working with those institutions? Oh, it's just great, definitely. They, they, you know, these institutions exist to be at the forefront of this kind of technology. That's, that's their purpose. So if you're looking for the bow wave, that's, that's where it is. I think the thing that's been really exciting and that's made it really great to work with them is that there's been a real willingness. I think from both sides, for us to sort of step into each other's world. So we're on the industry side, we're trying to learn as much as we can about the, about the technology and how it works on a technical level with them, but they've also stepped into our world a little bit as well. So we've had them in operations, we've had them having familiar sessions with our operational controllers and training. And so it's that kind of fusion in the middle, that's where the magic happens, I suppose. So yeah, it's worked really great in that regard. Fantastic. Great stuff, Ben. Emily, it's been, I think, credible to hear from Ben about all of the work that's going on in Bluebird and the potential that that holds. But I understand that from an analytics perspective at Nats, AI is already being used. Can you tell us a little bit about a project? I believe it's called Michelangelo. Can you tell me about Michelangelo? Yeah, sure. And interestingly, I think the timings are fairly similar to what Marco was talking about there. So our team first started using and creating AI models back in 2017. It's probably fair to say that to begin with, we would be using that to look at one element of performance at a time. When the team then designed a project Michelangelo, the tool called Michelangelo in 2019, that was probably the first time that they were starting to bring together multiple aspects of that performance. So if I just describe a little bit about why it came about, we were looking to firstly be able to understand the drivers for Etraf Controller workload. So we were collecting workload surveys from our operational Etraf controllers after they'd all plugged from live sessions in operations rooms and synthesising that with data from our systems. So AI is clearly a really useful tool for doing that kind of thing. And not only will we be able to predict the workload based on using that system data and using those techniques, we will also then able to extend it to use that predicted workload to then further predict the safety, the environmental aspects and the service elements of what we deliver. So on the regulated side, there are typical regulated metrics and revolve around those three things. So it's become a really powerful tool, especially in our portfolio projects that we're bringing in as a business to be able to assess the impact of a single change, whether to airspace or to a system on all of those aspects at once. So on the regulated metrics, but also on that impact of the human. And so I think that's a really interesting shift within that two-year period and to go from looking at one element, looking at all of it together. Yeah, there's a really holistic view, isn't it? So I'm right in thinking then that Michael Angelo allows you to kind of think about how the tech comes together ahead of deployment then, okay? Absolutely. So it could be six months ahead, it could be six years ahead in helping that portfolio team work out, you know, are there plans sufficient, are they necessary? We potentially going to be over delivering in one area and also the timing, what would be optimal, as you say? And where did the name come from? The link of it's exactly what you described as also the big picture. So if you're familiar with the painter, Michael Angelo, he's got some huge paintings. And my understanding from the team at the time is they named it as being a big picture, but part of an even bigger picture. So some of those Michelangelo paintings paintings within themselves, but actually part of much bigger picture on chapel ceilings, for example. Yeah, that's fantastic. Your team are also involved in the development of the demand capacity balancer, not such a nice name, but I gather a really interesting piece of technology. What does it do? Tell us a little bit more. Yeah, so it's rather than the regulator side, the business that operates in the airport's domain, and although the name isn't quite as capture, it does exactly what it says on the tin. So it's there to balance the demand for aircraft coming in and out of an airport with the capacity that you have. So the capacity is fairly finite. So how do we help people to balance those two really efficiently? And so if you imagine a busy airport, there's so much information, so many things changing in any given day, that's where DCB would come in. And I know I've certainly heard it described as providing predictability when you know that every day is going to be different. I think when you start with that assumption of every day is going to be different, you quickly realise that AI is definitely going to be your friend in that situation. And so again, similar time frames, you know, the team have put together those algorithms back in 2017. We worked with our partners, for Quent has also gone to develop them further and rolled out that tool initially to Heathrow Airport. So listeners will know that that's a particularly busy airport in the UK. And the fact that those those operational efficiency cell staff are are using that today to help them make those really confident decisions. So it takes in similar to what Marko said years worth of historic data. It's trained on that. It learns from that. But you'll then batching that with your new real-time data that's coming in. So as you know from anybody that's been on a fly, you know that delays can occur from anything from whether to passengers, being late to a gay, to regulations elsewhere in the network. So the fact is it's able to take in all of that information, but turn it into something that that pops efficiency cell staff member can use to make decisions. And so it's ultimately helping them decide do I need to intervene, head away smooth what I'm expecting and to see for the day. And it's actually allowing them to to trade off lots of different options at once. So you can have up to 10 different options. And and that runs from what's going to happen today out to what's going to happen six months from now. So really flexible proactive tool that's crucially helping them get the most of what they what they have into as a capacity. And providing better service to the airlines. It's great from a passenger point of view, reducing environmental impact, for example, by reducing the delay and ultimately batch that predictability. So if those staff are more confident in knowing what's going to happen that day, they won't need to make as many cancellations, for example, just in case. So really powerful tool that operates over that really wide time span. Thank you, Emily. And Marco, do you see some of the benefits that Emily's talked about there in some of the applications that you have in with Siege? Yeah, yeah, absolutely. I think that what Emily mentioned, especially both the emissions that that is one of our goals as well, what we're really trying to do is make the operation much more predictable. So we try to increase that time window of decisions so that you don't have to go to the conservative approach and say, I'm going to cancel some flights. I'm going to diverge to different routes. But I build up the confidence and the tool so that I trusted decisions and the recommendations that it made. So for example, we have a installation in the Middle East where we use our traffic light automation system. And we instead of going fully automated and just letting the human be the supervisor at the beginning, we went to a semi automated state where the human had to confirm all the recommendations from the AI so that really let us build up the trust from the users. We're coming in and we're installing a system, but it's still the user in the end that is responsible for the safe movement of aircraft on the surface. So it was really essential to bring them along the journey and involve them in workshops early on. Make sure the UI just like Ben mentioned shows them the under the hood. It can't just be the final decision, but do you see the AI actually making the decision and is there is there a chance it's making it based on glare on on the aircraft or not? If you can look a little bit under the hood, it just gives the user a lot more confidence in the decisions that I made. I'm getting a very clear picture of a very kind of careful step-wise approach to the adoption of AI and aviation. Is that consistent? I've never seen lots of nodding heads. Great stuff. Because I've asked about names at Michelangelo and Amy, I think I missed asking Ben about where Bluebird comes from. That's a good question. Is it an unknown? I wish I had a crap story to tell you. The answer is that we just being aviation you naturally lead towards, it's nice to have bird names for everything. So it actually isn't just the project itself, all the little elements that sit underneath it, they all get their own little bird name as well. So the simulator package is called Starning and then every time somebody creates a new agent, that'll take a name as well. So our best one at the moment is Falcon because it's a good, evocative name for an agent, but then we've also got, there's a magpie agent, there's a puppet, any different approach gets another name, so that's it really. It's just leaning into the whole bit, nice things at fly. I love it. I'm glad I asked. Thank you. But sticking with you, Ben, I've been reading some of your quotes and you said that artificial intelligence is really what we use when nothing else will work. Can you expand upon that for me? Yeah, absolutely. So it's almost, this is just a slight cautionary thing, really. The other thing everybody here has a belief in the the power of AI to do some really incredible things. It does come with some caveats and it's things that I'll chuck over to Marko shortly on this one, I think would be good. But if you're going to use this technology, then there are caveats to making use of it in the areas of both explainability and in regulation that you just need to be aware of when you're choosing to make use of it. So all that quotes men say is that if there are other things that can work, if you've got simpler approaches, which are going to work just fine, then go use those. It's fine. That's okay. But if you get to the point where those traditional approaches aren't helping you, then it's worth maybe taking a look at some of these more advanced techniques with the understanding that then you've got that burden of maybe it being a bit more challenging to explain exactly what that model's doing and also that the regulatory frameworks you've then tried to fit in with tend to be less mature essentially. So that's it's reflected in the way that we're approaching things in blue. But so I mentioned we've got a little family of agents now with their own little names. The reason for that is that we're not just looking at the stuff that makes the news. We're also looking at some more traditional approaches as well. So essentially, like rules based approaches that you might have seen the plenty of time ago, just to see how far we can get. And naturally the thing you come up against is issues of scalability. Dealing with uncertainty, the kind of thing that we then try to rely on some of these more modern technologies for. But you just need to be certain where you need to use it because it is a bit of a double edged sword. You come up against those issues that I mentioned. So yeah, thanks Ben. In there, you used the word explainability. What about AI? Is it that you're explaining? I think you touched on it about the actions of the model. Is there more to it? Yeah, so it was one of the really interesting outcomes from the simulations we ran over the summer is that one of the key criterion for controlling is planning and being able to plan effectively and have a list of it essentially be able to say what your plan is for the sector is how you assess a assess a human controller when you're training them, you asked what they're planning to do for all the aircraft within the airspace and then hopefully they got good answer. That's a bit more challenging to take out from some of these techniques, but it's something that we have put a lot of effort into. So as part of the development of the simulation, we added some interface to be able to interrogate what the agent's plans are for all the aircraft within the airspace that it's controlling. So yeah, it really helps when you're starting to involve the experts in that decision making process and understanding how the agent's operating. So it's it's quite a key component of a system. I don't know Marco Emily want to jump in on that one because I'm sure you've both got plenty to say. Yeah, sure and just on the explainability, I think when we first started using AI, it was really at the early beginning and we really got asked the question right away from regulators as well as this shows what you're training on, explain to us how the decision is made and at the beginning we had no idea. So the best thing that we could cover with is just start blacking out some of the part of the training image. So we learned really quickly that when we started training on aircraft in Canada, funnily enough, it learned that every aircraft was had to be in snow because we were only showing training images with snow. So we ended up actually going into a model where we could black out all of the wings and the tail and it would still say it's a plane. So then we realized that really quickly that it was important to analyze really what is being learned and making sure that you have a diverse training set. So if you bring them our middle east images in there and that's sort of a touch on the diversity of training images bringing in a little bit of a sandy background that really helped the model learn the actual aircraft and not the background. Nowhere in the sort of second generation AI where we can really train only on the aircraft and not the background but at the early beginnings that it was really important to understand what are you actually training on and what part of the data is the decision being made on. Yeah and I was just going to say what's all completely agree with the points made about explainability right from the outset that's been a key requirement for our team and I think like been said you know if that means that you're able to use simpler versions of the models because it gives you that explainability and certainly for my team's point of view we're not providing anything that's you know a safety critical system that's in live operation we're providing it as a decision support to that human to be able to turn that massive information into something that gives them that confidence and make a good decision. I'm pleased about I asked about explainability now it felt like a crucial concept hidden in there so thank you all for that and I'm going to cut back to one of your quotes Ben another one I've been trawling them so use also said we can learn things in days it might have taken months or even years to do beforehand so in hearing that it kind of made me think well it sounds like AI is very useful in the development of systems but can AI really deal with the complexity of air traffic control if it's left to its own devices. So this is a great question because it's sometimes quite difficult to articulate what the work looks like in Bluebird we talk about these threads of development that are going on but in terms of where the really difficult stuff is exactly what you've just mentioned right it's how do you link up this technology with air traffic control that's where the trick is so I think I'm sure Emily and Marco have had the same experience it would be lovely if you could just take something off the shelf and plug it in and it goes and learns how to do everything and you can just sit back and sometimes I think that's the way it gets portrayed in the media as well that it's some kind of all powerful entity you can just hand the task over it's not quite like that and however advanced it is it's always still essentially just a computer program that does what you ask it to do so it might be able to do some very clever things but you have to ask the right question fundamentally that's the that's the trick so defining what good air traffic control looks like that is the work of Bluebird right so by defining that environment properly imposing the question correctly then you start to see interesting behaviors come out and be developed but really it won't learn it all by itself and that's why we've had such strong involvement with operations because you have to go one step further than the surface level understanding you need to get right into the core of what it means to do air traffic control draw those elements out and start to try and find ways to represent them in a way that means we can then start to apply these technologies and see if we can develop effective behaviors but it yeah it's it's really that's where the trick is it's not just the technology it's posing the problem that you've got in a way that you can then link it up and have it learn so yeah that's the challenge absolutely thank you band great great answer Is there any Mark, was that a build I was so there? Yeah, I think that's a really good point then, mate. And I think I'd like to touch a little bit on the training data as well, especially if I can give an example of our voice AI. When we started looking at translating what the pilot is saying or what the ad goes saying into text, we thought as well. It's everybody saying, well, just let's just do Google's language model, no problem. Let's translate that. And as soon as we started putting in the voice from the pilot, especially to the app goal, the first thing that we learned is that the signal to noise ratio. So there was a lot of grainy transmitting that over over still analog channels. It was completely throwing off the model. It's not the same quality of voices you're talking to your Google homepad or anything like that. And then we also learned there's a lot of jargon. So it actually took a lot of effort from the operations people as well to translate it into text, because I couldn't, even as a programmer, I couldn't understand all the nuances, all the different jargon and putting that then into the training data. So it really comes to the point of, although the models exist in academia and not all the big tech companies are doing a lot of good things, but there is that specific operational context of ATC that there isn't anything off the shelf that you can pick up. Even the visual aspect, it can detect certain types of aircraft. But then when you come into specific settings, we really always have to fine tune and build that rule set of the operation on top. So it isn't just a plug-in play. Thank you, Marco. I'm getting a real sense of the detail and analysis has required in order to work with AI and aviation. This is fantastic stuff. It's a great conversation so far. Thank you very much. We've got a few questions now, perhaps from the audience. So the first one I'd like to go with is from Oli. Oli has written great AI outputs require very good data inputs. I'd be interested to learn what is being done currently to prepare data for these use cases. Who would like to take that one? Yeah, I don't mind that, Russell. Go ahead, Ely. It's going to be a bit to talk quite generally, I guess, from the beginning on that one. So I think we talked earlier. The first rule would be really clear what it is you're looking to do. So depending on what your question is, just write us off, you may select different data, for example. We're very lucky in our team that we have great access to high quality data. And then I'm sure he uses lots of similar data. But what that means is because we're not just using AI for all of our pieces of work, we have those subject matter experts that are using the data. They understand it. They built up that trust. So actually that kind of gives you that fast forward when you're using it in some of your AI programs. So I would say if people don't have those SMEs, then reach out to all the people, but that's in your business or outside. I think checking also that you're going to have those inputs when you're actually running a model in real life. So for example, if you're using weather data, don't reign it to expect data every second if it's only being updated every half an hour, for example. But I think probably the most critical thing when it comes to data is sense checking, validating. Does it make sense? And does it make sense not just from people who are in that technical space, but those end users that you're ultimately delivering at two? Thank you, Emily, perfect answer. Next question from Jedaya. Similar to autonomous driving, simpler environments are usually a lot more achievable for AI agents, a motorways to city driving. Do we see a similar effect in ACM? Great question. I'll happily jump in on that one if that's all right. Roswell, so we're going to bend here. So yeah, it's an insightful question, definitely. As you can imagine, trying to bite off the whole of air traffic control in one go is a could be a bit of an intimidating thing. So as we were getting rolling on Bloober, one of the early things we did was exactly as described. You start with some toy environments, some simplified environments to try and get things working. And indeed, training, even some of the more exotic machine learning techniques on those simplified environments, you could get quite good behaviors quite quickly. So you just draw some simple geometric shapes, have some simple aircraft models. And yeah, because there's no uncertainty embedded in there, because there's no variability in the model, then you can learn to do what looks like on the surface quite good air traffic control. Now that was one of the motivators that then pushed us to go and start using real world data for the sims that we've just run over the summer. So we had a real big push to try and get towards something that was a bit more representative. And then you start getting into the real meter the problem, which lives in those uncertainties that you have to deal with in the data. So yeah, it's absolutely a very relevant question to ask. Yeah, you have to get the fidelity to a certain level before you get in feedback that's actually meaningful. And I think that by bringing, like Emily said, take advantage of our operational data sources and the pedigree that comes with those. And yeah, you can start to get closer to the real world. - Thank you, Ben. One for Mark and then from Toma. What are the biggest obstacles to adopting more AI in the airport environment? Will AI be allowed to continue to learn after deploying or only at the development stage? - Yeah, that's true. - And you touched all this ever so slightly earlier, didn't you? - Yeah, for sure. So I think I can make another point. So for sure, we freeze the model when it's getting deployed into the operation. But we do, just like with any other software, when we would do a security patch, we do sort of periodically mixture that the operations are running at an acceptable level. And we then release model updates. We go through the normal ATM release procedure where we do regression tests, roll out tests, reliability tests. So it's just another software component in that sense. But we do continuously make sure that it's performing the level that is acceptable to the operation. And then we continuously improve as we do more research or as more data becomes available, we make sure we roll that out. So for example, one example I can draw on with together with NASA is we had a hold line surveillance system that's actually running in Heathrow right now in a pseudo operational sense. So that means it's not actually feeding to the life air traffic control tower cab, but it's running in parallel to the operation. And almost what Ben was describing sort of in the human and loop test where we run it in parallel and we showed it output and make sure that the human can validate that that is a correct decision. So we trained a model to understand if an aircraft is exiting the runway or continuing on the runway. And we then ran that on 50,000 movements at Heathrow to get the data necessary to then make sure that it can go into the operation. And I think that is really one of the big obstacles as well as that we still try to figure out how to have enough explainability, how to make sure that the regulator is comfortable to put this into operation. So we're still at that point where we need to have the human and loop and as Emily mentioned, it's a decision support tool. It's not autonomous and we're not aiming to replace the controller. It's really just allowing them to make decisions on a higher level of information that has been pre-processed by the AI. But even at that point, we're still working on the regulatory framework to make sure we can put this live into the Tower Caps. So we're all working on that together. And there's a lot of good working groups and advances being made. But I think that's still my main obstacle is that there's no golden rule how to safely ensure this to put it into the Tower Caps. Thanks, Mark. So plenty of challenges ahead. You mentioned the human in the loop there. And we've got a great question from Josh on that topic. I might pose it to Ben. So Josh has asked, are there certain aspects of their traffic control that require human capability that AI cannot address, for example, emotional intelligence during an incident or accident? I wonder if this is the kind of thing you might have discussed perhaps during Gertl Bluebird. Yes, definitely. So again, an insightful question really. So the truly human aspects of their traffic control that live in things like command of the RT, I think is probably the one that immediately springs to mind when you're making transmissions to aircraft, they need to know that you mean it. And it's amazing how much that sort of human side plays into that. That it's really a key part of what it means to do control. The one thing I would say is we are trying to look beyond just the core tasks that a controller does in Bluebird. So where there's potential for AI to provide support, we are looking for those use cases. So what that means is not just sticking to business as usual. So we've done some simulations already on avoiding action. So when aircraft get too close together, you've got to step in and be ready with clearances to get them apart again, the sharp end of safety critical operations. And over the next year, we're looking to bring in some extreme weather modeling as well. So starting to look at unusual circumstances and emergencies, things that happen which are a bit outside the box, which would require, I think our controllers would describe it as more creative thinking and being willing to do things you might not usually expect to do in the standard course of operations. And so we're trying to push our agents into those places to see how they react and to try and make it more robust, just so that we understand where the potential in some of this technology lies. But it's a great question. Yeah, and in that, then, are you seeing the potential for the AI to make decisions that the human wasn't expecting? I'm thinking of where AI's been used to play games, that some humans have been playing for hundreds of years. And then the AI comes along and makes a move that the human player wasn't expecting. Are we seeing that kind of thing, player? Yeah, so it's a good question. I'd say we're not there yet. You know, we've not revolutionized the world of ATC just yet. I think we're still learning. What I will say though is that the work that we're doing with controllers, which is really critical is finding down what the acceptable constraints are that we can operate within. Because if you were to get three controllers in a room and ask them how they solve a particular situation, the chances are that you get more than one solution, in fact, you might get more than three. There's always many ways to cut it, right? So it's about finding down what are the real constraints we're operating under? And then within that being creative can be a good thing. But you have to be certain, I mean, the obvious one is safety. We've really, really serious about safety. So until you've got that baseline performance on safety, then it doesn't really matter what you're doing elsewhere. So you set the constraints right. And then within that you can be creative, but there's certain things we can't do. Thanks Ben. I feel like it would be remiss not to mention last week's UK AI Safety Summit that was held at Blechley Park. It was all over the news and Elon Musk had a chat with Rishi Sunak. But whilst the summit was going on, the Blechley Declaration was signed and it said that AI presents enormous global opportunities, but it should be developed in a way that is human-centric, trustworthy and responsible. So I guess my question for you is how are we ensuring that the development of AI and aviation is human-centric, trustworthy and responsible who'd like to take of that? Marco, you're smiling, so I'm going to start with you. Sure. I can certainly talk a little bit about that. And I think that that goes back to the example that talked about our traffic light automation system, making sure that the human was in the loop as we were doing the reliability testing, making sure that they become comfortable with the decisions before they're fully autonomous and they have to deal with a whole new system. So I think that that's sort of the human-centric making sure that we run those shadow modes and we make sure that the human has a visibility of the decision under the hood. I think that does really the human-centric part. Ensuring safety is a really important one and that really comes only through to running it and we still in all of our systems have a big red button that if the controller feels that the decision being made that are in sort of the out of the side of the box that Ben talked about, there's a weather event, there's certain actors on the traffic, surface traffic that do things that they don't normally do. There's still a big red button that says, look, I'm taking control again, I as a supervisor say that there's a human in the loop that needs to step in and make sure that safety is insured, so that safety is always number one and when we make sure that there's that that ability for the human to intervene when they need to. Thank you. Link to that and something that I think you've all touched on is aviation regulation but there's a really good question that's come through from Peter. Peter's touched upon AI regulations saying that it's been in the news a lot recently, so how does all of that affect or relate to your work? Emily, would you like to say this one? Yeah, I suppose the short answer is as my team aren't providing safety critical AI tools now, it doesn't affect us yet, however we are keeping abreast of what is going on and what may develop to make sure that even if we're not ahead of it, we're ready for it as much as we can be. I think one of the interesting angle for that is Marco, especially talked about the regulator aside and you can imagine with the pace that AI is developing at the moment, the need for your regulators to understand the concepts of AI as well as that safety case in our case that we'll be putting forward. That's potentially going to be quite a crunch in terms of it isn't necessarily a widespread skill set or knowledge base now, so I think that'll be really interesting to see how our respective regulators not just in aviation but in all sorts of fields deal with that. Thanks Emily. Can I stick with you for a moment because during the safety summit when Elon met with Rishi, Elon said that AI will mean that people no longer need to work. So I'm curious, what's the direction that you're suggesting for AI that's in the much longer term? So I think that's probably quite a overoptimistic view, if I'm honest. Well, a seven-day weekend will be great. I really don't think that that's probably where we're going to be heading. I think if I think about it within a next context, we spoke earlier about, you know, be really clear what you're trying to use AI for. For us, we would be saying, what are those big challenges? The things that are really difficult, if not impossible for a human to do, let's use the AI technology in those spaces. So things that as I say are hard, they're difficult, or they're just mundane and not good use of your time. Now, I think that they're quite different scenarios, but that's where I would be suggesting we'd be best placed to use AI. So therefore, I guess I'm extrapolating from that and thinking, AI isn't going to do every job. Some of those, I don't think it physically will be stopping somebody from burgling your hair or rescuing you from a sinking ship, for example. And some of the things will be culturally unacceptable, and we actually wouldn't feel comfortable with AI during those things. So I think it's great that you throw that out there to prompt some thoughts. I guess I'm not in the same headspace at the moment. Thanks, Emily. And perhaps I guess a challenge that faces AI adoption really broadly has been raised in a question from Cheatham. So Cheatham has asked, "What are the information security challenges adopting AI in aviation industry and how do we mitigate them?" Marco, let's go with you to start. Sure. Sure, I can talk to that. I mean, cybersecurity and information security is becoming more and more of a challenge in all the ATM systems. So we used to be, say, everything's locked away. It's in the private network. Nobody can ever touch it unless you have security clearance. That's not really true anymore because we're connecting things to the internet. We need to make sure that everything is cyber security patched. So the overall ATM system thinking is changing, that the system isn't frozen for 10 years. We used to produce one radar system. You know, there's still radar systems from the 60s running. That isn't the case anymore. We need to continuously change, and that translates a little bit into the AI as well, where we need to make sure that the AI is connected to the systems in a cyber secure way. So when we connect it to an operational, so for example, an ATM system that had flight strips, we have to go through rigorous security risk assessment process like we would with any other system. One thing that is unique to AI is definitely the whole conversation about who owns the data that goes in. So the training data. So there's a whole debate about chat GPT and in other AI techniques that who really owns a training data who owns the model comes out. And that's probably going to become a challenge going forward when we talk about airports earning a lot of data and you know, a system provider is providing information back to them in AM models back to them. So that whole part of making sure that commercially sensitive data and operational data is safeguarded in a way that can't be extracted out of models and given to competitors or really on the internet. I think that is a bit unique to the AI setting when it comes to information security. Thanks Marco. And changing tech ever so slightly perhaps for our last question because I had a feeling that the chat would run away with as it's such an exciting topic. Euro control have said that artificial intelligence has the potential to tackle the challenge of making aviation more environmentally sustainable. How can it help? Because we talked a lot about efficiencies which I guess in turn feeds the sustainability angle but who can help me out with this one? Then you were looking like you were about to say something now. Let's go with you, Ben. Yeah, I think this is a really nice one. So I spoke a little earlier about how we have to try and represent all these objectives for air traffic control in order to effectively train automated approaches. This is one where I think that's actually has a lot of pedigree. So I'd other work that Emily's team, there's significant work done on understanding our environmental performance and what efficiency means fuel pernall this kind of thing. And if you have rigorous objectives defined that can describe what good performance looks like, then that's something you can then introduce into a model as one of the objectives. So I'd say representing safety is a real challenge but we've done a lot of the hard work already in terms of environmental performance. Thanks to the work that goes on for in the ops analysis side. So what you have the potential to do if you're building a model that's looking at maybe the bigger picture of air traffic control, say for the hold of the UK by introducing those kind of objectives and seeing how you might control if you were looking at the hold of the UK rather than just one specific sector, then you could maybe come up with tactical actions you can take on a local basis that improve the global performance. So this is the thing that's naturally quite hard to do. There's always the large scale traffic plan and then there's the precise tactical actions that are taken within a sector. Something to bridge those gaps, that's something that I would say isn't exactly there just yet but with the application of some of these techniques and applying the right objectives you could perhaps start to bridge that gap and say right well what actions could I take in a local sense that's going to actually improve that global environmental picture. And if I could just build on that Russell, if that's all I just mentioned. I guess if I'm thinking about it from a past present and future in the past our team had designed that three-dimensional inefficiency metric and which we're now regulated on but alongside things like CO2 and fuel burn we then moved to the kind of present where we mentioned the Michelangelo tool earlier where that is absolutely looking at that trade-off between environmental aspects and those other things like safety that we said are really important. If I look forward to the future I think AI can help us identify more opportunities. So looking as Ben said you do not just within the UK, not just from an A and S P point of view but that's exactly one of the things that AI is super capable of is finding things as a human. You wouldn't necessarily pick out and try to do that so that you can all reach that net zero target, for example, even more quickly than we first thought. Thank you. And thank you all. That is all that we've got time for today. It was a fascinating conversation. It was a really interesting picture that you've painted about AI and the future use of AI as well. So thank you, massive thanks. I hope you've all enjoyed this show. Thank you for posing some great questions. The show will be available on our YouTube channel and please do keep an eye out on our socials for the next episode of altitude. I think it could be supersonic, focused on Concord. So thank you and goodbye. This episode of altitude was walked to you by Nance. You can find more episodes of the show at Nance.airow/altitude.