From Politics to Machine Learning and AI Engineering

I've never been in a hiring panel where we talked about someone's certificates. On the other hand, I have been in hiring panels where we were like, okay, this guy built an app from scratch. It is production ready. It has users. He's shown all of these different skills. I asked all these questions and he really knows this stuff. That can make a big difference. Welcome to Day 2 DevOps, where the dev oops is in the details. I'm Kyler Middleton and I'm joined by my venerable host, Ned Belvance. Hey Kyler, today we're discussing AI engineering and how a background in politics can help make engineering work. Guiding us through AI engineering is our guest, Marina Veece. Marina is a Senior Applied Scientist at Witch. Would no further ado, here's Marina. Marina, welcome to Day 2 DevOps. Thank you so much for joining us today. No, thanks. Happy to be here. We're here to talk about AI engineering. Before we get into that, your title says that you are the Senior Applied Scientist at Witch. Does that mean? That sounds super fun. Not the Senior, there are other ones, but one of the Senior. Yeah, so I am in a lucky role where I get to do a lot of different things for a lot of different teams at Witch, so my kind of project areas span all the way from machine learning infrastructure to a lot of stuff, mostly on content understanding with Gen AI, then I've also done some work on fraud, finance, and the recommendation system. So I've gotten to do a lot of different stuff. So it all sounds like it's around that same machine learning data analysis kind of field. Yes, for sure. We ask you to come on because you've been talking about AI engineering. So how would you define AI engineering in comparison to some other roles out there? Yeah, that's a really good question, and I don't know that there's like an industry-accepted answer to that yet. The definition that I use comes from Chip Huyen. She wrote the book AI Engineering, so I would assume she knows what she's talking about. And her definition is basically that where a machine learning engineer or a data scientist doing machine learning would use data to train a model from scratch, and AI engineers primarily using pre-trained models, usually just via an API, maybe locally, and then building products on top of that. So that's the main distinction is whether you're training from scratch or not. Okay. And so have you started training models from scratch or are you starting with pre-configured models? Both. Yeah. Early in my career, I was all training from scratch. So I started in kind of more of the statistics and data science world. So it was all custom models, and then now it really just depends on the use case. So if we're doing something in recommendation systems or fraud detection, financial forecast, sometimes have ways to use pre-trained models, but by and large, we're still training from scratch. And then when we get into more of kind of like the content understanding stuff, so trying to figure out like, what's going on in a stream or what topics are discussed, what kind of vibe of the stream is whatever the case may be, for whatever product we're building. Those are perfect use cases for pre-trained models, and those models might just be taking in text from chat or the stream or something like that or it might even be taking in the audio or the video as well. Is that, I know that's an example, is that literally what your models are doing sometimes where you're trying to feel the vibe of the stream, and it's less about like these specific keywords were said and more like, this is a relaxed stream to classify them. Is that something that you'll do? Because that sounds very challenging compared with like, I don't know, forecasting sounds easier because it's numbers and you just, you know, move forward in time. Yeah, I wouldn't say forecasting is easier, but they have a different set of challenges. Yeah, so there are cases where we're looking at no more subjective things like mood or sense of humor or whatever the case may be, but, you know, it really depends on the use case. Sometimes we're also using pre-trained models for more objective things. I guess you could say like, did they talk about this or did they do the ad read in line with that? That kind of thing. Okay. Could one of your models tell us if we are funny today because that would, that I'm, okay, very interested in that subscribe. Yeah, so we could, of course, the issue is how you define funny, right? That's always the issue with actually basically every machine learning and AI engineering problem is your problem definition and how you're going to evaluate that. So like, what is your criteria for something being funny or not, especially when it's so subjective and there's so many different types of humor, for example, you know, is sarcasm funny, depends on who you ask, you know, is a, like, a dad joke funny, depends on who you ask. So, yeah, I really would depend on your criteria, but you could technically build a model to identify a type of humor that you're looking for, super cool. So what you start with is not, I'm going to go find a model. Starting with what am I actually trying to find in the content or the sample and defining that really well and then seeing if there's a model that already has that definition in it. Kind of. Sometimes. So it would be less about a model that has the definition in it, but more a model that could be told how to find what you want. So you would want a model that has capabilities you're looking for. So if you want text only or if you want images or audio, that kind of thing, that would be a first pass, right? And then different models are good at different things. So some models are better at coding, for example, some are better at creative writing. Sometimes you just like the way a model talks, you know, so there's a lot of different, different kind of personality characteristics in addition to their performance on benchmarks and that kind of thing. And then once you had kind of your pool of models narrowed down into which ones would be reasonable candidates. At that point, you would test different ones using different prompts and it's really just about either creating a really good prompt that would explain what you want the model to do or potentially also fine tuning a model so that it more naturally does what you want it to do without having this like really long set of instructions that's guiding it each time. Okay, so fine tuning is almost like embedding those instructions into the model as opposed to having to give it every time. Yeah, kind of fine tuning basically means you take a pre-trained model and then you train it a little bit more. So you just train it on your set of examples. So the most common use cases for that would be like if you wanted a particular output format, like you always want a special JSON format every single time. That's really annoying to do with prompt engineering, but you can teach a model to do that pretty quick. It doesn't take that many examples to teach something like that. Or if you want it to have a particular voice or domain knowledge on medical stuff or legal financial news, whatever your specific domain as you could also continue to teach it a little bit more about your use case specifically. Okay, I got you. I'm curious, how did you get into this world of AI engineering and data analysis? Did you go to school for like statistics and math? Was that your focus or did you just kind of fall into this somehow? Yeah, a little bit of both. So my education background is unconventional for this field. I studied political science and undergrad, then spent many years working in nonprofits and for a small business, a small jewelry business. Didn't know what I wanted to do and figured I needed to, I needed to make some move. So I went back to grad school in the same area that I had done my undergrad, so public policy. And it was kind of a happy accident that that public policy program happened to have a really strong social data science component. So I did take a lot of courses in statistics, causal inference, coding in R and Python. And then they actually, as I was a part of the program, continued to add more machine learning courses. So that was quite lucky. I was able to take courses in machine learning and deep learning and NLP as well. So a lot of the core data science fundamentals actually ended up being taught in my master's program, except with a social science potentially oftentimes public policy specific, but sometimes general social science kind of tilt, a little bit of a focus area. And after graduating, I worked for a statistical consulting firm in Berlin, which led me into data science and then data science turned into machine learning engineering, which turned into what I'm doing now, which is kind of a blend. That is fascinating. In my head, just my, my gut says, you know, machine learning and statistics, like that's hardcore math and computers and like, that's what you do all days. You just think about the circuitry and, and I find it fascinating. How much it's not a computer science domain world? My partner's in technical communications, getting a PhD and one day she was like, I, I need to import pandas into my R workspace so that I can do some inferences. And I'm like, I thought you would just be reading books all day, but it's much more statistics and computer science tools type focused. And it's, it's very much a cross functional domain. Yep. Do you feel like your, your wide background is, is helping you succeed here in a way that like if you were just, just to compete, I say, just a computer engineer with quotes helps you more than if you were just in that domain? I think that there are advantages to both. So it's, in data science in particular, data science tends to feed into machine learning and AI engineering. It's a very diverse set of backgrounds. So you do get people coming from CS backgrounds, you get economists to really calm in, sometimes people coming from the business world because they go through like many steps. So they'll start as like a business intelligence engineer or data analyst and then get into data science. And then, you know, so it's a very winding path. So there's a lot of different backgrounds. And I think everybody has their own, their own things that they have an advantage and disadvantage. So it took me a very long time relative to someone who had a computer science background to be a decent like software engineer because that is part of it. So I'm not, I'm certainly not a software engineer, but being able to deploy production systems and write good code and feel good about following software engineering principles in my work, that took me longer. And it took a lot of self study outside of work. I've been studying now in the weekends for eight years, you know, so it took a really long time. But the advantage that I feel like I have having the social science background really comes when you're in the problem framing part of a project because the majority of AI engineering or machine learning engineering projects are measuring human behavior in some way, not always, right? Sometimes you have sensor data from machines. It's like many layers away from human behavior. But, you know, recommendation systems or fraud detection or even financial forecasting. There's always a human element to it. And there's often more of a social science tilt that you could take. So that can be quite helpful for coming up with metrics or interesting data sources or novel experiment ideas can be another advantage. So yeah, a little bit of both. So I really like what you mentioned about the fact that you've been studying after work for eight at eight years. Like this was not something that you went to a 12 week boot camp and you're like, I am an AI engineer now. Not even close. Yeah. I mean, to this day, I'm still studying all the time. I mean, I think to an extent that's kind of expected in technical fields, just as the fields are changing so much all the time, there's always going to be something new to learn. But there was also, you know, many years where I had gaps to fill in my background and I didn't really have much of a choice. If I wanted to be successful, I was going to have to do the extra work to get there. Right. We think of AI as such a recent innovation just because of all the glitz and glamour of chat GPT when it launched itself upon the world. But I think the other point that you made about studying for the last eight years is, this is not net new. What sort of problems were you working on eight years ago before the big rise of the LLM? Yeah. So eight years ago, I was learning a Hello World in Python, but within the last five years since I've been working in industry, I would say the biggest change is what we are able to do with language in particular and video, I guess, as well. So I guess multimodal. But a lot of NLP techniques that we were doing before were not as good as they are now. So for example, in my first big tech jobs was at Coursera and we were trying to work on a project to figure out what skills were taught in a course, which seems like it would be pretty easy. You would just look at the course description, you would see it's a machine learning course, you'd say they're teaching machine learning, right? But there are also many other subskills that are taught. So this machine learning course may also teach some basic statistics, it may teach TensorFlow or PyTorch, it may teach specific machine learning algorithms that you want to identify. So the project was basically to take all of the techs that we could find from the course like assignments, quizzes, that kind of thing, and build a classification model. And so at the time that was a traditional neural network using labels that we were able to infer. And because the labels weren't as clean as we would like, the model never performed as well as we wanted it to, and evaluation was also very difficult because weirdly there's subjectivity everywhere in the world. So the performance was never as good as I wanted it to be. If I was approaching that now, it would be so easy because I would just put a bunch of text into an LLM with a nice prompt, and there you go. I would just have probably an extremely good result with very little effort. So that would be, I would think, the biggest change is mostly on language and video image stuff. Right. Yeah. That's stepping into my domain because I write courses, and I can tell you for a fact, I do my best to include everything that's going to be part of the course in the outline when I submit it, but you don't really know exactly what the course is going to look like until you've written it. And even then, you have to record it because I do video courses. And so you still don't know exactly what it's going to look like until then. And then someone else has to go and transcribe it, and I guess catalog it. And that was much more difficult when I started doing this eight years ago. Yeah. The transcriptions, like if you fed it into something that was meant to create transcriptions on the fly, there was a lot of hand editing afterwards. And now that alone has gotten so much easier. And the ability to summarize it, it's huge, but also a little scary. Like I don't want people consuming my courses that way. Yeah. I get that. I have seen some really, and this is, I guess, dovetailing a little bit, but a lot of conferences that we speak at or attend or want to attend or post it on YouTube and YouTube generates a transcript that's really good for natural language processing, like it's pretty solid. And there are some tools that have come about in the past six or nine months because LOMs are getting so good to summarize key points out of all those talks and create these sort of superset lessons of like at this conference, the people there covered these things. And if you want to zoom in, click this keyword and I'll show you that, and it's like, that's incredible. There's so much that is going on behind the scenes to generate those cool little snippets for me to save a little bit of time from watching some videos. And I don't have a question. I just think that's amazing. Do you, does it still seem magical to you? You've looked behind the curtain, right? You're building the magic. Does it still seem magical and amazing when, you know, chat you be dispatched out something that like, how the heck? It's definitely impressive. I think if you dig into the details on how the models work, it is demystified a little bit. It really, not to oversimplify, but essentially it's just auto complete. And so once you kind of have that context for like, okay, it's just generating a probability distribution overall, the potential next words and picking the most likely next word, it becomes a little less magical, but it's still very impressive, very useful. Hmm. Totally. So speaking of building, and this is, you know, something you do in your day job, what is that involved? I think you've mentioned Python, and I'm curious what other tools you're working with on daily basis to build out these models and potentially automation pipelines and whatnot. Yeah, sure. So I will stick with kind of more AI engineering pipelines for this example, but it's pretty similar for machine learning. There's just a couple of different components that we do. So yes, everything is Python. If we need to bring something in from a database, of course, there's going to be some SQL that's obviously more common for machine learning where we're training from scratch, but, you know, there could be occasions where you need to pull something in in an AI pipeline as well. Then I typically am using Airflow or an equivalent. So that's still quite common, I find. And so if we're doing a batch pipeline, a very standard workflow would be like, pull any data you need from the database, send off a data preparation job. So like one of your airflow tasks would be potentially starting a SageMaker processing operator or ShazeMaker processing job. And doing any data prep that you need and then triggering a batch LLM call with whatever prompt that you're using at the time. And then those predictions are saved usually back to a database. So it could be as straightforward as that, just read data, prepare it, send it to the LLM and then send the final version back to the consumers, downstream. The vast majority of pipelines are batch still, though there are, of course, many, many, many real time use cases. And in that case, usually I am typically only responsible for the prompt engineering and evaluation and any model selection, more of the science side. And I can't speak as much to how people are actually setting up the front end, for example. Okay. I got you. I'm not familiar with Airflow. Is that software project? Yeah. Interesting. Yeah. Kyler and I are coming from like a DevOps background, but we don't touch any of this. So this is all met, like met new. Okay. Cool. Airflow is just a data pipelining tool. So it's an orchestrator and scheduler. So it's, I think actually a little outdated now, that's why I was like a little hesitant when I said I still use Airflow. So there's newer ones like DAG Stern prefect, but a lot of the kind of bigger companies, you know, takes a long time to change. So yeah, it basically is just setting up a DAG where you have different task definitions and then you can see it in a visual UI and it runs on a schedule. So it's fairly straightforward, conceptually. Does it have specialty hooks that are aimed towards like data science and machine learning? And that's why it's used for that purpose? I would say it's actually a data engineering tool primarily. Okay. Yeah. And then you can create specialty, they call them operators for your particular use cases. So we would have like a special operator that's going to trigger a training job on AWS SageMaker or a special operator that would deploy a SageMaker endpoint on AWS. So you can create your own. And then there are some built in as well in Airflow. Okay. Gotcha. It's probably the closest thing that I work on on a regular basis is like GitHub actions. Mm-hmm. Yeah. Yeah. There's some similarities. Yeah. And I've started to mess around with maybe Jupyter notebooks. Is that maybe a step closer where it's, you know, it's like a scripted, I don't know, automation type call. And it's third party. It's not a service or no. It is both. So it can be not actually 100% sure how everybody uses this, but it's open source and then available as a service on AWS. So you can, you kind of can pick whichever works best reuse case. I don't think it would be closer to Jupyter. Jupyter would be something we would more use when we're very early on prototyping. So if you really want to be able to like actually look at the columns of your data and train a little model and take a look at the inputs and outputs, make some plots, that would be the use case that we would usually use Jupyter notebooks for. This is more you use a Python script to define the tasks that you want in order and you can define dependencies. And then when you actually have it deployed, you have a UI where you can see the state, the status of each task. And so you can go through and like restart tasks or see the logs, see why they failed. So it's more about the pipeline and less about the individual model components. Interesting. Is this for sort of early days, like you're like, I've got an idea, I'm going to do an experiment or is this, you know, you're looking at prod and you're going to make some tweaks to what is going to happen in production? Both. So usually if I'm working on a new project, I would create a new airflow pipeline very quickly. So there might be some little experiments that I would do offline in a notebook first, just to kind of validate that my model works, but I like to be as close to production as soon as possible with my work. So I try not to have too much, I have to copy paste or, you know, move from one environment to the other. So typically it would start developing in the, you know, the beta environment right away. So it would make my airflow pipeline, make sure everything is running properly, do any experiments I need to and then once that's then, when it's merged, it would get, you know, merged to prod. Cool. Okay. I've got some grounding for it. I feel like an intern on my very first day. This is so exciting. Is airflow something you would use to train a new model or to define to an existing models or both? Um, actually, neither um, you could trigger a training job. So if you were doing machine learning and you're training a new model from scratch, you could trigger a training job within airflow, but actual training would happen usually at the cloud service. So you would trigger the SageMaker training job and that would have all of your input data in AWS with, you know, um, some kind of definition of where to find that, what kind of model you want to use and then the training job would do the actual model training on different compute, um, that you could also specify depending if you need GPUs, that kind of thing. You could also probably trigger a fine tuning job, um, though I would imagine that that would be fine tuning is unlikely in my opinion to be something you would do on a schedule. And the point of airflow is to have a pipeline that runs consistently on a schedule and has some way to, um, fail nicely if one of the tasks fails. So it's, it's something you would have like running in the background for repeatable tasks, fine tuning is something you'd probably do once or twice, um, you know, and then you would use that model potentially in an airflow pipeline. So maybe you use your fine tune, um, model for whatever your downstream use cases. If that's bad, you might use airflow, if it's real time, you might just set up an API. Right. And imagine you're working with really large data sets. So these batch jobs have to run for hours, possibly even days, so having that graceful failure mode got to be really important. Yes. Yeah. For sure. Um, so yeah, it depends. Sometimes things are, you know, smaller, there are definitely like small use cases that are still very important. So not everything is on massive scale, like forecasting, for example, is not that huge of a problem. Well, it's very difficult. But it's not big in terms of data scale, um, but yeah, if you were going to do something like go through every Twitch stream and try to find some keyword or something, yeah, that would take quite a while. I love that we learned about a net new tool that I've never heard of. I was beginning to think that there just wasn't a lot left in computers that I am very wrong today. Oh my goodness. I am very wrong. This is exciting. And it's part of the Apache product or group of projects, which is so like I have, uh, you could throw out six Apache project names, then make one up and I would have no idea. There's so many. There are so many. So if someone's interest is peaked at this point and they were thinking about maybe getting into AI engineering, say they're coming from, you know, like a system administrator type background, where, what skills would you recommend them starting with? Yeah. So the lucky thing about AI engineering compared to machine learning is it's quite practical. So you can get started building stuff really early. The first step that I would recommend is actually just to read the book, AI engineering by Chip William, because it's, it's a great book and it gives you a really solid overview of the whole field. So you can have some awareness of the different kinds of problems that you could solve. And I think that's really important, because there's just so many cool things that you could do, but you just need to know that you could do them. Yeah. So I would start there and then at that point, try to find something annoying in your daily life that you would like to build an AI tool for. Nope. So that could be like a, like a Chrome extension that summarizes the website you're on or said, something that organizes your tabs or organizes your emails, makes your schedules, writes, email drafts for you, whatever. Just anything in your daily life that you wish you didn't have to think about. And then reverse engineer how to actually do that, usually there's frameworks to help. So find it like, I'm a bit of a tech news junkie. I have entirely too many RSS feeds that I'm subscribed to. And oftentimes, four or five of the same feeds will have the same story they're covering. And so it would actually be really cool. And maybe this is a good project for me to write something that could ingest these RSS feeds and summarize common stories across and then point out key differences in coverage for me. That's perfect. Like a ground news, but for tech news specifically, and I have to build it for myself. Oh man, now I have a project to do over the holidays, but probably well beyond because this sounds like it would be more than just a week's worth of work. It depends how good you want it to be. So the reality with all the AI coding assistance is you could probably vibe code that in like an hour. But if you want it to be high quality, so if it's a thing you're using for yourself, I mean, go ahead, vibe code at school. If you want it to be something that's more of a learning practice, of course, can take longer. Or if you want it to be something that is robust enough for users, then there's a whole new world that you have to think about as well. Getting something functional in the AI engineering space is quite easy. But then the most difficult part is evaluation. And then if you're actually working with users, security and compliance, these are really big things. I would say those are kind of the big areas that newcomers don't think about, which makes sense. So if you're just working on like a portfolio project to learn, you're not going to be thinking about compliance, right? But evaluation is a really, really big one because if you're, let's say you're building some agent system, you've built a multi agent system that reads your emails and then it's going to draft emails for you and potentially it's going to operate your calendar and take all of these autonomous actions for you. You need a way to evaluate every step of the way that it's doing exactly what you want. And that can be really tricky when you're going to do subjective things like drafting emails. So how do you evaluate that? You need some rubric, you need rules, you might need another LLM to come in and act as a judge, you need some quality gates. There's a whole world of steps to make sure that your model or your project is performing the way that you want it to and not going off the rails. It's so interesting that the evaluation portion of things, because it, you could get it to do something in the shape of what you want. But to get it to actually do the thing you want requires a bit more nuance and digging into it. Yeah, absolutely. I had a recent experience with that. I've been trying to code up something for a product that I work on called Terraform. And I thought that the agent was writing the code to do one thing and then when I actually got to the, all right, show me the product. I'm going to test it. I realized that it had totally misunderstood so much of what I'd asked for. It happens, yeah. And it's like it had written unit tests and integration tests and then test and they were all passing and they were all wrong. Oh, man, and part of it's on me. I just did not express myself in a way that was clear enough that it wouldn't go off in the wrong direction and I didn't catch it early enough. So maybe I needed to work on that evaluation layer and have a tighter feedback loop before I got too far. You mentioned that for certifications, that's something you don't, you don't recommend for beginning folks getting into this field, which is the opposite of what I normally recommend. I normally say like when you're, you know, getting into networking, get a CCNA when you're getting into security, get a security plus and it just sort of proves your interest in the field. But those are kind of one silo, I suppose, of skillset. And AI engineering, it feels like it's a collection, it's very broad. Is that what your experience is in the field that AI engineering requires some predicate like statistics and cloud and compute and programming and it feels like it's a whole bunch of skill sets together or is it just, what do you think? Yeah, so I think it depends, it depends on a lot of different things here. The kind of point I was making on certificates being a distraction early in your career comes from my experience career coaching where a lot of people will stop themselves from applying to jobs or stop themselves from doing projects, not feel like they're ready for internships, whatever the case may be, they're not ready to take these more proactive, self-directed steps and so they can use certificates as a way to procrastinate. So they're like, "Oh, I'll do this, I'll study for this certain and then I'll build something, I'll study for this and then I'll, so I don't think that there's anything wrong with getting these certificates and they can certainly be an advantage, but I would be careful of prioritizing them at the expense of more of the self-directed work. This of course assumes that you have some level of credential already, so a lot of times I'm working with people who have like a CS bachelor's and they're like, "Oh, I'll just go get this certain, I'll go get this certain and then I'm like, no, no, no, at this point I feel like they would be more benefited by doing self-directed projects and spending a lot of time networking, honestly. So it's mostly about leverage and just what's going to be the thing that gets you actually farther in your career, because most of the high-end roles in this field don't actually care about whether you have certificates at all. So yeah, it's just a bit sad. But on the other hand, I have been in hiring panels where we were like, okay, this guy built an app from scratch, it is production ready, it has users, he's like shown all of these different skills. So that's kind of where I'm coming from with that. Yeah. Yeah. Yeah. When it comes to certifications, I think they can be a really good way to learn a very limited set of information, but they are going to be hyper focused on that smaller set of information and don't always give a more general background because the point of the certification is to teach you about this specific thing, especially if it's a vendor certification. They're like, we're going to teach you exactly how to do it with our product. Now you know that product really well, but yeah, if it doesn't teach you the larger background, as soon as you move away from that product, if you go to apply for a job that doesn't use that product, it's kind of like, I don't know how to do this outside of that bound. What about training? I assume there's a lot of training out there that has been developed recently. What if I'm thinking about taking some online training or maybe attending a boot camp? What should I look for and what should I look out for? Yeah. So in terms of, it also, I guess, depends a bit on your goals. If you are early in your career and you're looking to get your first job or if you're looking to change jobs and you really need some kind of career assistance, then of course, I would look for boot camps that have a job placement guarantee or have some kind of demonstrated career help. That's kind of the first thing that I look at. Then it's also really important to see that the curriculum aligns with what you're actually interested in because there's a lot of boot camps or actually programs in general right now that call themselves AI, but then they're actually one of two things. So they'll either be just teaching you how to prompt engineer on chat to BT which is not really what anybody is looking for or they will be machine learning. So they'll have several weeks on data cleaning and EDA and model selection and machine learning theory and math. Sometimes they'll even be that's not really important for you as an AI engineer. So if you're interested in the AI world, I would focus on programs that talk about prompt engineering, fine tuning, rag, agents, and maybe things around like model selection and model evaluation. Those would be kind of like the big buckets that I would be looking for so that you could at the end have the skills that you need to be building these AI apps. So there's a lot of different programs right now. There's a ton. There are many, many programs coming from universities and I have found just looking at their curriculum that I don't think most of them are worth the price. The vast majority of them are very expensive and then if you look at the curriculum, it's much more theoretical. So they rarely actually are teaching the tools and frameworks that we're using on the job. They might get again more into math or theory than it's actually useful. And they're also often taught by third parties. So they're not even taught by like UC Berkeley faculty, for example. So now you've spent $6,000 on a program that doesn't actually get you as far as you might think because recruiters don't really value those programs the same way they would if you know you got a master's degree from Berkeley, of course. So at that point, I think it's better to focus on smaller industry-led boot camps or courses that are very specific with the skills and tools that they're teaching you and then save your money and focus more on building your own projects with whatever skills you develop in the course. That's so much good advice packed into a very short answer. It's interesting with the university courses that you were pointing out because we had someone on, I think last year, who had just recently graduated from computer science program and she spent the first year working in industry learning all the tools and concepts that they just didn't even bother with in university. And she was like, "I had to learn what a pipeline was and how get worked." Oh, yeah. These are like critical things that you would just be expected to know. And I was amazed that they didn't teach any of that in the computer science program it was in. I guess, you know, maybe their curriculum just doesn't get updated as often as it should. Yeah, and also learn new things. There could be an element of don't want to and then there's also just an element of it moves really fast. If you're not completely up to date on all the AI engineering stuff, I can understand that because I don't know how you would know most of this if you weren't working in industry. How would you know what of the hundreds and hundreds of new developments that are coming every month are actually relevant for your students? So I have some understanding of it and then also some, I'm like, "Yeah, you can probably just ask some people who work in industry." So if I did want to connect with some people working in the industry, are there any online resources or communities that you're either part of or you're aware of that you think someone getting started would find value in? That's a good question. So in terms of places to actually, essentially, network, I unfortunately don't have any good resources on that. I'm actually hoping to create something in the next six months or so. For more like learners, people are trying to break into AIML. That's not to say it doesn't exist, I just don't know about it. But I would definitely recommend following some newsletters. So the batch and TLDR AI are my favorite ones. Okay. Especially the batch because it has kind of more in-depth analyses of things that are going on and it can give a little more context for what's actually happening. It's not just going to give you a big list of tools. So I would definitely recommend those resources. I wish I had a better answer for communities. No, that's fantastic. We'll include links to all of that in the show notes. And you know what? You may have created a few resources people might want to watch. If folks are interested, where can they find you on the World Wide Web? Yeah, for sure. So yes, like you alluded to, I make YouTube videos on AIML data science stuff. So it is everything ranging from technical breakdowns. Like I did a video recently on how to pick the right LLM for your use case and a pretty long course on AI agents. And then also some more practical things about the job market. So how to make a good resume, how to network, interview prep, all that kind of stuff. So that's a marine of East AI on YouTube. And I also have a newsletter that I send out every week at GratitudeDriven.com, where I have a mix of technical topics and also usually some weekly mindset and motivation stuff. All right. We'll include links to that as well. Marina, thank you so much for joining us today on Day 2 DevOps. Yeah. Thanks for having me. Thank you so much to Marina for appearing on Day 2 DevOps and virtual high fives to you. Dear listeners, for tuning in, thank you. If you have suggestions for future shows and future cool peoples for us to talk to, we would love to hear them. Hit either of us up on LinkedIn or send us some feedback via packetpushers.net/followup. If you've enjoyed this episode and want to keep the conversation going, we've got two hot tips for you. For starters, packet pushers has a Slack group open to everyone. It's a marketing free zone filled with nerds like us who love to chat, compare notes, and solve problems together. Even better, Kyler and I host a monthly live stream where we talk about previous episodes, take questions from the audience, and just generally commiserate about technology. Want to know more about my deep feelings on mechanical keyboards? Hmm? Subscribe to packet pushers on YouTube and you'll get notified when the next one is coming up. Until next time, just remember that doing DevOps is awesome, and so are you.