Leveling Up in Data Science

Everything is chaos, man. Yeah, the news has been hopping. No, I mean it worked. I don't even know how many servers I have. Have you tried Fatem? It can describe and map your applications to your infrastructure. You can learn more at the break. Think about what you have and think about what you don't have. You understand the domain, but you don't know how to program. Go and learn this. So just find the missing piece or missing pieces. And don't try to follow the path that is like for everyone, which is advertised as being like data science in three months, something like that. Because most probably you already have some pieces. Welcome to day two DevOps where the dev whoops is in the details. I'm Ned Bellovance and I'm joined by my magnanimous co-host, Kyler Middleton, a Kyler. Hey Ned, guiding us through leveling up in data engineering and data science is our guest Daria Patraschka, a senior data scientist at SLB who made the leap from management and economics and linguistics to data science, a journey that fuels her passion for using data to solve real world problems. As an AWS community builder, Daria loves sharing knowledge through talks and workshops at events like AWS Community Days, AWS Summits, and Pycons, always curious and eager to grow. She's constantly learning through courses, hackathons, and hands-on experimentation. With no further ado, here's Daria. We just want to chat about data science, honestly, because I mean, personally, I don't want to speak for Kyler, but I find the area really, really interesting, but I don't know anything about it. And I get it confused with other terminology, data engineering, machine learning, big data, like have all these terms in my brain, but I don't know where one ends in the other one begins, so maybe we can start with that. Daria, how do you even define data science? Yeah, it's a good question to start, because, honestly, a lot of people confuse terms and a lot of companies will use job positions interchangeably. So once you will call your data analyst, another day they will call you data scientist, and if they need something more from you, they will call you data engineer. So all the same from one person. But there are some lines that could be drawn between those areas. I believe a data science is like an umbrella term. And under it, we can find, for example, data engineering, which is to engineer your data. So imagine that you have an axle file with some sales. You have a store and you have sales per day each day of the year. So if you want to have some kind of impression, what is going on, that would be like data science and big terminology. Yeah, but that would be kind of data analysis. So you need to analyze and understand what is going on. But of course, you can go through axle file, you can add up numbers, you can divide and do some manipulations. But if you want to process multiple files, if you want to convert them, upload somewhere like to the cloud, and then to do this like an automatic way, that would be data engineering. Okay. So to manipulate your data. And then if you want to predict, okay, this year, my sales were good, but what is the pattern on how it would look like possibly next year? This is also part of data science. You can run some statistical modeling, but you can also run some models like machine learning. You can use machine learning to predict what can happen in the future. So you see like all of this can be done by one person. But if the project is huge when the company is big, they tend to split it. One person takes care of data engineering, transferring, manipulating, splitting, cutting, and all this stuff. And as a person can do solely like data analysis, visualization, graphs, buildings. And third person can do like monitor it in production, deploy these models, hold them and just, you know, serve them in production. So the answer is like it depends. Right. Really like it's umbrella. It sounds a lot like DevOps, how it gets used in tech spaces, where it means so many things, and it means so much, it means nothing. Because everyone defines it a little differently. And what you were describing for, I believe it was the data engineer term, sounds a lot like what I've learned as ML ops or machine learning operations, where you're moving data and establishing pipelines. Are those synonyms to you or is there like a line where it starts to get to AI and starts to move towards machine learning? Or how do you define machine learning versus those other ones? Yeah, I think that those pipelines, they could be like data pipelines, they could be machine learning pipelines, they could be data slash machine learning pipelines. Because if you just want to let's say, transit and transform your data, extract, transform and load ETL, that would be like data engineering. But if you start to have some models like trained models, you want to deploy them and you want to monitor them, that is already machine learning engineer. So yeah, the line is tricky here, the border is tricky. I like that you started with the Excel file, because in my experience like so much of business just runs off of formulas and Excel files, because like it's just easy enough that most people can figure out how to build a complex formula. I have a very clear memory of when I think it was Excel 2007, moved to 64 of it, and all these like data analysts were super pumped because they could go over however many rows the limit was before, and build like basically a database in an Excel sheet. And I was like, but don't do that. At least stop just scaling way out. One of my early companies, we were growing really, really fast, like 10 times year over year for five years in a row. It was great. And we had just crested 50 million dollars a year of ARR, which was wonderful, we were all very excited. And so we brought in some consultants to tell us like, where are our weaknesses? Where are business process weaknesses? Because you break everything when you grow that fast. And they told us, by the way, did you know that there's one guy with the spreadsheet on his laptop that's not backed up anywhere? That's running the whole company. Like, if he leaves or if this spreads like his computer corrupts, your company will go under. So just like, by the way, keep an eye on that. Wow. Like legacy code. And it's honestly not all that surprising. I'm going to be honest. I'm curious because it is such a big domain that you described. You know, the data analysis, engineering, machine learning, and there's different roles inside of it. How did you first come to and discover data science? What was the thing that sort of hooked you in or sparked your interest? Yeah. It's also a good question because I didn't study data scientists at the university. Okay. I'm a career switcher. I came from management and economics background. For me, management. It's not a thing that was learning for five years, honestly. But yeah, after my graduation, I was in a sort of crisis. You know, I didn't know what to do. And at the same time, I started my family and I started having kids. So it was like a perfect time to pose and reflect what I'm going to do. And I clearly remember that the most attractive part in my studies was foreign languages in the university. Maybe it's a nerdy, but I was so passionate about like French grammar, about the regular verbs, something like that. I tried to find patterns because they were different. But at the same time, they were some patterns. Yeah. And I saw it like a system, a complicated one, but still. So I was thinking about, okay, I cannot pick up one language and to be interpretive just later. Because I was interested from the, you know, high level perspective system level perspective. And I decided what if I combine or search for something that combines language and mathematics or computing or something like that. Because I've heard about studies that try to analyze the language with computers. I tried to read some books, but they were so complicated because it was written in a, in a scientific language by people who already were studying it like for 10 years. And I came across natural language processing. I think it was 2018. Okay. So the data science was already here and it was already like becoming popular. And it was natural to do this link from natural language processing because it's kind of part of data scientists to data science. And yeah, that was basically it. I understood that I want to work with systems. I want to understand data. I want to actually train myself to not only like speak a bit of French, but also speak a bit of Python. And yeah, that's how it started. There's an interesting path. I've never heard that path before. It's like you took an interest in languages, but not just like learning a language, but the actual rules and systems behind that language. I think I'm just, I'm an introvert, and I cannot, you know, like communicate with people, especially like in a friend language. I cannot just attack them and say, oh, I want to speak with you. So I focused on the language itself. Right. I listened to a podcast called "Linthusiasm," which is a podcast about linguistics. And they do talk about like analysis of languages and developing your own. They call them conlangs, like Klingon as an example of like just a totally made up language that now has formal rules and structure and people who speak it. And it's wild that all that exists. So if listeners are interested, definitely check out the "Linthusiasm" podcast. It's a really good, double recommend. I remember reading, this is maybe 2015. I wasn't into this yet, but I read old books and articles and stuff. And the idea was that language is our demonstration of intelligence. It is the way that we are able to establish and build intelligence, because we can think of things that are abstract. And so the idea was that if we can teach a computer to do language, it will establish intelligence. Like it is a foothold to getting to intelligence. And we've sort of seen that with chat GBTB's like, so what a true one, sort of not true, because it's still very silly sometimes. But that's really interesting that you approached like data science via linguistics. I think that's fascinating as well. That's really cool. So with that background, what are some typical projects you would work on to apply this natural language processing and this love for language and data analysis? Yeah, so I think from one hand we can name some kind of typical project in data science. And from the other hand, all projects they will be widely different from each other because there is a bunch of tasks and there is actually a translation. So imagine a business person would come to you and say, I have a store, I need to figure out what is going on and what potential will be going on. So as a data scientist, you need to translate this business question into your data science question. You need to understand that, okay, I need to do some data analysis and I need to do some kind of regression to understand the numbers to predict the number. Or if I need to classify something, I need to do a classification task to predict whether the client would charm or would not charm. So there are some typical tasks and some people, they tend to solve those typical tasks. So we already can see that some people are working with, for example, audio, they work with voice manipulation, like transcribing the voice, voice production. And this is like a complete separate area, where you can just deep dive and stay in it forever. Yeah, some people work also like by domain, for example, this demise completely wild to me. Yeah, but this is a huge one was biology and molecular structures and they also trying to predict something, you know, and I know nothing like I know nothing about it, but I know people who work inside it and there is also like a place for the science. And if we talk about language are there is also like an I already mentioned natural language processing before charity era, of course, it wasn't like so fascinating like we have it now and it was mainly about understanding the sentiment, you know, is this review on the trip advice or positive or negative. Also, it was like summarization question generation, but it was a simple maybe not so fascinating for business users. When I worked at a consulting company, we had brought in another group that was doing social media sentiment analysis. And this was I want to say 2016 or so. So before the advent of chat GPT and like natural language processing was still in its infancy, but it was still growing and developing. And I remember there were a few data engineers and analysts there that were tuning the model to detect sentiment by pulling in everything from Twitter and Facebook and whatnot. And then they worked for these different brands who see what are people saying about brand X is a generally positive has there been a sudden spike in negative sentiment and oh, was that tied to a botched rollout or a bad product review or something like that. And I found the actual process of you know figuring out is this sentence bad is it being sarcastic because they would get a lot of like false positives where it would seem like it was a good review, but it was somebody being really sarcastic about how much they didn't like a thing. And it was hard to get the models to understand it because humans are weird. Yeah, yeah, I like this project, I will buy 10 next day, something like that. Next day in air quotes. So like when you're working on these projects to do the data transformation or analysis, what are some typical tools that you would use? What forms the foundation for data science from a tooling perspective? From a talent perspective, I would say it's mostly Python unless you're doing some for data analysis, then maybe you need to use some tools like sequel. Yeah, if you just need to query database, join and produce something like a table results for the output. Also big data tools comes in mind if it's not just a simple sequel database, but if you have like really really big portion of data. But in my work, I mostly work with Python because a part of fallen data from somewhere, I need to do some data manipulations. And a Python is a good way to do this, especially given inter consideration that I maybe never wrote something from scratch, like from complete scratch. Yeah, there are a lot of frameworks. I can just people style it and I can just use it for manipulation of tables, like data frames for computing some embeddings. And for like pretty everything, even for reading files, like reading PDF, I have a framework to do this. Python is a great tool. What else? I think visualization tools, some of folks are working with Power BI or Tableau or something like that, if they want to present and want to have the resultations look great. Yeah, of course you can use Python also to do the same kind of visualization, but it would be just more like technical. If you think about the not the actual work, but the scenery or landscape of the work, then we can think about infrastructure. Because if your project will be useful, they want to deploy it somewhere, if they want to live it somewhere, yeah. Here we will face the infrastructure of the company, some cloud resources, I don't know on brand resources, depending on the place you're working on. Not sure. I mean, I'm really into the MCU, and I love Carly Rae Jepsen. No, not fandom, Ned. Fatem. Their platform helps you discover and map your applications across any infrastructure, on prem, cloud or hybrid. They can map your environment in under 60 minutes, building a unified source of truth for your IT ops, development and cyber teams. Yeah, but I bet they need an agent deployed on every machine and domain admin privileges. I've got enough agents on my boxes. Thanks. Surprisingly, no, the discovery is network based and doesn't require an agent or credentials beyond flow records. Well, I guess then I'm giving my data to another SaaS. Not at all. Fatem runs on a self-hosted system and your data stays inside your organization. Even their AI offerings don't send your data off to Fatem servers. What kind of AI are we talking about? They've got Lighthouse AI for anomaly detection and Compass AI for plain English queries. You don't have to be an infrastructure expert to find out what the heck the Dev2 Thanos server is up to. It is always a good idea to keep an eye on Thanos. Sounds pretty cool. I'll call them. Maybe. And if this sounds interesting to you at DeerLister at home, you can check out Fatem on Fatem.com. That's F-A-D-D-O-M.com. You can start free for 14 days. No sales call needed. And they even have a free version. And I'm going to check out Fatem.com right now. I want to talk about cloud, but I'm curious in software engineering, which is a world I'm much more familiar with. Software engineers write code that works on my machine, you know, that classic refrain. And then they give it to sort of a DevOpsy person or software architect that gets it deployed to, you know, your hosting provider, your cloud, wherever to run its scale. Is that kind of dynamic exist here too, where data engineers are writing like a Python script. And they're like, well, this works on my machine. But then you need to productionalize it. You know, you need to add error handling and logging and stability checks and circuit breakers and et cetera. At what point do you shift to a different job role or handoff to like a DevOps person to implement? Or do you kind of own the whole vertical? Do you implement all the way to your hosting provider or cloud provider? I think it also depends on the company on the tradition you're working in and culture you're working in. Because if you're joining like a small startup with some small company, most probably you will be doing anything from end to end. And that would require from you to be like a more senior candidate who can handle some pipeline drops in production who can just be like jack of all trades, something like that. But if you're joining rather big company and you're joining like a junior data scientist and there are like more senior colleagues, they will help you, they will onboard you. And of course there will be some kind of maintenance team who can just do the DevOps stuff and you can live for years and not know what is going on there. So it depends. Of course, even if you use the second candidate, you need to understand how to make life of your DevOps friends easier, not to just, you know, let them debug your code at 2 a.m. Because you are sleeping and they have issues, you need to understand how the config file works, how to dockerize your application, yeah, how to basically work with kids, yeah, not to break anything. So the CICD pipeline will be successful and it will be like pass all the tests and get deployed. So you need to have this understanding, but not necessary, you will be establishing all of this stuff. That makes perfect sense. I was imagining it. You know, a wonderful file holiday future where I just don't have to ever write another GitHub action, but it sounds like you do still have to understand how to productionize your code and dockerize it and make it like prod ready. So it sounds a lot like software engineering to me, like the standards and patterns and stuff like that. And maybe you have a DevOps or hosting team that you can work with to help you push it live and that's very interesting. Yeah, and also one is the sense that it's kind of software engineering because you cannot just, you know, create a function from 400 lines and just push it. Yeah, you need to think about good quality. You need to think about best practices so your code can be maintainable after you and for your team members. So you cannot just, you know, let this work. So you need to have this software engineering culture, right? Yeah, I think I saw posts from you on LinkedIn where you mentioned having to debug somebody else's 400 line Python function. And all I read was 400 line single Python function and I couldn't fathom that. What was in 400 lines? That seems like it should be more than one function. You know, I think initially it was like a long script what was like someone was playing. Let's do this and this and this and this. So where very long script some pulling somewhere data, transforming it, juggling it, pulling more data, like joining, splitting, all of sort of things. And then suddenly they decided that they need to do like a function instead of like a whole script. But the easiest way to do a function is to add function in the beginning and to wrap everything into a single function. Why not? I think we can all think of many reasons. That's interesting that I would guess that some data science folks are not coming from programming background. So they might not be aware of general software development principles like don't repeat yourself and create small reusable functions that you can call multiple times. So I'm sure you encounter all kinds of interesting Python challenges. Yeah, I know Kyler and I both use AI assistance for writing our code. And that's been very useful, especially for me. Have you made use of AI in your daily workflow to improve things? Yeah, I think most of people use AI for code generation previously folks tend to go to start over flow and to search for some answers, but also it depends. If you encounter in time, you know, like classic problem or you cannot remember the exact syntax of joining some data frames, you don't remember, should you put like this kind of squared braces or curly braces or something like that. You will just Google it. It is simply yeah, if you start somewhere, you don't know how to you have a general idea of what your function should do, but you don't know the need to create of it. AI is good thing to see how it may look like, but this is more like a tool nowadays and it's good when you have already a background and you can understand what is going on and treat it like your more junior teammates so you can check and have your judgment. I know that we started to see new tools that are AI powered and database focused. Have you started to play with any of those tools or integrate with them. I think there's a Microsoft SQL copilot extension for Visual Studio code that I've seen that you literally like log into your copilot and then you say read the database and help me write SQL queries. And that just makes my heart start to pitter pattern. That's so scary, but I wonder have you played with them yet or what are your thoughts on that kind of database focused AI where it can maybe run commands directly. I haven't tried this copilot for database. I remember that I was inspecting this text to SQL problem because we had a similar task. We wanted to add the ability for business user to do some kind of natural language queries, but it was mostly like from my perspective, so it was like research, I need to do it in a proper way. So something like this copilot, but I haven't used it as a user yet. Yeah, I am famously terrible at writing SQL or whatever reason my brain doesn't work that way. And so the idea of letting copilot like build the query for me, as long as I can accurately describe what I'm looking for, I feel like it could probably do a pretty good job of translating my words into a viable query. But I would be very concerned if there was an insert statement someone there, I'm like, no, no, no, read all memories. As you're looking towards the future and working in your role, what are some big challenges that you deal with today? Like what's standing in your way of being more effective as a data scientist? That's a good question because not all tasks are technical ones. Yeah, there is a lot of let's say work efforts and work time that you just spend on the communication. And I don't think that we can possibly, I don't know, actually, if we can possibly make it better in the future, because it's always a communication bit bit like business and technical people. You know, it's like two different words I communicated and there are a lot of back and force going on. So yeah, it's nice that we can train people, a train business to understand how they I work. So they can grasp like some features of my work, but they can understand that maybe they will not apply it to me. So this is like one challenge. Another one, like a general data challenge I see now, it's not only about my work, it's about like the whole field. It's about data quality because people, researchers, companies, cloud providers, they are trying to provide the best, the best models, best agents. And from all of this race, we often forget about what was the fuel, what was the source of all these models. And this is like a beast. You don't know, you cannot dig into and explain why this model that you built, it produced such an awkward result because there are like a massive amount of data. It was trained on, it was spent here and you don't have enough observability into this. And also there comes a security people copy paste into AI systems, everything. They, you know, split scenes and the oops. So should I say that's like a people related problem. I don't know. I think that's a really apt insight. I've heard many people talk about software engineering in lots of different lights. We interviewed tons of people on this podcast. And the crux of it seems to be that software engineering in any of its forms is communication. You have to understand the business goals, the people goals, the limitations of your system. And then you have to write them down. And the way we write them down is code. And that's the specific very, you know, exact way of communicating our intent. That's what code is. So I think that's a really apt insight that, you know, you have to understand what people are looking for to make it happen. Totally. And Ned wrote one of these questions ahead of time. And I really like it. What do you see is the difference between a junior data scientist, someone who's kind of just getting started in this industry. And a senior data scientist that's rocking it, but knows how to get things done. What's the difference? What's the gap that folks should work on to level up? I had two different answers to this question when it was starting. And now when I was starting, I thought, okay, I will learn Python as a pro. I will learn like more tools. I will learn how to tackle big data and all of this. Okay, I also learn how to do things in production. So it was mostly about like technical answer. I will learn ABC and I will be just expert. But now I understand that mostly this is the way of landscape understanding. This is the way how you understand the business. You understand your teammates. You understand other people and the way you can lead. So it's like an understanding plus decision making. And for that, you need to be kind of architect. Yeah, you need to ask right people, right questions. You need to connect the dots. You need to connect people and to be more or less like T shape. Yeah, you need to understand how the deployment is done. Where are your data lies? You also need to understand what is the ultimate business goal. So for me, it's drastically different from what the junior specialist may be doing. Yeah, because they are coming and they are waiting for. Okay, I'm ready. Please give me the task. But in reality, you should be one who will understand what to do, how to do it and when to do it. And with what kind of resources. I think that was an exceptional answer. I love that answer. And that mirrors my personal experience so much moving from like a junior help desk person up to, you know, helping lead the help desk to being a cis admin. Every time you started with someone just giving you tasks to do and you just had to be really proficient at the tools and the technology. But slowly it became as you moved up like no one's going to tell you exactly what to do. Now you have to interpret objectives into tasks. Okay, cool. Now I'm at that level where it's objectives become tasks. And then you move up another level and it's like, no, I'm the one establishing objectives at this point. So I have to talk to someone who's planning a strategy and turn that strategy into objectives. And then I guess you move up another level in your CEO. I think that's how it goes, right? Technically, I am the CEO of my own company. So yes, I made it. No, you get to do all the jobs. It's lucky you. And it's terrible. If someone's just getting started out, would you still recommend taking the same approach as you like learning the tool sets first? Or would you recommend a slightly different path or a different starting point for them? Should they start with languages? Should they start by learning French? Where should we start? It was a long, long way for me. I would say like what I see people asking me same questions. I'm just starting. I know X by that. I know how to code in Python, where to go. Should they learn? EWS Azure should I learn? I don't know, Hadoop's Park. So the jungle technical turns. They think that one more tool, one more framework will do the magic and they will get hired. But in reality, especially for data sites, I would say hold on a second and think about your background. Because now we have people from the universities who just deliberately starting data, data science, mission learning, but many people come from different experiences. And think about what you already have. Maybe you have a background and marketing. That means that you can present things in a nice way. You can communicate to people. If you have a background in a mathematics and statistics, this is perfect. You can run the numbers. If you have a background like me, if you were interested in languages that is kind of systems. So think about the main. Or maybe you was like a good specialist in your retail industry. But you didn't know that you were so close to data science. So think about what you have and think about what you don't have. Go and learn this. So just find the missing piece or missing pieces and don't try to follow the path that is like for everyone, which is advertised as being like data science in three months, something like that. I love everything you just said because you really said if I could distill it down, I'll try to. It's not just about knowing the tools. It's about bringing your whole self into the job and leveraging the skills you already have. And then filling in those gaps. I love that. And it recognizes the whole person, not just the automaton that's doing data analysis. That's wonderful. Yeah. And people don't start you know, people coming from I haven't 10 plus years of experience somewhere, for example, in audit in a legal document processing. And it's already amazing. You already can do just learn. Now you can learn a genie how to programs something in Python. And you already can work with system that analyzing like legal documents or something like that. So you don't need to delete this from your CV. This is like a business domain. This is much needed. I love that. Darya, if folks are interested in hearing more from you, what are some good places for them to find you on the Internet. I think Lincoln is a good place. That's our preference. Yeah, I also have a small GitHub where I try to keep it updated. So I also try to some project there. They are just fun pet projects, but everyone is welcome to, you know, fork the repo and try to do things. Yeah, I think those are both good places to find me. Awesome. We will include links to those in the show notes. And thank you so much, Darya, for being a guest today on day two DevOps. Thanks for having me. It was a lot of fun. Thank you to our guests for appearing on day two DevOps and virtual high fives to you, dear listener, for tuning in. If you have suggestions for future shows, we would love to hear about them. You can hit either of us up on LinkedIn or send some feedback via packet pushers.net/followup. You can find me, Ned Bellovance at Ned in the Cloud.com and my amazing co-host Kyler Middleton blogging over on Let'sDueDebops.com. And we're both terminally active on LinkedIn. Stop by, say hi. If you want to talk to us live, our next live event is Friday, December 5th. Find the link on our LinkedIns. And if you like engineering oriented shows like this one, visit packetpushers.net/subscribe. All of our podcasts, newsletters and websites are there. It's all nerdy content designed for your professional career development. Until next time, just remember that doing DevOps is awesome and so are you. [BLANK_AUDIO]