Klaviyo Data Science Podcast EP 39 | Are you going to science fair?

(upbeat music) Hello and welcome back to the Quavio Data Science Podcast. If this is your first time joining us, I hope you enjoy your stay. I'm your host, Michael. And I'm your host, Zach. A quick exercise for everyone listening to this podcast. I'd like you to think back the last time you had to explain something you were working on to someone who isn't familiar with your work. Maybe a friend, maybe a family member, maybe a stranger, you met at a professional event. Whoever it is, I'd like you to think back on that conversation. Feel free to pause the episode if you need. I'd like you to really hold the conversation in your mind's eye for a second. Okay, now that we're all back, I have a couple questions for you. How well do you think that person followed you? How easy was it to give that advice? How easy was it to give that explanation? If you're anything like me, I'm guessing the answers are something in the neighborhood of not quite as well as I'd hoped and not quite as easy as I would have liked. That's far from a unique feeling. Explaining data science, explaining science, honestly explaining any technical work at all is a distant skill all by itself. And it's one that people don't always get opportunities to practice. This episode we're going to talk about an event that Clavio R&D put together that gave people exactly that opportunity, the Clavio R&D science fair, and what we learned from it. To help us with that discussion, we brought on two people who were heavily involved in the planning of the event alongside myself and two who presented at it. Let's go ahead and start with introductions. For each of you, could you give a quick introduction of yourself, the team that you're on and what you do at Clavio? Let's go ahead and start with Aju. - Hey everyone, my name is Aju. I'm a software engineer too on the reporting team. - I guess actually for the introductions, we should probably also explain what you did with this event. So if you don't mind adding that in as well, Aju. - Yeah, hey everyone, my name is Aju. I'm a software engineer too on the reporting team and I was one of the organizers for this event. - Fantastic, Jessica, you're up next. - Hi, I'm Jess Brettz. I am Senior Executive Business Partner here at Clavio. I support the R&D functions and for this event, I took it from the idea of wanting to do it from the R&D leads to making it happen with the right people. - Excellent, next up Maya. - Hey all, I'm Maya Negrin. I'm a senior software engineer on the audience growth team at Clavio and I presented a recent audience growth project called Formal Arts, kind of the past, present and future of it at the science fair. - Fantastic and Jason. - Hi, I'm Jason Grant, design manager leading the design systems team. At the event, we showed two boards. One was for the component library and one was for the design system and both of them kind of captured what we have accomplished in the last 18 months at Clavio. - Fantastic, I believe all of you, this is your first time on the podcast. - Yeah, welcome, it's fantastic to have you love hearing from all of these new voices. Thank you for being on the show today. I guess I should probably give a quick note. So I'm going to be acting a little bit more as a guest this episode than a host. I was also heavily involved in the planning of this event. So I'm going to hand things over to Zach after one last note. Our fearless co-host, Zach, was at this event. He went around on the day of the event with a microphone top to some participants, some organizers, some audience members. You're going to hear some of those clips sprinkled throughout the episode as you listen. So when the audio shifts, that's just, that's what's happening there. You're actually hearing from people on the day of. - Hello and welcome to the Clavio Data Science Podcast. Could you tell us who you are, what do you do here and what you're presenting on today? - Sure, I'm John Darby. I work on the K-Ops marketing team and we own the public-facing marketing site and the front end for Clavio's Help Center. I'm our technical product owner and a senior engineer. - Can you tell me a little bit about what you're presenting on? You have a great looking poster board here. I'd love to hear about it. - Yeah, we're presenting on our architecture to make those properties possible. So like I mentioned, the marketing site for Clavio and the Help Center, they both use Gatsby and static site generation approach. In the case of the marketing site, we have a WordPress content management system where our authors and contributors go and make content updates and we ingest all of those updates and combine that with code that lives in our GitHub repository and what comes out the other end is a collection of static assets like HTML, CSS and JavaScript that make up the Clavio Marketing website and are exposed to the public. In the case of the Help Center, the back end is Zindesque and onto contributors make their updates in Zindesque. We ingest those using Zindesque's API into an internal system with our own database. Combine it with some of our own data and expose a GraphQL API that we then query with Gatsby and do the same thing. Run it through a static site generation process, outcome static assets and lead to the front end you can see at help.clavio.com. You know, I know that we recently redid our website for expanding into other countries and other languages. Have you been a part of any of these projects? - Part of all of them actually. So the Kops Marketing team has been responsible for the marketing site when it lived as a legacy WordPress site and now a static site generated with Gatsby. The rebuilt site is what you see today at www.clavio.com. We are expanding internationally, the UK site is built using the same technology. So we did some work earlier in the year to convert to a monorepo architecture which allowed us to modularize effectively all of our code and it makes deploying a new site, a new international site extremely easy. I don't wanna say just how easy, but it's very easy. The early investments we made in the monorepo and this architecture paid dividends and allowed us to easily expand into the UK and soon beyond. - How has this event been so far? What's the coolest question that you've been asked? - Yeah, the event has been great. It's been fun to see everyone in person. I'm remote and a lot of my colleagues are also remote. So being in the office is always exciting and putting names to faces. - The most interesting question so far. - I think it was why we chose WordPress because it's a good question. It seems like in this sort of decoupled architecture WordPress might not be a natural choice. And we chose it for a few reasons. One was our content contributors were already familiar with the editing experience within WordPress. So no retraining required and that would have been a substantial investment. Another reason is the flexibility. So WordPress has a very robust plugin ecosystem. We have existing expertise on the team managing WordPress sites, writing PHP sort of developing plugins, themes, all the things you need to customize WordPress installation. But critically, there are plugins in the WordPress ecosystem that allow a WordPress site to expose a GraphQL API and being able to use GraphQL across the board for all of our properties has been incredible to say the least. So yeah. - Thank you very much, John. Good luck with the rest of the event. And thanks for sharing with me. - I'll go ahead and hand it over to Zach now. - So let's start with the basics. You told us the name of the event. But what exactly was the Clavio R&D science fair? - I can start off here. It's pretty much exactly what it sounds like. It's a science fair. The goal of it very much was we don't want this to act like other events where you have a bunch of people presenting to a central stream one after another. This very much was we wanted an event where people could walk around, could see different pieces of work and really we wanted people to be able to present work that they had done to people who aren't familiar with it in an interesting way. When I say science fair, it really was in conception like what you're thinking of back from middle and high school. - So the idea was like to maybe look at how we're spending the money and looking at ideas for how we can maybe make this more efficient, especially for areas that maybe our needs have changed over time. So partnering with Chris and platform engineering. All right, so the outcome of this was very long stats RFC. No one wants to read, it's 60 plus pages but the kind of the core points of this are five ideas. Number one, what we're calling forecasted auto scaling which would go into detail for it. Number two, hardware optimization on clusters that have maybe the wrong instance type right now. And number five, the spicy point is like maybe we should hire a bit more for some of this work because it's really valuable. All right, so number one, when we talk about forecasted auto scaling. So as you might imagine, we send way fewer emails at 3 a.m. then we do at 5 p.m. And this is really important because our demand on our clusters is really seasonal. Number two, so some of our clusters over time people leave our needs change or maybe not well optimized. So this is an example cluster, this unnamed one. Two thirds of that is always free. They max out at around 8% CPU utilization. So this is an area we're working with cluster owners. This one, we're doing this this quarter, we're testing on a specific cluster and seeing how that works. This one, we've reached out to the cluster owner teams and kind of talking back and forth about whether it makes sense because they need to do their own feasibility studies. This one, we created an executive summary and we're preparing a document to send to executive folks to hopefully get them to push because we cannot push on this, I think. This one, Chris is in control of this one and Chris I think is supportive of this in general. So I think we're able to make some progress on this. So why didn't you decide to hold the Clavio R&D science fair? - Yeah, I'm happy to answer this one. Initially we wanted to follow up on the success of a small group. sign fair event within data science by doing a second iteration of the event, although it was initially scope to just include data science leadership and communications suggested expanding the event presenters to include all of R&D and for the target audience to be all clavios. So although this was ambitious, we believe that this would be the best way to like maximize visibility and understanding of the work that R&D was accomplishing. Yeah, it was a bit of a light bulb moment for me. We were kind of in the planning process talking about, okay, we're going to have this data science event. Let's get a room that's large enough to fit all people from data science in it. And then just contacted us and say, hey, why not do this for people outside of just data science? Why not do this for the rest of engineering? Why not do this for the rest of the company? And, oh, right, we should do that. It was kind of what I thought about it. So, you're being kind, I kind of swooped in after the R&D leads heard about it and said, we should do this on the full level. And I said, can I please deal your time and your planning process and everything you've already done. Can I copy paste what you've made successful and do it on a larger scale? Do you think that's possible? And you want to both said, yes, we can do it. I think there's a reason science fairs work in school. And they're engaging and they explain things in a way that not everyone can understand not having a background. And I think professionals are no different. We're tired of sitting on Zoom. We're tired of sitting in those scale rooms looking at slide decks and people wanted to be engaged. And I think this was a really great way to share, to celebrate all the work that R&D is doing. But also teach people at the same time. So it was truly the energy there was incredible. That celebration was a really big part of it for me that I wanted people to be able to show off work that they were proud of work that they had put a lot of effort into and rightly so. That a big part of it for me was definitely just giving people the ability to in a format that's a little different. We have celebrations of wins at the roadmap update meetings, things along those lines. But there's a big difference between taking up half of a slide for a couple minutes and being able to actually walk someone through a feature that you built for seven minutes and answer any questions. They have on it. Okay, and we are here with Onika. Onika, could you remind us what you do here at Clavio and tell us a little bit about what you're presenting today? Yes, I'm a data scientist embedded onto the onboarding team and I'll be presenting on personalized welcome series flows. Cool. Tell us a little bit more about the personalized welcome series flows. Sure. So when an user signs up one of the first things they're going to want to do is set up a welcome series email flow. And right now our templates are just defaulted to generic text and this is going to take a lot of editing time for the user to get it to a place where they're going to actually want to send it so we want to do a better job of creating smarter default templates that are shown. So first we're extracting brand information from the website. So what brand voice are they using? How are they writing things a little bit about the company in their backstory? And then we're using that to generate email copy for six different pre selected templates that a user would include in their flows for example best sellers and like a welcome series with a coupon. I see on here a few different examples. One thing that I see is a non-English site. So does this project cover non-English websites in its scope? Yes. So we connect detect what language sites in and then we can prompt to write the welcome series email in that language. How has the event been so far for you today? Has the reception to this really cool project been? It's been great. It's been really neat to show kind of mid-project work to Klavio as a whole and get people's feedback and thoughts. What's the coolest question or most insightful question that you've gotten today? I think there was a lot of interesting discussions about other future directions we could take this so building out the actual structure of the flow for the user based on what we know about them in addition to the content. I thought that was pretty interesting to hear about something we hadn't really thought about before. Yeah, one of the benefits of being in person and holding an event like this obviously is that you get to kind of bounce ideas off of people who you might not otherwise see on Zoom. Yeah, I'm actually very excited. We're connecting with a few other teams that we hadn't been before from content and marketing to get more feedback and try to make this a better experience for users interacting with this generated content. Jumping off that too, I feel like oftentimes at those like the roadmap meetings or other places where you're demoing to a larger audience or people that aren't directly aware of your work. Sometimes it may not even be you doing that demo, right? Like it might be like you help put together the slide but then like the PM or the person that runs your department or somebody else whoever's available is actually talking through that. So you don't not only do not really have time or space to delve into any of the details or like custom tailor what you're talking about to what the interests are of the person that is asking the questions. But also like you get to talk about your work, which as an engineer is really nice to have that opportunity. I think I just want to double down on what Maya was just saying too is that a lot of people on my team don't get the opportunity to present. And so we were rotating through who was presenting the whole time and you know a lot of people got to practice those skills that they don't typically get to do on a day by day basis. Yeah, and I think that's great because there are other opportunities like that you can go to a professional conference and present there. But it's rare that you get an opportunity to do something like that where you have a friendly audience. People don't go out of the way to be unfriendly at professional conferences. But questions definitely can come off as high stakes as in the worst cases someone can ask a question in a way that seems confrontational. You get a lot less of that in an event like this. So it's really a friendlier way and a safer way to practice a lot of those skills I think. And conference is also tend to have exclusively technical audiences right part of I think what's great about I think what Jess and Michael we're talking about before with opening it up to having all clavias welcome. You got to talk with not just people with strong technical backgrounds that are asking technical questions but also like leaders and product that are asking more user flow focused questions and people that are CSM's that are like interested about how this affects the people that they work with on a daily basis and having a diverse audience definitely. I think added both to like the experience from a presenter side and also hopefully to the people walking around. I think the synchronous science fair presentation style created kind of like a positive feedback loop where the more foot traffic and the more chatter that we were generating in the areas tracked in more foot traffic and chatter. And I think that was like one of the main reasons why we wanted to hold a science fair event particularly in a week as populated as hub week where we have all of our remote based employees coming into the Boston office and getting to see what each other are doing. What is like a characteristic difference between a transactional message and a promotional message what would an example of each one of those be transactional messages specifically relate to a purchase or some sort of interaction with a brand. A common example would be your order is out for delivery or here's your tracking ID something like that where is promotional are often like here's some new products here's a coupon code we have a sale going on that sort of thing. I was this model going to end up being used directly in production right so in production we want to make sure that our users are able to request that their messages be labeled transactional. Automatically we don't want to have a person go in every single time and manually label them and so this is going to make it so that it's a self serve feature automated and allows to scale. Alright thank you Tristan great presentation and I appreciate it all the arrested development gifts on your slides so thank you very much. Yeah that gets that one of the other important reasons and this genuinely was I know that this answer is going to sound a little tongue and cheek but genuinely this was a very important part of the event me I wanted it to be fun. I wanted this to be an event where people could just walk around enjoy learning about new things enjoy talking about stuff that they've worked on with other people and I think that having it in this format where like we're not just asking people okay get up there and give some slides. We're saying like if you want to you can physically put together a poster board like bring yourself back to that mindset do some creative physical design you know actually tape some things on to onto a piece of cardboard that adds to the fun of it for me. You're so spot on both I was so shocked from an event planning perspective how many people wanted to do the poster boards we are here for glue scissors like sparkle glitter give it to me and I think it was a team building activity and then presenting it people walking around and seeing what they did with their boards was amazing but I also think. You know you haven't seen this feedback might well it wasn't just the energy in the room from the presenters the feedback from the survey that we sent out post you know this week of events this was far in a way the most highly reviewed event that we had people found it valuable from both the presentation and being their side like no one had anything constructive you should change this to say about this event it was all so positive so that fun was felt by everyone. Yeah from the presenter side I can definitely validate what just said like it was my team was so excited about like building the post board and even after the event we took our poster board and we brought it up to like the conference room where we always hang out as a team and it's still there. I went back into the office yesterday and I walked into the room and the poster board is hanging out like this is our space look at our post. And like also in the and the fun dimension, we were brainstorming other ways to kind of make our presentation like engaging and fun and we decided to put together actually like a little game that people could play when they came to check out our board where we set up a laptop with the previous experience right before we rolled out this new feature and we told people try to find all of the accessibility problems in this form, right? Like just go through it and see if you can find all of them without using the tool that we built and it was kind of both a fun way for people to like people show up and they're not sure what question asking we're like here play this game like see if you can figure it out and then when they're like oh my god like that one was impossible. I don't know how you expected me to find that that's when we can be like exactly and that's why we needed this feature and then we launched into the like this is exactly the problem that we're trying to solve and it both I think was like a really fun game for people to play. If candy to people that like found you know above a certain number of problems in the form and I think people really enjoyed that but also it was like such a engaging way to kind of show exactly why we wanted to build the feature why we rolled it out in the first place. Tell me about what you're presenting above to you got a beautiful looking poster board here and I want to hear all about it. One of our most highly requested features from our upmarket customers was the ability to show discount prices so say you have a Shopify store an item is $20 and it's now discounted $10. The previous state of the art was that our emails would only show the $10 price it wouldn't give the customer the notion of the price being reduced or a strike through price being shown in the emails. This was requested back in 2021 after BFCM and we're able to deliver this feature in time for this years BFCM so we hope that a lot of customers are happy with it once we do get back to it we'd like to expand this discount pricing to include such things as filtering on discount prices and other integrations like magento or magento to so excited things down the pipeline coming for discount prices if everything goes well. Thank you very much and how has this event been for you both so far. I was so happy with how this turned out seeing so many faces here and hour in I think I'm losing my voice already so it's packed here like I hope that the mic is picking up some of the background noise because it's just it's right as near there are a lot of people here it's been a really great turnout yeah. I am very disappointed that I missed your presentation you had a poster board you got a game there's candy involved I don't know how I missed that but it but it sounds like a really great time. Thank you so what actually goes into planning an event like this happy to start here so initially we want to probe interest for the event so we started up by sending kind of like testing form just to see what kind of reaction we would have people. Would would want to present at an event like this and there is also that we're very positive we saw that people were interested in presenting on an event we didn't make them choose a topic initially we just wanted like a team name and a point of contact. And after you know some advertising in various meetings like engineering all hands pillar specific meetings some team specific meetings we are able to get a list of I believe 16 presenters for this event which was really good once we had those numbers it became time to decide all right what physical space do we actually want to host this event in. And there we had to weigh in like the pros and cons of some spaces so for example if you happen in a room you might be able to fit a lot of people but it's kind of closed off it doesn't have as much of fluidity to it and that's where we settled on the atrium area which is one of our largest open areas in the office very central a lot of foot traffic there so that was what kind of went into planning the event from a logistics standpoint. Yeah and a lot of that was just working with our office team to figure out the space like I completely transparently I didn't think that it was ever going to be in the cards that we could get a space like the atrium that was I was like well you know it would be super cool if we could get a place like that but there's no way they'd let us but sure enough that's pertinent of being. Like the advertising part that you brought up that really was a big part of it there were kind of two sides of that obviously we had to kind of advertise the event to people that we wanted to present at the event both so like you know talking to engineering and data science teams and design teams and saying like hey this is an event that's out there we'd like you to present at it. Sometimes working through leadership to hey can you remind your teams so it's coming from a different lens than just us reminding people but then also it like advertising the event to everyone that we wanted to show up and enjoy the event both of those were crucial because without both sides of that this couldn't have been a successful event. My go to your point of thinking we weren't going to get the atrium event planning is a large part of my administrative work here at Clavio and I've learned really quickly coming on board ask for the stars and then backtrack if you can't get them and I mean our office ops team and our office was really built for this type of engagement and the more I do it the more I ask for and the more flexible they become and I'm just consistently impressed by it but I think to add on to you all of you. Event planning I think important for me from a project management representing the R&D leads standpoint I needed to identify the key stakeholders versus event planners versus showrunners so who's giving the sign off who's driving the work and who's actually on the ground the day of the event so we don't end up looking like that spider man meme at the end of it. So I'm well I thought you were doing that no that was on you and then there's someone which was me I think keeping all three on track and just doing occasional check-ins and saying we probably don't need a full meeting for this but can everyone just give me the thumbs up that you're spending your plate and still in the air this team I think I consistently set it in the slack like dream team everyone how their responsibility and their plate up in the air so well. Yeah an important part of the event obviously physical space that means we do need to actually work with IT to like have physical equipment they were very easy to work with the kind of identified here's the ask for like what we'd like we'd like monitors for people to present from if they actually do have slides so it's a little more engaging and active and again definite props to that team as well was very easy to work with them we ended up with a great set up at the end. We are here with Alan Alan I believe that this is your first time on our podcast would you mind telling us what you do here at Klavio. Yeah my name is Alan Chavez I run the engineering team here at Klavio Alan is our CTO it's great to have you here Alan how's this event been going for you so far. Oh it's been fantastic it's a great reminder of all the great stuff that we are doing I mean blown away by the amount of cool stuff that we are doing here today. Yeah it's been such a variety of interesting projects too I feel like it's amazing that this is one company's worth of like R&D work honestly do you have any favorite presentations that you've seen. It's hard to pick a favorite child so it's so many so this great stuff on generative AI I think the website a walk that was done is fantastic. Data dictionary is on favorite so I could go on and on and on to your point right it's so diverse and so interesting that the same company is doing all this at the same time right. Yeah is there anything that you know the coolest thing that you've learned here today. Oh that's a hard one all the then we are a fantastic company that is doing great stuff across the board no I can't call one thing because I will be unfair to the others. Yeah I mean obviously there's so much that goes into planning an event like this but you all did a really great job it felt like you had just showed up and everyone spontaneously was there with posters and it just you know it's like an effortless. Spontaneous event so really good job to all of you for planning and making it seems so easy that definitely takes a lot of work. That particular piece was partially because just put a meeting on the calendar hey everyone get there 30 minutes early. You know that is always a good way to make sure that people have have their stuff together. So there were a lot of moving pieces for this were there any surprises that came up during the planning process. I think the biggest one for me was how many different iterations of the map we had to put together output together the first iteration and then went out on some very well deserved pto in the couple days leading up to the event so I had what I was like alright we've got the final map here we want to make any changes to this team drops out okay well I guess we're going to move another team wants to join in the last minute well I guess I can swap them in another team wants to join in the last minute. Well we have this backup table another team says that they need to be close to another team because they're presenting closely related work. Okay well I can make these swap just there were a ton of teeny little iterations and then also. Okay I T says the monitor actually has to be here rather than here is mean to swap this team around which means mean to swap these teams around. I posted probably four or five different versions of the map that I told the channel were final. And then the one at the end ended up actually not being final because there was more room in the large age or in space than we thought so we had an extra team there so it was a mess. I think you handled that beautifully that's what I experienced with every that is just a event planning you're planning a wedding basically and people were. You know saying my kids sick I can't come and so you were changing the table scapes and I think with big events like this especially you just have to be prepared for things to not go the way you thought they were going to and that map was just one really great example of it but. You're flexible and everyone was really cool about telling us what they needed and why they needed it so. And in the end did we hand out the maps I can't even remember. We know. We know. We know. We know. We know. surprise for me was how willing people were to make poster boards. I didn't anticipate this going into planning this event. My initial thought where people was bad people would be more inclined to just, you know, create like a slide deck on a laptop and present that a la the style of classic remote presentations that people have gotten all to accustomed to. The fact that so many people are asking questions about poster boards and like how should we structure them? What should we do? How do we expense them? I think it was a good call on our part to just end up purchasing the poster board so that anyone that would want to make one would make one and I think that kind of had the knock on effect of one team would see another team making a poster board so they would feel like inclined to make a poster board and the fact that it was already there along with the supplies to make them. I think really helped contribute to like the look and feel of the event and I think this was alluded to earlier but the team building aspect of it as well super valuable. I think that was the biggest surprise for me too but in hindsight we should have known that people were into it because the number of inquiries that we got about building volcanoes we should have known this was going to be a tactile hands on event that people were into it but I'm glad that actually you actually had the idea of buying all the boards for as many teams as we would have and you know get those supplies there so props to you for seeing that forehead. If I can add in something that actually isn't a surprise I think something that I was not surprised to see was just how good the prep was by the teams. The teams that presented had their stuff together they had exactly what they were going to present on we had it well in advance to kind of I wasn't there looking at it like oh I'm gonna suggest edits to your poster board mostly just like for my sanity kind of know that you've put some thought into it and what I saw was instead of just like okay like an outline of the bullet points teams had their poster board fully made and they upload photos of it and stuff like that like teams had their stuff together they knew the story they were going to tell them they had it together well in advance and again that was not a surprise to me that's kind of just how I expect things to be here at Quavio and they knocked that out of the park. Michael how's the science fair been for you so far. I'm going around representing the remote viewers so I'm kind of going around telling people hey it's your turn to present to the webinar and I'm running out of people who have slides that they're presenting things are going to get real weird when I have to get the poster boards on to zoom. Do you have a strategy in mind for this or you're just going to do it live or my strategy is figure out something as I walk up to the first one. Okay well that sounds very excited I'll check back in with you later to see how that see how that goes it's a little early in the in the day to be running out of running out of slides. Trust me I know. All right good luck you have another hour and a half of this. So obviously planning has to happen not just for the people organizing the event as a whole but also for those who are giving the actual presentations. The next few questions are mainly aimed at those who presented at the science fair so let's start with the basics. How did you decide what you're going to present. I can take that one first for our teams like a lot of people still have questions of what is the design system what is the component library. How do they work together and so we wanted ours to really to be informative and also to kind of showcase a little bit of what has been done and then what our future plans are you know we kind of see the design system as like we're still in kind of a teenager years of like getting through we got past the infancy the toddler part and now we're like getting into that teenager angst and so you know we really wanted to showcase that and so like you said a lot of preparation still had to go into what we were doing as well so we discussed amongst a team we created the boards and then I also created a cheat sheet for each of them kind of like the talking points and then both of the managers did the first presentation like the first time slot and then we handed out to the rest of the team so they could see how we kind of presented it and then they had the little cheat sheet in front of them to kind of if they got stuck on anything they could look down and see a talking point. Yeah so my team during one of our team meetings we all sat together and we're like okay like what do we think would be good to highlight we kind of all knew that we wanted to be involved but there was a question about like what project in particular we wanted to talk about especially because my team tends to work on a lot of things in parallel ultimately we decided on showcasing the formal arts project partially because there's been kind of that project right now or like the version of it that we just release is really the culmination of like a couple of different features that we've released over the past year and really just something that we've been thinking about for a long time on the audience growth team so not only did we get to kind of present the form alerts feature which is something that I built but also as part of that we got to talk about the accessibility assistant which Chaz built and Catherine helped resolving some of the accessibility issues and we had to talk about all of these alerts or warnings that have been added into the product while building out other features you know things that other engineers on our team worked on and so like not only was it a project that highlighted a lot of different people's works but we kind of got to speak to I think problems that exist not just informs at Clavio we you know we're like hey like we literally had a customer hold this morning where somebody was like hey I think modals are like less than ideal and you guys have it everywhere in Clavio and I'm like yeah that's exactly what we're trying to fix here and so we got to like speak to problems that I think are widespread we got to speak to how many different engineers on our team then tackles those problems and we also got to kind of show the way that engineering can kind of address things in an incremental way right so we're like the here's how it was like two years ago and then here's this fix that we did like six months ago that took us here and then here's another improvement that we did that took us here and now here's the feature that we're rolling out that's going to take us all the way to here and then here's where we think it can go in the future and like maybe we built this in a way that's generic so maybe like you on your team if you think that this is interesting you could add this to your part of the product and and just like all of these different aspects of engineering and product and the way the development happens at Clavio I think got to be highlighted by talking about this particular project and so once we settle on that we kind of like split up we got all the people with free time together and we wrote a big list of everything that we wanted we gave like action items to everybody and some of them were just grab this picture and put it in the slide deck and other ones were like right out you know the things that we want to put on the slides my job was like throw together the little example form that we're going to use for the game and then we kind of just talk through who would be presenting and how we want to handle that once we kind of had it all figured out then it was just a matter of showing up and making it happen how has this impacted data scientist now so earlier what happened is as I mentioned once data scientists had them already they needed to build their own service they needed to like write their own unit tests they needed to like deploy that into production and now they just take their model file upload that to s3 poppype is their code into our repo they already have their prediction code from training and they give an example for an input to the model and they expected output for the model right and then we'd run all of those tests in CI for you and then once you merge that PRN you have an endpoint available that you can call from app we are in the process of starting the opening the service up to the public so resaws a lot of that by itself so how it works is you can specify the number of resources you require like you can specify a number of CPU memory that you need all of that if you don't specify that it just resets to defaults and then it uses Kubernetes so what it does is based on like what resources specify it would get a single Kubernetes node and then try to fit in as many models as possible those models would still be isolated they would be running in their one environment and then once that machine is full it just creates another machine does the same thing yeah so this event was open to everyone and not just R&D and indeed quite a few people outside of R&D did show up so you both started to touch on this Jason mentioned your talking points and you know my candy and games are always a winner but how did you approach making your presentation accessible to those who don't necessarily work in the same area or have the same technical background I can start on this one I think for us it was really about like the framing of the feature so we wanted the thing that we led with and a lot of the focus on the board and the things that were interactive the things that were going to be catching people's eyes we wanted the focus to be on like what is the problem that exists how does this affect our customers and what would be the ideal solution right if we just like close their eyes and thought about an ideal world where there isn't a problem what about it is solved and how we then went from that until like okay well here's the design here's the flow that we decided on and then from there we could get into the technical details like we use this package it's open source that you know means that we get accessibility updates for free and that's awesome and talked about the way that we built it from a technical standpoint and so the nice thing about doing it in that order I think is that you're kind of starting from like a like a wider funnel where you're starting with the things that I think are easy for people to understand and then getting more narrow and so at any point as you're starting to like go down that funnel people can interject right and start with questions about the things that they're particularly interested in so for some people I never got to the bottom of that funnel we didn't really talk about the technical stuff because that was less relevant to the questions that they had in their mind and the things that they wanted to talk to us about and that's fine right we still had an excellent conversation about the feature and what we built and things that they thought they had questions about and wanted to talk about and for other people I didn't even go through my whole spiel because like three sentences in there like but I want to know what you did here in this technical thing and so you kind of start I think with the more general stuff and then allow yourself to be flexible based off the person that you're talking to and where their interests reveals itself to lie. Yeah, we built our boards with kind of more themes and then you just kind of like are talking to get into the details and so the themes allowed people to kind of understand things from a holistic standpoint and if they didn't have a lot of knowledge, we just talked at a high level and then if they had more knowledge, we then dug into some of the details. So some people, we had a lot of people come to our table that were new to Claudio. I've never met before. It was more people that were new to me or the ones that came up and it was really great because we had a lot of questions of like why didn't we just use open source component library? Why did we build our own? What is the purpose of it? And so they really wanted to dig in deep and they could use the boards as a way to understand where we're at now, but they wanted to know how do we get there and where we're going as well. Yeah, and just to tack on and that's something I saw on both like Maya and Jason's boards and also like in the board that my team product merchandising had built was visuals lots and lots of visuals. I think that was the main way that we were able to be able to like cast our net to people that were kind of walking around kind of inspecting boards, seeing what they were understanding, what they weren't understanding and once their eyes fixated on a visual, it became pretty easy to kind of like hook them and reel them into a conversation or a presentation about what it is that we accomplished or what we were working on. So I think a lot of people were visual learners at this company and when you can really explain something, explain a feature that you've built, explain the technical challenges involved in building something and they're able to visualize what the end product for that looks like. It really helps with their understanding. And I ask a question even though I'm not the host here, I'm thinking you know, visuals if you're you doing a PowerPoint, PowerPoints can have a lot of text like what visuals did you use any of you that really grabbed people? It worked for you. I think one visual that I found myself like gesturing to a lot and I think a lot of people were like pointing out and being like explain that was one where we had and this is a little bit silly, but it's one where we had taken every single modal error message pop up toast like every single like little intrusive thing that would pop up this part as the published flow and we just like kind of put them all on top of each other. We're just like look at all these things and that was when the people were like customer see that and I'm like yeah, this is what it was like look at how chaotic this is, look at how much is going on. And I'm like does this feel like understandable to you? Can you look at this and understand what's happening? And they're like God no. And so I think we did leverage it a little bit. We intentionally made it mildly chaotic, I think to demonstrate the point, but that was one where I think there was text in that image, but the text was not what was important in that image, right? It was all of the different colors and all of the different boxes and I think that was one that seems to catch people's eye. I wasn't one of the ones who presented, but one that I remember seeing walking around that's really stuck in my mind, this was for an AI-generated templates project that Charlie is working on. And I just remember kind of walking past this board and doing it double-take like, is that a picture of Crocs? And sure enough, it was like an email generated to talk about the selling points of these Crocs. So sometimes it's literally that simple. It's that silly. Oh, I will also say in one of our example forms, my team personally is really enamored with images of frogs sitting like people. I don't remember how that started, but it's been a thing for a while. And so in all of our example forms, we had frogs sitting like people. And I think a couple people did call that out. They're like, is that a frog? I remember like, it is, he's very important. So that one's maybe a little bit less relevant to the cause, but just if you said about Crocs, Michael kind of reminded me of that. I really missed out big time on your presentation. I want to see these frogs. But can you confirm that Charlie's poster had a lot going on? That's very exciting. Okay, we are back with Michael for an update on how this event is going. Michael, what do you have for us? All right. So I finally ripped the bandaid off, went to a presentation that was entirely a poster board. And it went better than I had any right to expect. Charlie was the one doing that. I was like, Hey, could you use my laptop and kind of point them to the presentation? I was expecting him to like set it down on the table or something. He had that thing held up. He was like holding it right up to the poster board like look at this picture of Crocs. It was incredible. You know, I watched that. That was that was impressive both skill and dedication to making sure that people on zoom saw that. Like he was like panning over the images. It was an academic and acrobatic presentation. We appreciate Charlie very much for this. Cool. Well, otherwise, how has this event been going for you? It great. There have been so many more people than I expected. Yes. As someone organizing an event like this, my worries always 10 people are going to show up. I've seen like hundreds of people circulating through so far. I'm just happy that this many people are getting to see the cool work that our engineering teams are doing. Yeah. And people have been really excited. Like people have been having a really great time. They're really engaged with all the presentations. It's been really been really great to see. Yeah. So this event had honestly quite the crowd. We really filled up the atrium. What was it like presenting your work to a crowd? There were a lot of people that attended, but at your table at any one point, it was like presenting to a more intimate like group of two or three people. So I think it was really approachable for people that were not typically used to presenting to large audiences. Our atrium is this large area with multiple tiered steps. It's like a stadium going up. If you were to stand in front there and present to everyone sitting, it would be pretty intimidating. But this made it much more intimate setting and just would have a rotating group of people. Some that only wanted to glance and other people that really wanted to begin. Yeah. I will definitely say that getting to like getting to like pull people out of the group. Like I remember like waving like shaking the bag of candy, being like come come try and enjoy our game and like gesturing people over and then you know giving them this feel. I think that that made it a little bit more it just made you be able to like tailor your presentation to people in a way that you can't when you're presenting to a large group, which I personally really enjoy. I've always been a talker. I love talking. I can run my mouth for ages. But I think there's just something different about talking to one or two or three people getting to to see what interests them and speak more to that or answer their questions and have it almost be more of a discussion than a presentation that I really really enjoyed about the science fair. And which I don't get to experience as much when I'm presenting my work say like in a demo meeting or out of road maps meeting where it's a little bit more formalized and it feels more like okay I'm going to give my presentation and there will be a structured Q&A and there's something about just like the fluidity of the conversation that you get in a science fair that I really enjoyed. Yeah I really need to drink a lot of water. I was yeah really parched after just like a couple of like presentations and one thing I noticed myself doing was you know at the start you give like a quick 30-second elevator pitch slash summary of like what you've been doing and the first you know summary that I gave I was like oh I didn't really like how I explained this part of this project and then I iterated on that so for the next part the next summary I gave I was able to make that a little more concise and then you know you figure out what you like and you didn't like about that summary and then you keep iterating on it it's almost like speed dating where you get to continually improve on how you're explaining a project and keep making it more concise and I feel like that's like a really valuable presentation skill that you don't get from just giving a one-off presentation the fact that you get to keep iterating on it with like multiple different audiences I think makes you more well-rounded presenter so I really enjoyed that aspect a bit. Yeah I always joke that I never got my clavio pitch down until I represented us at a career fair because you know the first time you get the pitch and you're just like oh god that was great by hour five you're like you've just got it so now after after talking about formal arts for a couple hours I feel like I'm very well equipped to tell people about this feature it's a it's a type of practice that you don't often get to do. I have to give a quick apology for throwing a wrench into some of that and I was I was walking around kind of with a laptop to represent the remote crowd so that we could kind of get these presentations presented to people who weren't able to be there the day of as well and I know it was definitely a different style when you're kind of all right I need to talk to the microphone on the laptop but I also need to hold the laptop so that it can like the camera catches the poster board behind me I know that that was a whole complication that to add in there as well. Yeah but I think it was worth it to try and include people that couldn't be there in person right like this is part of the challenge of being a hybrid workplace that you want to make sure that everyone has an opportunity to be involved and I think we managed. Oh yeah yeah we we had like a larger number of people watching online than I expected honestly throughout pretty much the entire day and got a lot of questions like you know different questions throughout and definitely got some kudos afterwards and people were very very thankful for presenters for kind of going through that so that they could actually learn about the features too. So throughout the course of this event I'll talk to a lot of of people but have your projects. Did you have a favorite question that you got asked by someone in the audience or to Maya's point? Did you have kind of a favorite discussion or conversation? I don't think that I have a favorite, but I think one that stuck in my mind was the conversation I had with Alan, who's the CTO at Clavio, and he was just kind of wandering around. He came over and asked about the project. And I think what maybe stuck in my mind about it is that as I was talking to the future, and I started to talk about the future of this, where me and the rest of my team thought that this could go and started talking about incorporating this into other areas at Clavio, and how we took extra time to build it generically so that that will be easier for other teams if they want to do it, and talking about incorporating intelligence and suggestions into it, and how this enables that in a way that nothing has ever really before in the forums area, or really in the editors area in general, and talking about all those things and getting excited about that, and then having him be like, "Yeah, this is great, and have you thought about that?" And like engaging me in those conversations of future things, it's one thing to be able to talk about the future with the people on my team, who I think have lived and breathed the things that we work on for a while, but it's another thing to A, get to talk to somebody who's higher up in the train of Clavios, and have them be, I think, as excited about the future as I was, and be also for them to validate me and my team on our ideas, and be like, "Yeah, I do think that's good." If you haven't talked to this team or if you haven't talked to editors, that's excellent. Actually, shout out to Christina Dedric as well, who literally came up, and as I was saying that, and was like, "I have someone I want you to talk to," and then slacked me the name of the person, and was like, "This is going to be a really good person for you to connect with if and when you start building that." That was awesome, too. Just really getting to talk to people that I think do have influence over the direction that Clavio moves, and having them validate ideas for the future that me and my team had, it felt great. I got a lot of questions around one piece of our board that I kind of threw in as a pillar, like I had a gap in the board, and I was like, "What can I put there?" I was like, "Oh, I'll put the different layers of a design system from foundations, the components, to feature components, to patterns, the templates, and how, as you go up, the flexibility is less, but the efficiency is more because you can just kind of grab and put it in, and as you get more towards the atomic elements like a button or a checkbox, you have more flexibility, but it takes more time to assemble those. I thought it was just kind of a given that people know about that, and kind of like, thinking that way, and I had more questions about that than I had about anything else. I didn't even plan on talking about it much, so that was a real interesting piece. Not sure to do have anything, particularly. I'll probably echo what Maya said and didn't really have a favorite question, but definitely had a lot of a lot of fun discussions, specifically about what our project was, which was surfacing discount prices in emails. Yeah, it's tough for me to remember, then, though, apologies. So I'd like to wrap up with a few questions to everyone. First off, did you have a favorite moment of the science fair overall? I can say it from, I mean, because I wasn't presenting, and I didn't have to prep anything. Just standing, I remember watching everybody filing in and getting ready, but then coming back an hour later when it was in full swing and just watching and feeling the full environment of it. It was incredible. Yeah, I feel like for me, it was probably the moments where people were playing our game and got frustrated because that, it felt like people were engaged, right? You don't get frustrated, something if you don't care about it. And so on the one hand, I was just like, wow, people are really into this. And on the other hand, it was so great because I was just like, you understand, this is the problem that we're trying to solve. I just remember being so giddy about it, like having done exactly, it really felt like the moment that the plan came together. And I was just like, yes, that's teamwork, maybe we did this. Yeah, I think just the energy just was great. You know, like all these people showed up, it's one of those things it's like having an open house or something is like someone gets like show up to blonde by my house. When you have a huge crowd, the excitement gets there. It's kind of like that. And we also created some like branded swag for the design system. And people were super excited about that too. And so we're like, now we're on people's laptops. Yeah. And wherever else they stick the stickers and things. So my favorite part of it was just seeing, you know, the sheer creativity that people had. I mean, it's evidence and the presenters we have in this podcast alone, but going through at least initially when everyone was setting up, because unfortunately, I didn't have as much time during the event to circle around. But initially, when everyone was setting up and I got to see what fun ideas, what cool designs, what funny memes people had come up with on their poster boards, I was super proud of the work that the presenters had put in definitely surpassed any expectations I had of the quality of the presentations for sure. Yeah, agreed on that. And I don't think this is my single favorite moment, but it is one of my favorite moments from this process. It actually comes from before the science fair. It comes during kind of the the sign up process of all things, kind of just when we were gauging team's interest, one of the first responses we got to that forum came from the chaos team. And we kind of had like a as you do informs like this, we had a section where we just kind of say anything else you want to tell us any other recommendations. And they said, make sure we have room to move. We're an active group and we will have accessories. And I was like, this is the energy I want all of the teams to have. This is amazing. And they did. They all brought it. We are here with Christina. Tell us how your experience has been at this event so far. That's been really awesome. There's been a lot of different work being shown. Whenever I see a graph, I'm like, oh my gosh, I need to go talk to this person because there is a graph in front of me and I want to be explained when I am supposed to see in the graph. And there's been a few presentations with graphs in them, which has been very exciting for someone who's obsessed with numbers. Do you have a favorite graph that you have seen so far? My favorite graph is one by David on one of the data science teams. And he was working on an email churn model. And they were using the traffic cone as inspiration or not traffic on traffic light as inspiration for how they're addressing customer issues with CSMs. And so they were using intuitive colors like green for low risk, yellow for medium risk and red for high risk. And what was really cool about David's graphs is that he used that same color coding on the graph to further emphasize. This score is associated with the green graph. And it's in green. And this is what the yellow, which is in yellow and red and which is red, which is focusing. Little touches like that are fun. Anomalies also had a ton of graphs. You know, I have not talked to anomalies yet. They're over there. If Willie is there, she is the main expert on graphs. Rumor has it. She may have seen as many as 8,000 graphs in developing some of the anomalies features. I remember talking to her about that actually. Yes. I've heard those rumors. So another question to the whole group. Do you feel like you learned anything from participating in the science fair? Yeah. Outside of obviously learning about what the other teams were up to, what their projects were. I feel like I learned a lot about, I'll approach this from like an engineering standpoint. I feel like I have the tendency to kind of focus and hone in on the technical aspects of a project and kind of immerse myself in like the engineering side of it. And one cool learning experience from this event was being able to like kind of take a step back and actually reflect on the impact that your project has had in the general play view application. And what most people actually end up seeing as the result of your work. It's not the technical challenges. It's not the like funky little button that you're able to style the exact way that you wanted to. It's the outcome of it. It's like the holistic product that you've delivered. I think this was like very cool reflection on the undertaking of a project. And that was a learning experience for me. I think mine because I'm a little bit of the outlier here working in the administrative function will be a little bit different. But I mean, I learned a lot certainly from the presentations. I support R&D, but I don't do the work. So it was great to understand it. But I think for me as an executive business partner, there's always a lot of fear in planning an event. And that has your name attached to it because you are associated with the success or failure. But this event really taught me to trust my peers. Like Clavio, I've been able to do that since stepping in here. And that's been wonderful. But this is the most I've ever stepped back and said, yeah, I trust you. You know what you're doing. Just run with it. And I'll get you the okays that you need. And what do you need for me? And so that was in a week where it was just a event after a event for me. It was such a light and load and taught me that I can do that. So it was super valuable for me. I think I learned that I need to find more opportunities to have casual conversations with people because I learned a lot from the questions that were being asked. And I think they learned a lot by not having, as we kind of mentioned before, like structured presentations. We have a little too many of those. And like finding other ways, not just office hours, but just maybe having a topic once a month that we talk about for this area. And people can just join and just have a casual conversation about it, I think would be beneficial to the culture. - Yeah, I think I agree really strongly with what Jason said, getting like a really pressure reminder of a really strong reminder of the fact that like having opportunities for less structured presentation and something that felt more like discussion, it ends up teaching more, other people end up learning more about the things that I've made, and I end up learning more about what other people have made. And additionally, I feel like I also learn more about what people want to know about the things that I made. And additionally, I think it was a good reminder to me that there's something really special about getting to have those kinds of discussions with people in person. I work hybrid a lot of the time and pride myself on like being able to do my job just as well from home as in the office, but also getting to really be there and fuel the energy in the room and hear just the amount of conversations happening in that room at the same time. I think it's something that I was grateful I was able to like be there for. - Yeah, along similar lines, I don't know if mine's a learning exactly, it's probably something that we all already know, but it was just a good reminder of the power of just putting smart people in a room together and not telling them what to do too much. I think exactly along the lines that Jason and Maya you went over, it was constantly surprising to me just like how many interesting discussions I was hearing a snippet of as I walk past a table or the anecdotes I've heard since then on like, oh, these people kind of met at the science fair talked about a thing and like that's a collaboration that might be in the work soon. All of that, it's just a good reminder that yes, we plan events like this and part of the reason is to have fun, but genuinely just putting smart people in a room together and kind of saying like, all right, go do your thing. Can we do some really, really interesting outputs in general? - David, welcome. - Isaac, how's it going? - I'm good, thanks for asking. David, I think it's been a while since you've been on, could you remind us who you are and what you do here at Clavio? - Yeah, I'm David, I'm a data scientist here at Clavio and I work on the KOPs data science core team helping other internal teams to be more efficient. - Cool, well, thank you for that. And David, how have you been enjoying the event so far today? - Oh, it's been phenomenal. I've learned so much by seeing the presentations of my coworkers really fun to interact with them and I got a lot of great questions on the work I presented. - Could you tell us a little bit about the work that you presented and some of those great questions that you got? - Yeah, absolutely, I'd be happy to. So I presented on the account health initiative that my team is taking on and this is an internal initiative to help our customer success and support teams identify opportunities with existing customers and flag customers that are high risk so that they can act and help them to make the best use of Clavio as possible. And I got some really, really great questions asking about the rollout of this, what this looks like from the CSM's perspective and how we're thinking about this as a company to ensure smooth structure. So finally, as always, let's close with some practical advice. Let's say that some of our listeners get inspired and decide to do something similar to be Clavio R&D science fair in their own workplace or department or club or whatever. You all contributed in your own ways to making this a really awesome event. What advice do you have for someone who's in the same position that you were in? - Go all in on the art supplies. Do more than you think you need because people will use them. I'm talking 3D sticky letters, all of it, buy it. It's worth it. - Yeah, I would say don't be afraid to think outside the box when one of my teammates suggested the game idea. I was just like, that feels a little much and I was so happy we did it. I was so happy we did it. Yeah, kind of going off with Jess said, be extra. It's only gonna benefit everybody. - I think in my shoe is one of the big things that I take away is trust the quality of what you're putting together. I didn't know how many people were gonna show up to this but we had taken the right steps to get people to know about the event and have the awareness and at a certain point don't stress about things like that. Do the correct prep work but at a certain point just trust that you've put a good event together. - For me, it was communication. This is my first time being part of organizing an event at Clavio and communication. And in my opinion, there was no such thing as over communication ensuring everyone was in the loop. People were being filled in presenters knew what to do. Sometimes repeating yourself is the best thing that you can do to make sure everyone is on the same page and that was something valuable that I learned. I wasn't being annoying by asking people to upload their posters or slides to a folder. I was just trying to make sure everybody was in the loop and everyone knew what was going on. So that was a valuable takeaway for me. - I think from a presenter standpoint, it's less about having something super, super amazing to present and it's just about having a topic that you're comfortable to talk about and the conversation will go its own way. For presenters, don't worry about if you have enough to share or if you're going to have the most awesome graphics. It's just really about like do you have interesting things to say and the conversation will make its own path in turn. And that is the end of the time we have for the episode this month. This episode as all episodes of the Clavio Data Science podcast are with sponsored by Clavio. If you're interested to know more about Clavio, Clavio's intelligent marketing automation platform makes it easy for marketers to centralize and use every piece of their customer data to deliver hyper personalized experiences across all their channels, increasing conversions and revenue. If you want to know more, you can check out clavio.com. That's k-l-a-v-i-o.com. If you liked what you heard on this episode of the Clavio Data Science podcast, if you'd like to hear more than I have good news for you, this is episode 39 of the podcast. There are 38 other episodes. If you want to hear more in the future as they come out, a great way to do that is to subscribe to the podcast. You should be able to do that just about anywhere you listen to this. If you're interested in helping other people hear content like this, then I'm going to recommend three things. One is leave us a review. One is give us a rating, a thumbs up or a star rating or whatever it is, depending on exactly the platform you're listening to this in and then the third and the most important is, if you know someone who you think would be interested in hearing this episode, go ahead and share it directly to them. That's the most effective way to share the clavio data science podcast is just go ahead and share it directly. In addition, if you have any questions, the best person to contact is me. The best place to reach me is my Twitter account. That's Lawson_M_T, L-A-W-S-O-N, L-A-W_M_T, thanks for listening. Have a great month. (upbeat music)