Decoding AI: Expert Panel Insights for CTOs

From 7 CTOs, my name is ATN De Bruyne, and you're listening to the CTO podcast. Every week I spend time with fascinating people that enrich the lives of chief technology offices around the world. From perfecting the basics of building technology organizations to inspiring our minds into shaping our future. As always, the CTO podcast is brought to you by 7 CTOs. Helping CTOs become world-class leaders. Let's go! Welcome to the CTO podcast. We have Sharon CTO at personal.ai, Lina, CTO co-founder at the Good Face project, Cassini. AI Director at Airspace, we had a great conversation about all things AI, how they apply AI to their companies, they all have AI in production. And we got a little philosophical about where AI is going, the impact it has on our children and our robot children. And we give you a little bit of advice at the end on how to attack this problem with your CTO, so check it out. Great, so why don't we just do a little intro? I'm really interested to hear what your AI stack is right now, because a lot of people are talking about the evolving tool sets, and it seems like some tool is replaced by another tool every two months. So just if we took a snapshot of your companies today, you all have AI in production, so what is your AI tool your stack look like? And maybe as we talk about that, what your company is applying AI towards. And I would be amazed if someone is answering and you have a question about what someone is saying, just jump in and ask for clarification, and let's get the chat going. All right, I'll go first. It's just briefly 30 seconds about what we do our personal AI, especially about individual models for employees in organizations. So for example, we actually have five AI employees we call them in our organization, and we have about a dozen humans as well. So basically how it functions is we build something called personal language models based on their provided data, and then we combine that in a hybrid fashion with large language models to be able to produce accurate results for that employee and usually deal with quite sensitive industries like healthcare industry, finance industry and legal industry primarily. You can of course also use this for personal reasons, hence personal AI. So we do have both B2B and B2C way. In terms of AI stack, we actually build most of things ourselves except for some external alarm that we tap into such as on topic, but most of what we have right now are all in house. We do run Kubernetes, so all of them are microservices. There's around like 40 to 50 of them running at this, you know, each time. So yeah, I don't know what else I can say. The AI stack, but combination of open source, close source, and also for priority. I love it. Thank you, Sharon. Lena, how about you? OK, so I'm sitting out with this project and we develop the IP chemistry. And so as chemistry is not allowing for, you know, any great areas or ambiguity and all lambs are actually all about that. So LMS, we of course use them, but they're basically almost not so significant as the part of all the proprietary things that we're building. And yes, also Kubernetes, then like hundreds of different models interconnected, cleaning data, supplying data, pushing, you know, pushing back, pushing forward. It's where LMS were like at this point, like we're just trying to use everything that's on the markets before we say, OK, this is not working for us. So, of course, we're not using you know huge LMS, like, you know, that would be very hard to spin up. So we actually prefer smaller ones and they give very good results. So a lot of monkeys, very good results, mixture, mixture, you know, everything that you can actually get for free. And they can really compare to change equality. Love it. Cassania. I am at airspace, which is a time critical logistics startup that were more advanced face, not a tiny startup, we've been around for a while. So we have a little legacy models, machine learning, deep learning, they run in TensorFlow and they really power through a lot of our operations routing. Obviously recently, as of year and a half, we started creating. Compute different type of models based on LLM will live in the Google universe. So we're allowing Gemini 1.5 Pro, which is great. We build how it's executed. We would have an app that solves a particular task that consists of managers and agents. So a lot of little tiny single task agents that are just function polls and. A, the. With the query, Gemini. The old package and they run together on cloud run on Google. So we will very deeply integrated in the Google universe and they share some tools. You can think of those agents as little Lego blocks that then build particular apps. So some blocks are reused slightly modified and some of the spoke mode every block necessarily uses LLM. So it's been very exciting, interesting with automating tasks related to human understanding of text and digitizing it. So that's the main application in various ways. Obviously we get emails, notes, interactions. Personally also on my team for productivity, we use co pilot and chat GPD to really put the development on much faster speed given a relatively small size of our team. So with with personal AI and good face project, you guys are specific AI is sort of. Court your product. And then Cassini is it correct to say that AI sort of is enhancing the product at airspace. When, you know, as all of you have been involved in AI for a long time, when did you see. When did you start seeing that things were changing? Was it with the attention is all you need paper that was published and were you aware of that and were you sort of. Writing that train and you were sort of oh my goodness, something is going to happen or do you feel like you were caught by surprise as well and like a lot of CTOs today, I think feel. Like this came out of nowhere and a little bit and so I'm curious for you in your personal journeys. What happened. So quickly start I don't think it just changed rapidly. I think it's always been evolving. I joined this world and a late 2014 I believe and it was called data science and things were rapidly progressing. I remember being overwhelmed and feeling like I can never keep up because things were changing back then and I think there were a lot of things that. We're done it was just less it was more like a nerdy thing to do or very few people and the only thing that changed ultimately I believe is that how aware people have become of that. Things were building are surely possible because of the power of elements but also a lot of legacy things that we have and even the things we are building potentially could have been possible and just as powerful. It just has a lot of much more attention and I think that fear of missing out that people are having is because now they're hearing about it more but there was a similar conversations where. Executives were saying all big data I'm sitting on all those data big data I'm missing out same thing that didn't know what big data meant but they felt like they were missing out because they didn't have big data or they had it and didn't know what to do with that. Yeah I'm a little disappointed casino you and I spoke about this before but you know if you look at the trend with machine learning big data. TensorFlow I'm just disappointed that our people amazed right now only because there was a chat interface added to an LLM and now everyone was blown away or another non technical person is enamored with AI. I'll let someone else answer I don't want to dominate this conversation but. I think I'll address like the first question so i'm dating myself by starting a I like into which is way way way before any excitement. In fact I have someone who told me and now is does a LLP mean you know your linguistic processing sorry what's it called your linguistic programming anyway so for me I definitely did have a moment and that moment probably was in the attentions all you need which I think came out in 2017. The dream time frame but was actually alpha zero so at that time my boss who's the city of my last startup he was a huge goal fan right. I remember watching the matches even though I didn't really play go and I actually remember him being disappointed as a goal player because it's like oh no now I have lost like the thing that human can be machines for. Definitely right and I think alpha zero was more of a moment for me because the it was learning on self play right which was always a huge problem because you have to do human labels etc for what you call supervised machine learning usually. So that was a big moment for me and I think the attention is all you need definitely came into play once transformer become became a thing really and you can just do like import transformers for hugging phase. But yeah I think I do have a quite exciting moment was actually the chat GPT moment but it was definitely the alpha moment another really interesting one was alpha star. So which came out a few years later and it was playing essentially doda or corporate star craft. With the AI team against the human team right and that was like kind of the first very complex I placed the craft to so it felt like very complex as I suck at it for the human. It was awesome to see like AI being able to collaborate and at that time like agent right being able to collaborate and then reaching a goal that's quite expensive right. So that's kind of my answer for that and then I think second part of questions interesting I do believe though like personally that UX innovations do innovation because it does bring more accessibility to the rest of the world for this technology right. So like my parents for example I would have never imagined them like I spent 10 years on my life trying to explain to them why I didn't and then suddenly a chat GPT happened they're like oh I know I understand what you do right. So I think it did have a moment of you know just like bring it closer to the human I think of like the first programming languages even before the punch cards right. You would actually have to type in zero ones is a zero one so it was like human speaking machine language and now I feel it's finally a moment where machines are speaking closer to human language which was I think stunning and today you know I think zero one came out or one zero came out which mimics thinking I suppose I haven't read it yet but I thought that was awesome. Yeah it's funny that came out today right just I think it was a nice this morning I don't even know what it is I don't even know what it is what is one zero. So the new iteration of the model that tests at college level in certain topics such as I think especially mathematics and physics there's still a bit behind and more like language things but they have massive gangs over the previous iteration on. And tests I think a score is a hundred an SAT math and. Regid level physics and chemistry tests and not familiar with it scored much significantly higher than the previous. They're already thinking about implementing it into the smartphones so that you know the next generation of students will have it way easier. Yes I just have to pause you for a second what is that website where they do all the little tests for the AI models. That supposedly fourth graders and fifth graders can pass but that is a competition to run to you guys know about that. Okay I'll have to search that up. Lena how about you what was your what was your moment because good size project predates all of this right. Yes so I started the journey in the year in 2003. So at that point we already did image recognition and already we're building 3D models based on you know three pictures made up an object. It was text recognition there were like some cool predictions that were you know made a base of images of text of data. And it just evolved in quietly behind the scenes and you know it never stopped and it will never stop. It's just that this specifically couple of last year I'm completely agree with Sharon and saying it just became very visible for everyone it basically became a hype. You know this this moment of everyone looking at AI and asking themselves what is like what are we doing with AI like you know every CEO came to their team and asked this what do we do about AI. And many did not have answers everybody was worried I think everyone is still worried there they will miss out but also since AI is very you know general right now everybody is looking for the best applications of the product and actually it's like you know. Very get go when AI was more of data science and data analytics and it was mostly like you know even rules all the you know decision trees and other models combined together that were producing results. That era if you would give humanity a button that would say make it perfect like people will still would know what to do with that. That's the problem like sometimes even you have a tool you don't know the question and if you don't know the question like you cannot really use the tool and I think that's what's happening right now with the tool that we have. The questions are forming right now by the by different industries and we're looking at what is achievable with this tool and what not. That this boiling down will be going like behind the scenes in many companies and well it was a hype and many companies raised a lot of money during this type you know I think that you know the number of companies will actually significantly shrink. Because actual products that are both viable deliver results and make sense from this point of view of price there's still in the built and figured out and I think that's why we're all here right talking to you at the end. I'm reading a memoir by Kara Swisher I don't know if you're familiar she was a is a tech analyst she I started it and I'm going to use one third of the way she was very early tech adopter and she was a journalist who early on in the late 80's early 90's like okay internet tech. They're excited about things and really move to California to follow it closely and I think we're kind of in a similar era where obviously internet changed a lot of things right undeniable the biggest effect that bigger than if we probably even understand now. The people is understanding it back then at the inception drawn application was wrong and she had a good got to follow who is going to make it and I think right now what Lena said people who have a good got the good questions to ask and finding the best applications will come up on top and not people who have access to the largest models or even I think a lot of things could be achieved with the iterations of the models from a few years ago. From last year you don't need what came out today for most of your tasks and can have massive improvements if you ask the right questions at the same time. Even the best model one help you if you don't know exactly how to apply. I'm sure point of view they actually lack a lot so you can have like really use a lot to train fast like there is no real logic in them and you know when there is always a chance that the answer will be yes to questions that are very dangerous like you know that can harm somebody. It is just not the solution that can be really applied in the in important branches of technology. I'm looking from the point of view of chemistry but anything that's related to security or there is a great example of actually a couple of days ago a family from great Britain got food poisoning because they followed a book about mushrooms. Britain surprise surprise after 2023 and in that book there were actually suggestions obviously they were generated by LMS that suggested trying mushrooms to figure out if they're poison us and not and had conflicting images and so those people got food poison in God their life. That's just the quality of data since last year and it will continue. Yeah it's like I tried to get chat GPT to generate an image for me of a heptagon and it kept it kept giving me a hexagon. And then I had a conversation with it I'm like are you seriously telling me this has got seven sides and then GPT kept saying oh yes you know you're right I'm so sorry. He has another one and then boom six sides six sides six sides and and then I'm trying to not trying to be polite but at the end I was like WTF man. So then I think I posted about this in seven CTOs and then I went someone said I should go to Claude and I did it in Claude but then Claude was really determined to do it in SVG and like really coded for me. And all I wanted was a really soft beautiful background for zoom and and I thought I would maybe have a couple heptagons in there and boy that was a wasted two hours. But a funny experience right. Yeah and then someone I think Chris posted an ask chat GPT how many hours are there in set the gun. But yes what I was thinking was is it possible that humans. This comes through a hexagon for a head like humans just don't know what a heptagon is. And so then chat GPT went and learned all this stuff because on data that just thought that a heptagon and a hexagon was the same thing. And they also like misaligned you know that they are one after another and it read what's above or below that we deal a lot with PDF attachments and when PDF is attached and we need to read stuff off the PDF. LMS is that I could be just wild. Sometimes I would say fairly that those requests are for shipment are a little bit tricky. If you know very familiar with the format but the stuff that LLM comes up with looking at some of that is really you know there be a number unrelated to the number of packages. 735 and we'll have please ships 735 packages of that and like that was not near a number of pieces but okay I can see I guess how you got there but. So it's all about for us putting additional agents that verify and showing. So let's say one shot with say like this is how it looks like for this company and this is how many pieces they are to mitigate that so i'm curious about how you share and do that like the crazy hallucinations for your. Employees that you build yeah that's a good question so for us the way actually build another layer which is similar to what you mean by I think verification essential is like a constraint layer of like graphs that understands relationships so like hard relationships so it's not going to like attach background of Lena to background of. Etn by accident just because there's the approximate of each other right so I think one interesting point brought up is like I think you can build something very quickly with LLM nowadays but what you end up doing is actually building tons of things around it to make sure it actually would perform the function that you wanted to perform at the accuracy wanted to perform right. So like I think you would maybe spend more time on trying to parse the PDFs correctly or even PowerPoint correctly where maybe that wasn't really a challenge before so I do think maybe in a way we're spending just as much time trying to develop these applications because you have to build all the safeguards around it. So I don't know it's interesting definitely it seems to request much more actual pure engineering work just to figure out the rules and also I think that what emerging field I predict will be happening and it probably is already happening without being aware is kind of a. People who job is to break what you build because I can figure out hundreds of tests but then in production something something different will come up and because hallucinations are so non logical non intuitive. I think that would be almost a canard a person who really has a gut feeling as well as really good at saying how LLM's work to be able to find those issues and put the guard rails that Sharon mentioned and I really think that will be a big emerging field if I'm sure their company's popping out about that because that's the biggest blocker that's why everyone says keep a human in the loop and that's why I'm most successful apps are where human is in the loop and can catch things. Crazy automation on wheels because everything is happening faster but there's a person to catch an issue and that's not just going straight to production and making us look stupid. That's a that's a that's a really comforting thought so I wonder if the if the indeterministic or the imperative will be protected by the deterministic. So in the end we write unit tests for these very intelligent LLM well intelligent these very non deterministic outputs are still going to be crushed through a true and false deterministic test. and actually that's why I think that LLM's are not the final version of the model the model the god model did you see your face when you said the model. You know there's like there's a lot of fear of what will be you know the events when you know the age I happens when the perfect AI becomes live and takes over the world and like many people are worried that LLM's are going to be there. I mean I'm not worried about LLM's probably the next stage where LLM's are combined with this deterministic approach. And actually not just that's like there's there's just fundamentally wrong the way right now AI in LLM's operates where you cannot teach it's basic logic right away you know how you raise a kid and you think you tell something to the kid and kid makes sense. Like they're just not about that right so if you they're just about in the game of numbers and that's just going to work at the end of the day. Yeah it's actually interesting so like before I guess diffusion civil diffusion there was a generative of our zero networks so basically you have like kind of two players right there's a generative player which is kind of like generative AI right now and you have a discriminator which essentially. You try to trick the discriminator right so they're kind of like helping each other so sometimes the discriminator is like hey no this look like you're lying read just your output right and the generator or do something else and the discriminator would like kind of play that role right so right now I think. It feels like we have half of the christian is only have the generator and maybe the human right now is the discriminator human in the loop right but I think like ultimately there has to be something that like helps it to discern where there is like trying to be factual not being factual verify like something more automatic but yeah it definitely feels like we are only doing half. But in your in your companies and as you're building them you're not actually thinking about ag are you I mean it's it's not really about that it's just seeing the patterns in large amounts of data and having some sort of business value from that and you don't care if it's generalized. Are you ready to elevate your team's performance and your own career a CTO introducing the seven CTOs growth program seven CTOs you'll join a supportive community of tech leaders dedicated to helping you reach new heights. Our group coaching program led by experienced CTOs focuses on the proven CTO levels framework assess your team processes and technology challenges with our easy to complete assessments and then dive deep into monthly coaching calls where you'll get unstuck and gain insights on specific topics. Craft actionable plans tailored to your needs guided by expert advice and playbooks designed to drive results plus gain access to our exclusive CTO community for ongoing support and collaboration. Sign up now and start your group coaching adventure for only $300 per month or $3,000 per year your membership includes priority access to all seven CTOs events and activities. Don't miss out on the opportunity to join a global community of experience CTOs attend virtual events and participate in confidential group coaching sessions. Visit seven CTOs dot com slash growth and level up your leadership skills today. I was just thinking that not many people do care about that right now because as Lena was saying earlier, it's all about figuring out the task that you can solve and hopefully that will improve your margin of profit or maybe it's your entire offering. Yeah, and sorry to just one second, adding up to senior so the problem will happen when we say all right, the system is setting its tasks, then we're in trouble. So those tasks when they are not controlled and basically generated by the system itself. I can actually drive the unpredictable results, but definitely not in the chemistry, I'm not going to let you know the chemical system to okay, now I'm going to make something and I will develop some new chemical ingredients with the quality that I want to have. Yeah, because to Sharon's point, the discriminator isn't there to say, hey, you're building a Frankenstein, just take a few steps back. I mean, ultimately, I am concerned about a GI just like everyone else, I don't think it will come from companies building specific case. I think it will come from bad agents who will train the model and a fact that it does, I mean, it has been shown it is possible to at a certain extent have an LLM create task full with task and create new tasks that's possible and if it can be self sufficient in a way to finance itself, which I think for connecting to internet, it is possible. I think you can definitely make money of the internet and you can do a lot of harm potentially from affecting elections or including malware into people's software or stealing identities. There's taking over those tentacles. Yeah, did you say floppy disk. Well, I'm saying, luckily, sounds like all a nuclear power plants software run on floppy disk to be harder to take over. So that works for us, but yes. If you think about the multimodal stuff where you you really are having conversations with the LLM to for instance, you know, write you some text, generate an image, write you some code when those tentacles start going into create 10,000 social accounts make them all seem like real people and have them all vote for someone. Now you have those tentacles and then you have like you said the tentacles the floppy disks like whatever, you know, those different IO interfaces. It just seems like a kid that plays with fire like you just cannot tell a kid to not play with fire like the human species. It's too curious and too driven to not actually try and plug this thing into. You know, everything, yeah, but it does sound like that's where a genetic networks are going to be honest. I don't know if you guys I haven't tried Devon AI, but them and it's like a coding, yeah, essentially it's like a autonomous coding agent. So it can do like take a ticket from get up and then plan the steps and try to solve and debug it. Right, so of course that's like one innocuous coding task, but I can only see what Cassandra saying going array applied to other things like crashing the financial market quite easy to crash your financial market actually, especially with the trading nowadays. I think we made a few of those mistakes before right so it is kind of interesting but I feel like there is with disconnect sometimes in the like central valley versus maybe governance is the cost the right word. Because there's like advancement of technology and then there's like safe of unspeak technology I guess I don't know if you know the opening I kind of blow up has some sense of like seeing that split between like the business side and how do we make most money and how do we strive for AGI versus like I think I just raise you know one billion dollars to do. Super safe super intelligence or something like that right, but they're definitely seem to be have like that division happening. I don't know it's interesting to see part of me feels like and I don't know the inside story but feels like open AI is what happened when AI hits it like you have the purest of intentions you. You have your charter you you you try and govern the nonprofit in a certain way and then magic happens and then it just obliterates relationships and kills everything. And so I think the magic is going to happen and we're just going to be like what you know. A couple things that you touched on is Lina that I pasted in the chat the Arc prize that's what I was referring to. So the co-founders of Zapier and box I think. Yeah. So they basically was saying this and if a five year old like you said Lina if a five year old can acquire new skills like you just they just acquire new skills. You have AI models that just acquire new skills you know a kid doesn't have to look at 50,000 videos of how to cut a piece of paper with scissors to know how to do that. And so memorization of course is the memorization problem so it's like hey Tesla is learning how to drive by showing its models bazillion videos from multiple angles on how to drive. And yet when I learned how to drive I didn't have I didn't watch a bunch of videos. So the arc prize I believe is trying to say you know abstraction and reasoning corpus can we can we move towards a skills acquisition type situation which I thought was interesting and they've given I think the $10 million prize and they have all these little competitions and tests that that supposedly people you know can do we can do no problem but I can't do AI. And I guess it's a bit of a problem also in how AI is you know built we're trying to reproduce a human right. Sharon is Sharon's creating them right like we're trying to reproduce like Sharon is reproducing basically talking right like the communication. Some of this is kind of more complex than that right so there is vision and there are some other balance like other parts of the brain that work together and understanding text is just one part of our brain like image recognition is another part of the brain. And we're trying to just take one part of the brain reproduce it and say okay this is human like this is this is the right it's just not not that. Everyone was so excited but didn't it kind of under deliver when people started testing it I think it was my memory is that it was like greater prototyping and creating like little new applications. Giving it code base just could not comprehend the larger contacts kind of like you can think of it as a very junior developer who comes in gets overwhelmed but a talented junior developer probably can write a few functions really well can prototype can plan for things to do but it's the skill of being able to understand very large context and all the relationships and have common sense and the experiences. And the question is that I have is the temporary is that just inherent in ability of the eyes to do that or is it just like the conducts when those are small or the computation powers to small and we'll get there and I don't know myself if I if the answer is one we kind of I feel like people fear that eventually will be there but will we because yeah we're they're not humans. Data is also the problem right they took the code from stack overflow with all that like bad examples. And there is a lot of code that can be optimized you know I would have loops in it and that thing was basically built on this and at the end of the day when you already put that into the core. How many good code samples you need to put to actually fix that and where you're going to get it so I don't know. Yeah and and and being being a human or being a human means that you write bad code so who's going to model that. Who's going to model a bad mood. I mean are we saying we want AGI to be the perfect expression of a absolutely perfect human. Or are we going to have a new contributes to company success exactly does it I think so I think you can say something that is a momentary. Indiscretion at statement that seeds another brain to think other thoughts and then you know chaos it's chaos theory like you. I don't know how you model that stuff like if I have an allergic reaction to chocolate milk. And I have to stay at home that day and then because I stayed at home the other team members did something amazing because I wasn't there to interrupt them and so it's just it's silly but it's that keeps me up at night. We're not saying want to have humans right like Sharon are your employees exempt beautiful amazing people. What are they. I think the question always like wonder is are we do we really need to build something like a AI that's close to humans. I know that's always been the goal is to turn tests and all that stuff but like why would you want to do that like why would you want to build something that humans are really good at. Shouldn't you want to build something that like humans are bad at right so like for me is like you know if you look at the Eisenhower matrix you know in my not in toward a not agent stuff like origin stuff I will love that to be done by like AI right. But the stuff I love like coding or contacting things like that I don't want the idea to replace that right so I think sometimes like I'm not sure the goal is really like the most useful thing for society is to build something that. The trying to trying to get the agents to just do the things we don't like is it going to actually be part of the human experience that makes us better or is it just going to make us lazy or is it just going to make us. Is it going to is it is the general intelligence going to just go down or is it are we going to build on it and it's going to go up maybe this is getting a. I think for me is like you want humans to do the more complex reasoning complex tasks right so maybe that's not like reading through 600 pages of a PDF right but maybe it is for judges or lawyer to make the best decision for the plenty for the defense right so like you want them to spend time on that. What you want doctor to spend time on diagnosing the patient and treating the patient and not reading like 500 pages of medical history right so I think like those are the things I think we should be spending time on but there is definitely especially people who are like really like all of you. You know have me no task that you have to do right like there's no choice especially in startup but you know I rather like I to do that part so that we can be freed up and do something else and create something else because I think creativity like true creativity itself I think at least at this moment is still reserved for you. I worry that it would divide people even more in terms of how we already have seen effects of technology and social media on people who are creative who strive to achieve or eager to learn have reached unbelievable success even potentially without being able to go to university or do something like that. That just due to technology and internet existing while a lot of people you can talk to teachers you know like the youth right now a lot of kids just they can barely read beyond their text and tiktoks kids in high school are so behind it start happening before covid so definitely attributed more to the technology I think this would be that but more this people who want to create want to learn want to reason. It can just delegate to AI their tasks that I want to do but plenty of people will be completely satisfied not going there and now they don't even need to do the bare minimum tasks and their bare minimum learning and that to be honest scares me and I have good answers for that especially as someone who does have children that's constantly on my mind. I think you're completely correct like the separation will be way more visible and hard for once for actually getting the capabilities of AI and technology but they're not really wanting to do anything and I mean most of the kids are like that. You know there are some kids who are willing to learn and like it's great I mean they exist but then there is like the majority of kids who are actually you know they need to watch them to make their homework you need to make watch them that they're learning right maintain this all the time and that's actually the majority and that majority figuring out that there is the AI that can write and the safe for them they will just you know. Do that and they will never try to do anything else just because there will save time and then they go and play their games on the phone or you know other things so yeah I completely agree with senior it's actually quite disturbing for me as a parent to. Yeah I think the the thing that won't change I don't know evolutionarily at me but the thing that won't change in our lifetime is the way the brain does reward like dopamine levels and and so if you are in a if you are in a depleted state you know you're not going to be satisfied by much you know like for instance that's why you're sort of addicted to. Your game on the phone because in a way your dopamine levels pool has depleted to the point where really it just wants the one thing and then if it's if it's a pool that is overflowing I mean there's a lot of science around this but. The thing I'm thinking is that maybe as we go into Sharon's world will just learn a different way of reward you know so it's maybe for our generation. Of human with our school system our social systems our our growth we have been programmed to be rewarded certain ways and I think for our children maybe that reward looks different. The evolution has to happen so quickly I can years your describing and an evolutionary step in humanity and well actually with the with the dopamine not really because you can you can fill up the dopamine tank with different things you can reset your dopamine you can. There's lots of things you can do like for instance the whole called plunge thing a lot of that stuff has got to do with resetting dopamine levels and all that I'm just thinking that right now we have a context window that's maybe 15 20 years of well when I showed up to work my boss was happy and I am thrilled with that and so now your brain learns to to be conscientious and be having integrity and show up to work. I can't apply that to my child who's never going to go to work is going to just work from wherever they are so but but the dopamine level is going to you know right now my son and my kids dopamine levels are being programmed by solving puzzles so so so so so so. Coordinate coordinate coordinate talk talk talk move move move move move you know these are levels of training that I never did and so my my reward system is not oriented that way my reward system is did you climb a tree outside in the sunshine. My son is why on earth would I go outside climb a tree. That's really interesting yeah I mean I think that education system. Probably definitely have to adapt. Last night I was trying last night I was driving home from a pink concert did you guys go to the pink concert I wish. I took my two girls to the pink concert and in the Uber there was that little game on the back of the car and you know I looked at her and I was horrified. She's she's 11 years old. Just the things that she knew at that age was just I just couldn't believe it you know but but she see. There were a couple things that I was listening to her where where she would read out loud the question but she was clearly missing. Like if it was if it was if the number was 111 she would say 118 or. And she's generally has never had this problem she's a ferocious reader she's she does very well at school. And noticed that she would watch what was on the screen and what her brain was reading back was just off by a word or a number sounds like an LL. But but then she turns to me she turns to me and she says to me dad I want you to know something when I went into fifth grade. So I scored the highest levels on my reading when I came out of fifth grade my reading deteriorated. I was like that is fascinating observation. I don't know why I'm saying this somehow feel relevant. I have any explanation for that but basically what is happening is some like unlearning rates. And I wonder if it's a cycle of human brain. Yeah like like there's a growth spurt or something I went I went straight to hey you're not reading enough. She said she said that I read every single night and I have a whole stack of books next to my bed if you'll just pay attention. I think it's that is I would write it off as those completely imperfect systems. I have an 11 year old and yeah her school think tells me that she reads at a level of a university student and I'm like no she does not. I would not put any trust in those achieve 3000 or whatever they use for measuring kids reading abilities. Just give I guess like a positive no I think like you know few years right like everyone in the world should be able to talk to everyone else. There's no more like language barriers because your a eyes can just translate right which is pretty cool. I would love to talk to people at different places. I have a question about that and I actually wanted my plan was to write an article about after thoroughly researching. I think with English what will happen to English in the next 5 to 10 years because language is having evolving just due to humans interacting with each other written spoken word people academics creating dictionaries. So we have ala lens that are trained on the prior human language and yet in many many cases now the vast majority of the text we're seeing are written by ala lens. So how does that work and I really wanted to talk to linguists and philologists and people who really understand that what could and what they do to just even one specific language in terms of its development. Because now you just have you know it was trained until here and then the vast majority of big chunk of the new text is still generated by that. You know just that it affects and spoken language sometimes when you use a lot of chat GPT I can see people even start forming those same sentences is it just such a common response. So what does it do to a language are we going to lose languages especially in those countries where people maybe rely a lot on things like chat GPT that's a curiosity type of thing that I had and I wanted to reach out to a bunch of linguists. I never got around it I still might do that someone might have done it I'm sure this research being done but I do wonders specifically for language as a leaving breathing and evolving entity what happens now. I think that some words would definitely will be use more than others according to statistics right so that will definitely happen and yes some some phrases some structures will be repeated more and that will definitely affect again kids the way we discussed right where like they will you know just right using chat GPT. That that through that channel will make them learn those phrases even more so guess your corrects and this will change how they will according to statistics just what you said it just means that language will become poor right more poor because you frequent your words will become more frequent in frequent words become even more in frequent that means that they slowly will die and that will be reinforced every time the models are retrained. How do we know that when the new model and one of the foundational models is retreat. They do not scan for making sure that language model output does not make it into the input so I feel like it's creating this amplification effect that will just make languages simpler and hence will become stupider as a nation. Because I think it definitely was a dominated by English but eventually I'm sure that it is quite as a summary of our call. Thank you good bye. Did we all get to get a little down there on this call just a little bit. Is everything I say I quickly type in chat GPT and just read as opposed to over in original. I mean chat GPT just told me to do a little smile so like just the level one wants me to stop holding my face. I am noticing that chat GPT is training me to use less words. I'm expecting it to just no more so I'm starting to just say instead of saying hey could you please just do some research on Jan Le Koon and what he said about the LLM being an off ramp to age you know. I just said Jan Le Koon off ramp enter and chat GPT gave me a beautiful essay on what Jan Le Koon said. Just on the basis of the core words I just said those core words so what I'm noticing is I used to be super willing to chat with chat GPT and now I just wanted to know what I mean. Interesting that this evolution also happened like in a year right. Exactly and also that training is happening both ways right we are being trained by the by the machine. I think I constantly have to tell when I use chat GPT is like please be last for both of your words, not even pure now cut it 50%. Please stop being so freaking kind just be mean. Oh, have a great talking to an uncensored LLM. Oh it's such so much fun. It's very quickly in your day to day this is a bit of a matter question but how do you feel about your employees responding to you with something that they've clearly chat GPT. I don't ask such questions. I think the questions that I ask are very I mean the power to them are they tricking me but they're very specific and they'll be much faster just type of answer them. Are you a philosophical question or historic facts it just like have you finished or do you think what your thoughts on this or how do you think of that. Well for instance if you say to if you say to your developers hey man let's get come up with a plan for reducing technical debt in this next quarter. Like what how should we go about it and then you know sometimes you see wow they really just went to chat GPT. Got a couple ideas and then maybe they added some of their own words took it out the item the bullet points and just pasted that into. For tech for tech that plan I would expect to be so specific that they would have to upload entire code base into chat GPT and then ask questions and more power to them if they did that because. Oh you're right so that's actually a good cool thing that they did. If that is make very relevant and make sense I don't care how they can on so in your in your let's say over the last 30 days how many times if you communicate with someone through email i'm not talking about colleagues necessarily but just in general. How many times if you communicated with someone where the response back from that person was that they generated something from an lm to respond to you. Are you seeing any of that. Not in my communications I think would just like seeing either so specific and you know direct. It just makes no sense to use chat GPT. I guess it will be more of a problem for areas where this is an ambiguity like you know marketing. And you're talking to very technical people we talk numbers and facts. No i'm not joking basically at the email that comes into the inbox gets drafted with the response for him like automatically so sometimes he just send it I mean it was like very relevant right so I definitely get that. It's a bit but it's from his personal eyes now from his chat GPT so I think it's more relevant but I also I don't feel like offended by people using chat GPT or something else. I think as long as to what you know because then you're saying make it relevant to the question that's being asked like feel free right like you can go on google and search for something on copy and paste like research that you have done that's okay. It's okay for me but like if clearly is like the I'm pretty sure so I got our talk to her today I feel like the auditors may have generated some stuff from chat GPT because we definitely don't use those things. So that makes me a little bit ticked right like obviously it should be a fact that they should have checked and verified but I feel they did not take that right. So that makes me a bit ticked so if my employee was doing that like that would be not acceptable but other than that I think yeah there's like a group. Congrats on the talk too man. Oh thank you. That's huge. I think in my world I'm coaching a lot of CEOs and CTOs and C-Sweets and sometimes in the coaching session I would say hey let's just come up with a little plan for how you're going to do performance reviews for your engineers. And then they will email me something later that day and clearly they went to chat GPT and said if you had to answer some do first coach on how to do a performance review that from South Africa. They don't you know just how can you convince them that I know how to do performance reviews and then you know it draws a little lion. Is that different than let's say five years ago you asked the same question and someone found a template online of how to do performance review and they mostly copy pasted with a tiny bit of editing. Would you feel as bad then. That to me feels like a little more research. Why does that feel like more research and this feels a little more lazy doesn't have to be it could be as bogus of a source with mistakes and horrible advice you can't tell right so. I think that you can use any tool in many ways you can go through many sources yourself find really reliable one use common sense and the same with chat GPT you can say like more actually here you're not making sense improve iterated maybe they spend two hours going back and forth with chat GPT. And until they got to something that I think I talked to a friend of mine who's a community college English teacher and I said like what did you do now with chat GPT she's like I don't care as long as it's good most of it is such crap that I really don't care how they wrote it. I'm just wanting to see quality essays and it's really resonated with me like yeah I mean that's ultimately what we want you know that doesn't doesn't make me happy at all. There I remember I think there was a professor who they never banned chat GPT I know a lot of universities does but basically they said hey you can use chat GPT just submit your entire prom history with chat GPT right so yeah way is still doing research right so if your prom history is really like two hours long with you chatting with chat GPT trying to get the best out of it right that's still homework and offer. Yes effort so very new approach to education I must admit. And then I put the prom history into chat GPT and say do you think this is did they do research or not. Well I can give an example I write poetry it's bad I don't want to share I write it without chat GPT but sometimes when I do share it with friends I want to title it. So I put my poem in there and I say like come up with 20 titles usually I don't like any but like out of these three you kind of on the right way but let's keep iterating would you then accuse me of cheating if like I spend another half hour assisted with chat GPT coming up with the title that captures it but the poems created by me so then where does it put me. Oh you're you're opening like a huge box of war I would have to I would have to hear the poem I would have to hear the poem can you can you read the poem now okay okay now that that is difficult and again I think maybe this is a generational thing where for me I haven't been trained on that that's a good thing I've been trained on the other thing. How much you know how much you've applied from yourself like what did you come up with the same analogy with calculators right I laughed at my parents who are like you cannot use a calculator in school and I'm like that's the dumbest thing I've ever heard so I have just been trained on calculators and computers are cool and now my kids are going to be trained on this machine in space that just knows everything. But unless you ban it and ban any black market of it it's kind of an avoidable and I have bad feelings about it too but I just do not see a future where we just no never mind we're not going to use it no one's going to use it and if you or kids or my kids I'll be specific to my kids are going to enter a career and I trained them up to use that and then they'll be competing in the market with people who are successfully using that. Which is the human experience I agree with you I do agree with you and I do trust the dope that the the reptilian brain which is simply fight or flight does this make me happy does this make me sad we have such simple in the end such simple outputs that I do believe that that can be reprogrammed where what satisfied my grandparents. Does not satisfy me will not satisfy my children but what but they will still be those four signals that will satisfy them somehow so I trust that the brain chemistry is going to adapt because that's just that's how we've evolved as a in our innovation and technology I'm fine with that. I just want to be a cool dad while this happens thank you touching cereals here's here one of them is what is going to be a job right in future and how all jobs will evolve and how whatever we're doing right now and is in some way automatable will be automated to the extent where this human at the end will be checking out so I'm pretty sure that humans to be there until humanity is around and they still very find those answers and so like checking but I completely agree with senior you need to learn how to use these tools otherwise you'll be just out of the job market very soon and companies I think that the trend of not growing company but figuring out how to use technologies instead of using more people is going to be bigger trend within the next couple of years so you know a lot of people are actually losing jobs in like in all environments rates and new job places might not be actually created for you know for the roles that they are the way they are now today they will be created for the people who know how to use AI and move way faster with their day to day itself. Yeah, I have a few more questions but as we wrap up Sharon what are your employees your AI employees names. The first one was Andy so she was all of our customer sales and she knows all the zoom transcripts like actually anything about every customer right now so she automatically generates like contracts for customers. The other one that's like more comments gates so he's our fundraising expert so like we actually just give VCs gates access. They just ask questions to gates and we don't get involved anymore. Do you tell gates what not to say? Yes, so if you ask gates very specifically about very specific numbers it will be like hey you know you should talk to RCO and then we have a few more internal ones like we have one for compliance so that people understand what the compliance measure some policies say and then we have one for like branding and social. Fascinating and are they accessible? Do they live in do they live in a prompt slash slack slash something. Yeah, our platform is like a slack. Okay, okay, so they live in there. Yeah, very interesting. Are they do they have needs? Are they fun? They're fun. We recently have a new one which you guys will see on the website soon. So he and so she does means like as well. Now will Andy will Andy reach out to you when she needs something. No, so that's something we're probably going to be working on. Yeah, I mean that's that's the that's the LLM problem right now right the the agent problem. Just very quickly since this audience is mostly CTOs can maybe each of you can just sort of end off by some word of advice or some something that just comes to mind for you immediately when I say like what what should CTOs. How should they be right now around specifically AI. Is there just anything that you would advise like I like I love Cassinius advice a few months ago like the do's and don't sort of adopting LLM's like don't just rush to it. The other thing I learned from Cassinius that you can really go with the simple things first you don't have to go and like conquer the mountain with very intricate and complex solutions. Are there a couple things that come to mind for you at all any of you. That's the same thing that I said before kind of what you summarize but if you are new and you aren't sure and you concerned think of a small a smallish project to generate momentum think of something that could be really impactful even if it's small don't think large. Talk to someone consult to someone and see where a eyes or LLM scan really fixed a problem and what it does even if it's very small. If you start getting momentum you get buy in in the company you prove a concept you get intuition about how it works you start educating others and you kind of get a general directions of a sense where to go from there but a very specific and relatively small thing to start with I think it's perfect and it's always been my advice. Start there just challenge yourself to think of and don't care if it's really small I think like that's where people like for this. Here if it's successful that's going to be your foundation and you do it quicker too. I'm going to be afraid to just chat with someone and ask questions and see throw a few ideas at them and see what they say. I love that. Yeah so from my perspective it's important to remember that yeah itself is not a product and you always need to think about how to wrap it is make it usable. Otherwise you will end up with great technology but you know people just don't know how to use it. I think it's very obvious that right now everything is evolving very fast and so whatever stack you're using today just don't get emotionally attached because most probably tomorrow it's going to be too late and like you you should start changing it today. Yeah so basically try to continue evolving asking questions right questions like basically if you can ask the right question you will get the right answer. So it's mostly related to building the products and what is saying I was saying it's all about asking right. I love it Sharon think I have the same answers are well not to think a new one but I definitely agree I think like start small but impactful just for learning experience and how setting that first project. I think the second I will give is like learn the weaknesses of elements and you the technology really because I think there is always like operating space for all of these different type of technology. So really understand like where they fail I think right in and not just using them for everything right so I think that's something that perhaps like not everyone is really aware of. I think people vaguely understand like maybe knows what hallucination means but it is a big deal right like for example to be for I think the benchmark was like 2.4% hallucination but in legal is like 69% right so things like that I think really like understand how it applies to your domain. I understand whether it's a good fit us with all technology is really love it yes and what what was that thing you called it the discriminator. Was it the discriminator now the what. That's something I learned was great okay let me do a quick thank you thank you everybody thanks for coming on. That's the show check out CTO POD dot com stay connected we love hearing from CTOs we love hearing from CEOs anybody who needs to get the CTO plugged in check out seven CTOs dot com. There are membership levels for everybody so it's never too late to expand your network nurture your relationships and please let's see each other soon like next week cheers. [Music]