Go back

How AI Will Change Photography Forever

51m 41s

How AI Will Change Photography Forever

The transcription discusses the fleeting nature of capturing moments in photography and the evolution of computational photography from hardware-based to software-based models. It delves into the concept of personalized visual genealogy at Photo Lab, aiming to create images that reflect the individual's life experiences. The impact of generative models in generating compelling and realistic images is highlighted, with a focus on bridging the gap between capturing moments and enhancing them through technology. The conversation also touches on the future of photography, discussing the shift towards generative models and the merging of photography and graphics in the digital age.

Transcription

9566 Words, 51865 Characters

The critical moment, the decisive moment in photography, which is, you know, this thing that as photographers we seek to capture, that moment in space and time is so fleeting. A moment is no longer fixed when the shot is pressed. It's a very interesting like a new dimension of what photography like is. If you believe that premise, which I deeply do, then we are the land shift moment for imaging of photography. What if you could retake your favorite memories, fix the lighting, catch the smile, or even open your eyes years after the moment past. In this episode, A16D Journal Partner, Martín Casado, and partner Yoko Li sit down with scientists and Lightro Founder Red and Ing, plus photo labs co-founders Cecilia Zhang and Zach Shao to trace the wild arc of computational photography. From Stanford camera arrays and lightfield refocus to today's personalized generative retakes. Together, the unpack why identity and context matter more than filters and ask whether the future of photography even needs a lens. Let's get into it. I'm super excited to have the group here together because we share a lot of personal history and then chatting about evolution of computational photography. One fun fact is Cecilia and we went to college together and we actually know each other for more than 15 years. I kind of watched her working a first day of college. I don't know if you're remembered. You kind of carried a camera. I'm watching you. And I just so great to kind of see you turning that into like a top-notch research career too. Adobe and where you met Zach. Another connection here is Cecilia that her PhD was Ren, who was the founder and CEO of Lightro camera, which is a lightfield camera, very revolutionary. And another fun fact is Ren and Martín 20 years ago, 20 years ago, 15 years ago, maybe a 20 years ago. We were actually grad school classmates and when did grad schools together. We were actually portfolio founders in the first fund of Andrewson Horowitz. Wow. And I was his reference. You know this Ren guy. The best entrepreneur ever knew so to say something nice about me. So you know, Ben decided to take a look. Yeah, maybe the right place to start is kind of turning back to history, like thinking about historically how the computational photography has the field evolve. How did you come up with Lightro? Like the story behind us. Founding Lightro 2. And I'll let you start Ren. Exciting topic. Exciting times. I think for computational photography, photography at large. And I'd say that I think computational photography has been in the mix, you know, at the since the beginning of digital photography, right? I mean, you can't have a digital picture without doing an immense amount of processing and very basic things or things we would think of, very basic like making it make the right colors involves, you know, some of the most studied algorithms early on in computational imaging. And so I'd say there's been a ton of computational imaging where I arrived on the scene was that I was really into photography. Like Cecilia, I carried a camera. Actually, I got into photography because all my best friends were rock climbers. And one of them was really into photography and he said, oh, you should, you know, like try my camera. And it's a big climb at the top of a root and then try to take pictures of the nixers and climbing up and make ourselves look cool, you know, so that was my introduction of photography. And I got hooked. And then I decided for my PhD that the stuff I was working on, which I thought was great, was not going to go where I wanted to go. And I was like, I want to do photography stuff. So I tried to marry that stuff, which is computer graphics with photography. And so for me, it was really, you know, an attempt to bring more computation into photography by bringing all of graphics, how we make pictures with the computer into the core of the question of like, you know, how do we, how are we going to do photographic imaging? And so, and that was the start for. And Marti, you were there when we made the investment to your lightro and what was it like for computation of photography back then? So I'm a systems guy. And so here's what I remember. What I remember is you'd go into gates and like, because Mark LaVoy had this camera, which is not one camera, but a whole bunch of camera. 100 cameras in an array. 100 cameras in array. And then it created a, I was like, I think a gigapixel image and like we actually blew it up and we like put it on the wall. And you could see very little details of like the, the quad. And so from there, I think now like you can just have this in a camera, you know, like that you get off the shelf or a lot of features of that. Yes, I mean, a lot of a lot of ways I think that kind of computational photography is accessible now. But my entry point into life view photography started from there that array was built by Ben Wilburton, right? Who did this PhD around our time too? And I said, well, look at that. I was like, what this is really cool and it's really complicated system to build. I think it'd be great if it was inside my camera. So I can use it. And miniaturizing that, you know, 100 cameras integrate into the body of a single camera was essentially my entry point into life view photography. And then a long time later, Ben it, you know, worked at Lightrow, cave on fat alien. Also, did you work at Lightroom? He was at an IEEE for a year before you went out to start his presentation. I studied with Qual for calls. He taught me the graphic stuff. I taught him the system stuff. He line is one of my favorite people in the world. You know, we we roomed together in grad school. He was the best man in my wedding. So lots of, lots of connections. Critically too, Benji came. He was also finishing his PhD around that time too. He came back to finish his PhD after, you know, doing that whole SGI thing and being the most engineering company in the valley decided to finish his PhD. So we're all there around the same time. You may want to describe actually what Lightrow did. Yeah. I remember when you first showed it to me, it just absolutely blew my mind. So I just a lot of fun because it started with a very simple premise, which was at the time. I was like, well, if you try to take pictures of your friends climbing up a climbing road and you're hanging on the end of a row, it is very hard to hold stable enough to focus accurately. So a lot of pictures, if you're doing that kind of photography or out of focus. So I was like, you know, do you have to focus a picture before you take it because in computer graphics, with ray tracing in particular, we taught a class where we have students simulate a camera system, the optics of a camera system and make them synthesize photographs, focus at different depths. So if you can do ray tracing, it's sort of very core, you know, instinctually that you can focus after the fact. So the idea is to bring graphics into the body of the camera. You have to capture more data. You capture the rays of light in a photograph instead of just the pixels in a photograph. And then you apply ray tracing, computer graphics ray tracing in order to synthesize photographs after the fact and can focus in different ways. But once you got that and we have this higher dimensional data set we call life field. So life field photography. Once you've got that, you can do much richer computational photography and turn significant parts of the physical work of the lens and the camera into pure software. So when I map the camera system onto more is law and more. That's amazing. Yeah, from that point on, like how did it evolve? Like how did the computation of photography as the field evolve from like hardware based to more software from camera to phone or yeah, we'll have to hear more. I would say the main theme I would say is like mass speciation. It has gone in so many rich different ways. Computational photography has enriched our lives. We're all rich and really with it because in our phones, you know, for many, many generations of smartphones, the core marketing, right, decisive marketing feature set was the camera. Oh, it's got this new feature. It's got that new feature sometimes. It says a bigger zoom. But oftentimes it had additional features like portrait mode or cinematic mode or you know high-speed stuff. We were probably lumping their two for for competition photography HDR, which is like it's so standard now. They don't even tell you if they should do it anymore because every photo is HDR, right? Burst photography, you know, under the hood, all these kinds of things. But it's not just in photography. It's in scientific fields, black hole imaging, you know, taking this data that really doesn't look like an image at all from the event horizon telescope. This interferometric data and then figure out how to solve the problem like what does a black hole look like that would produce this kind of data. We think of that as computational imaging, I guess, as well. So speciation. So many different things have happened. Yeah. So I guess like tie it back to, you know, pulling on the personal side of the story, you and Cecilia met when Cecilia was in grad school and you all worked on so many things together. How did the idea of photo lab came to be? How did you find it, you know, rinse lab under rinse supervision? Well, I think I just want to add more, one more thing to light roll. I think the, there's a mental model that makes life very, like, looks really visionary at a time is because the couples that capture from, like, what you get at the output. So it's like, moment is no longer fixed when the shutter is pressed is like, can be tweaked afterwards. Like, so it's a, it's a very interesting, like, a new dimension of, like, how, what, even like, what a photography, like, is. Right. So I think that kind of a definitely effect, like, influence that what we are doing at photo lab is that I think this involvement from, like, focusing a lot on cameras and hardware and now to maybe software, like, just for image manipulation and editing. It also inspired us, like, oh, what we can do with, like, this era of, like, Genie I, where there's like, a lot of, like, yeah, like, generative models are becoming really, really good. Like, they can just produce compelling images. They, a lot of time, like, those images are just perfect in composition lighting and everything. That's like, fundamentally, difficult to do in photography. Like, it's a, so when Zach and I were, when we were, when we were at Adobe, like, we worked on, we were editing towards like Photoshop Lightroom. And also, we touched like, computational camera, like, building a capture, capture software as well. But, you know, like, editing, capture, we just realized people are still struggling with, like, capturing what they meant to. Because a lot of the issues are not really in the, how you edit afterwards, or like, how, what's the, whether you have the best, best camera in the world is really, when you have the camera, and when you want to capture some moment, like, how do you do that? I think it's really, like, just evolving from hardware to software to Genie I, it's really, we're just using a different tool to ask the question of what, how we can capture this moment in the best way possible. So, I can see the connections between, like, all these developments. And I just think we kind of fully, but Genie, I could be the next shift in this area. Yeah. Just from, from my education, is there still a big distinction between, like, photography and graphics? I think so. Is there? Yeah. Yeah. I mean, graphics is a, you know, a very broad term. But, I'd say it's sort of making pictures with a computer, but usually pictures that are almost like not taking with a camera. So, if you did a photo-real generation using a foundation model that, like, looks something, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, and then there was like a 3D engine and they were different. But like, now I can, yeah, no, I think you're asking the right question, which is that I don't think today that people would agree with a statement yet that photography is graphics in a generative sense. Okay. Okay. I think many people feel rightly that, you know, this is in the air, you know, and there's a lot of, you know, complex ways to feel about that or think about that. But to me, it's just really clear, you know, that the future of photography is going to be heavily centered in, if not, the core of it's going to be a generation. Technical distinction between generating a realistic photo and generating a non-one. There's, there's, honestly, this is just really clear. Yeah, yeah. To me, like, I feel them very similar. Yeah, there's a comment I saw on Reddit, which is like, I like it a lot. He says that, you know, if something is purely created from someone sitting in front of a keyboard, I wouldn't call that a photo of a photography. But if something is created from something based in reality, like, so you have something where there's a light world camera or just a smartphone camera capture some information about what happened there the moment. And then from that, you create a beautiful representation of that thing. That's what I was talking about. So in the past, photography was like optics in a camera and computer graphics was a computer. And so the technical stack was entirely different. But your definition right now has nothing to do with technology stack. It's like the use of it. But it seems to me like more and more of the technology is aligned now, right? Is that fair? Yeah, it is fair. And I think from a technology perspective, it's like, you still need the optics part just so that you can capture information about what's there, right? But then a lot of the thing happens when you're in front of the keyboard to create that image or that photo. Yeah, I really, sorry, somebody like you, I really like this definition. I think the future photography is going to include a lot of stuff that is actually just a keyboard or not even a keyboard that doesn't have optics that's capturing it directly. And I think that's going to be very meaningful too. I think for things that people will want to care about looking at, I think how we think of photography is often very intimate. Things we care about, people we care about, shown to other people that we care about. And so I think the person that's creating the photograph there and their thoughts and intentions in the generation of that image will still remain at the core of a lot of the most authentic and meaningful photography, but I'm not sure if it's necessary for the lens to be there in the future, myself. Yeah, on that philosophical question, which by the way, I love this philosophy. I feel what is photography? Yeah, yeah, I don't know if you want to go there, but how does that feed into what you're building at photo labs? At photo labs, we are bringing the visual genealogy into everyday photography for everyone. So I think the key here is everyday photography, and also everyone. People are taking photo all the time. And so many times, we just find a lot of things that just go wrong. It could be a bad lighting, it could be an awkward angle, it could be a missed expression. So we just have this trouble, frustration of not capturing what we really, really want it. So what we are building at photo labs is to bring personalized visual genealogy into your, like, basically, it understands you and tries to understand you and your life, such that it create and understands like photos or like images. If we do not want to call it photos, because it's generation, like generated contents that actually reflect you and your life. So it's, we kind of see this as, so we can do this because we, we have, we're able to personalize these AI models and existing foundation models, like they are built to be generic and a bit very powerful, but they are just, they do not understand personal content. So that's why, like, they can generate a compelling photograph, but like compelling image, but completely hallucinated, it's not relevant to what you experienced. So we're trying to bring that gap. This totally talking to the fourth law, but this is where we can, you know, show example of what it's like to have the retake. And Marti was actually the first test subject. What are the reactions based on, you know, the beta product you already released? Like, what was the first reaction? Like you had when you saw it, and then curious about your, you know, other user reactions? So, I mean, just, just to like, like, the foundation of this, I've worked with a lot of generative models, right? So I was a very early user of mid-chirney, I'm on the board of ideogram, like, of course, when investors in BFL. And so like, I know what it's like to do character consistency, where you take your image and you put it in, and then like, you generate something using a traditional model, like this was entirely different. Like, I can always tell when I use, like, ideogram and consistency, which is very, very good. Like, it's a likeness of me, or if I, you know, before I was like, you know, I have like some open source, you work flow, where I do it with a bunch of images of myself, and it kind of looked like me, but it was, like, there's almost like this, like, style, you know, that came from the images. It was almost kind of felt like art. When I saw the output of this, I'm like, "Hey, I'm old. That is totally me." I mean, it was like, it looked like a photograph. So it actually, it actually feels different than standard generative models. It doesn't feel like art. It felt like a photograph. Yeah. So I guess like, what, what did it take to make this? Like, a question for both of you, like, what's the technology unlock here? That made it possible that, you know, something like a foundational model couldn't do, which we use all day every day. So for me, like, I have this anecdote of like, the first time, the first time I actually know Cecilia a little bit more is when I was looking at her, the recording of her PhD defense. And I remember in that defense, you really shared a story about how Cecilia's photography skills has grown from, like, not so good to very good, during her PhD. I just knew where that story. And I was the only thing that come into it that they just barely have made. Cecilia's an amazing photographer. Just for the record. And then I was like, "Why did my advisor tell me that during my PhD?" I was doing a bad photographer. And then I wonder, Adobe, I had the hope, you know, we're building like an AI camera there. And, you know, the idea is, you know, you can build a camera that's so good that everybody can use it and they can take good pictures. That wasn't the case. Like, we were using the same camera as the other people. I was using the same camera as Cecilia. She still takes better pictures than me. I know, we're working on editing features. Afterwards, even using them, I still couldn't make a better picture out of it. And that just had always been a thing. And, you know, the people that the person that complained the most about my photography skills is my wife. Of course. She asked me to take photos of her and I just couldn't do it. Again, she's always, always on me about, you know, how bad a photo turned out to be. And there was this one time and luckily it wasn't me taking the photo, but, you know, this is our wedding. So, we got married. How I spontaneously, like, we didn't plan it at all. So, we decided to get married like two days before. That's incredible. Yeah. Yeah. And so, we like, funer location, beautiful location on the beach. We managed to hire a photographer and asked the photographer to take one to photos of us on that day. We waited for three months to get the photos back. And it was, it was good. I mean, it was decent, but there's just so few of them. And, you know, my wife didn't like, like, the angle is, it was on the beach. So, it's all backlit. The angle wasn't that good. Just because of when you have one person there with one camera there. And, you know, we regret that, right? Like, we really want to share it out. Like, so, and that, that gets changed when we develop our technologies at photo labs. You know, that we were able to use our product and technology to retake a lot of photos on that day. And to the different angles, much better lighting, a lot of different things. And, you know, when I showed the photo to my wife, you know, it's not like, it's like the reaction is totally different from when she sees like a cool AI image. She's like, this is so great. Okay, print it so that I can hang it on the wall. Like, it's, I think that's really the power of photography is when you connect all these beautiful images to something that you experienced and you want to share with others. And it's so powerful that, you know, when printed out, hang it on the wall and we can point to our friends and family when they're on your house. Look at this photo. There's something so beautiful there. I just want to, you know, just talk more about, which is there's a moment where you experience that's the truth of it. Right. You know, you take a photography, but then how much of a photo, what a photo capture, what a light capture was actually real. But maybe what matters is how you remember the moment. Exactly. And the question becomes, technically, how do you reconstruct the moment? What do you need to get there? Yeah. So there are two things that Martin sort of touched a little bit on that is the identity part of it. You know, you wouldn't, I think Martin described it in the best way that I can describe it is, you know, when you look at the photo, you don't think it looks like me. You think, oh, that's me. Yeah, totally. The imperfections, imperfections, all of it, that's me. So that's a problem that we're trying really hard to solve is just to get the identity right. And the other part about the moment is the context and the actual moment, right. It's not enough just to make sure that's a person in that photo. It actually needs to be the moment that you how you remember it. Right. And that's fundamentally different from identity because that's evolving and changing all the time. Well, identity while it was, it was slowly. Right. So these are two things that we we tackle at photo labs and we take two very different approaches for that. Right. The first one to get the identity right is about to make all these genetic models, large powerful genetic models personal so that they know you, they know how you look like. And so then can generate photos of you. Yeah. So there's so much nuance there, just not looking at ourselves, but on like people we know, pets we know, I guess like from a technology perspective from Lightrow all the way to photo labs, like, what was the technology, the technical trajectory that lead us to this point? What are the new net new on logs that you know, you have to innovate to make it work? Yeah. I say, I say there's a, there's a big leap. So from Lightrow, other ways than there's hardware and there's a communication of powerful other software they can do behind us in like to do basic stuff to improve a photo, like improve the lighting or improve the noise, improve the revolution. And then it gets a little bit more creative with like chain and lighting. And that's what a lot of CCTG work was on. And so the real, the reals, the big improvement that we have seen in the past few years is Genei. You know, Genei is able to generate all these very good images that look so realistic they cannot tell is real or not, but that's not connected to our life. Right. That's not that's very different from photography. Photography is about documenting, recording something and like, or like something they can look at and think about your memory from the past. So, and the problem with the existing technologies, these existing models are so generic, like they train all the data on the internet, but they don't know you specifically. They don't know about your life. So that's where our breakthrough was in we figured out the best way to personalize these powerful generated models on your context, on your identity. So they can generate images for you. Yeah, among the group, I think there's one constant person who's always been there in the history of computational photography. It's like when I'm 18. So I guess like throughout the history, like based on the trajectory you have seen with the new knobs, like now we can generate photos that capture the historical moment that we're eating like felt very personal. What are the net new use cases? Like everyone's most excited about and then based on what you've seen the past and what you expect in the future. I'm super excited about it. So, you know, it bit by me like I, you know, I described how I got into this for photography. And I have just always taken, you know, an unimaginable number of photos. Okay. I'm very serious about my photography. My family is a very, you know, photography driven family. My youngest kid is like really into photography now. So I love photography. I love taking pictures. And I'm a student of the composition of photography, you know, the expression of within a still frame, how you capture something that's really, you know, meaningful to you, which can mean so many different things. Whereas like the humanities all the way intersecting with tech right into like the frame that you see in a moment and that we all love to look at. So, so I consider myself a photographer that cares about the light. That cares about the authenticity of that moment. I shot a lot of slide film before I got into digital photography. And I was the person that will wax on about how it's so, this is such a meaningful photo because a slide photo for those that have not taken a slide photo before is you take it with the camera, right? But it's a the piece of film that eventually you show is not printed. It's you put in a projector, right? And light flows back through that slide. You went on to a screen. And so the film that saw the light of the world at that moment is the same film that you're putting light through on a wall to tell someone, hey, look at this thing, right? It's the same thing. And there's like some deep authenticity to that, I think, right? Which doesn't go away in the presence of digital photography, doesn't go away into the future of photography. But I'm that person, okay? So when I talk about like the future of photography, we're generative and not all photographers feel about this the same way I do, but some photographers I know that a few are very serious about this as well. Do feel similarly is that that future photography does not necessarily have to be bound to the authenticity of the light at that moment in a technical or physical sense, in order to have meaning as photography. And if you believe that premise, which I deeply do, then we are at a land shift moment for, you know, for imaging of photography. Because generation or AI driven generation is this completely surprising thing. I mean, I was just been floored by the beginning of this and each of the subsequent steps in this, you know, very rapid evolution of AI technology, visual generation, AI technology. And now we can with words describe something and have a picture come out that it just looks remarkably like the picture that we were expect to see, right? Which has been impossible in graphics. And to be really clear, this has never been possible in photography. Right. Because the critical moment, the decisive moment in photography, which is, you know, this thing that as photographers, we seek to capture, is so fleeting in the space of, as the photographer, in the space of where I have to position my camera exactly, the focal length I have to choose, the focus that I have to achieve, the arrangement of my subjects that I need to achieve without arranging them. Because I'm apparently, you know, in a photogelistic sense, I'm just there to document the scene. So if it were a decisive moment of picture, I wouldn't be messing around with that. That moment in space and time to be able to capture a photograph that has all of that essence is, is always impossible. Right. We can imagine it and we've seen examples of it because those are the ones that have prominence in, you know, in our minds and our memory and in the culture. But to achieve that in our own lives is impossible. But if we can generate pictures and if we have access to technology that can have identity and knows about us and can make images that we feel like our representations, authentic representations of us and are the places we've been and of the things that have happened. Then we can imagine in a completely software sense trying to identify these moments in space and time, right, in this generation mode and achieve, you know, these phenomenal photography photography in our most mundane moments. That to me is what going to change. The future photography is going to be centered there and we can expect photographs to have this kind of, you know, a look for all of us for our most mundane moments that we've never been able to get before. I totally agree with the statement that photography or like how we see photography is not about how it's made or it's taken. It's really about what's the emotion that it provokes in us. So I really see, I totally see what Rennie Scott described this. I think the relationship between like us and the photos that we make or like even the cameras that we it's going to shift or change like in a way that so kind of tracing back a little bit to back to my PhD thesis which is actually making casual videos and photos more like cinematic. So one of the motivations for that was I think maybe Renn told me this story like there was a journalist who made the most most authentic story using an iPhone. I think a lot of previously it was like when you do like interviews I go in journalism like you use like large cameras like you set up the lighting and make sure everything is correct or like it's correct in a sense of like it looks good but when you only have an iPhone which people are so familiar with in front of them and then they're you are asking them to tell your story like the I think the emotion that they express what like story they told just becomes more authentic. So I kind of see this so like yeah I'm just seeing this coming like how it's fizz into the story that I thought I was trying to tell is that maybe if imagine we have these the genitive models as it's like understands us so well and can generally like photos or like moments that we actually experienced then we don't need to spend so much time on how to capture. The capture becomes like a way to collect information to collect the context and so in that sense we may even have wear book cameras or like even multimodality sensors that just try to capture as rich context as possible not only the light but also maybe third more or even like different ways of like the signals so that gathering all this information allow us to produce the best possible photos or images that can really represent how we felt at that moment. I think that authenticity is like a very interesting like thing that I think we're hoping to enable in the future. That's just interesting because it reminds me of you know before photography became a thing there's portrait artists who go to rich people's homes and then have someone pose in front of them draw a portrait and then back then what they compete on the skill set is like how realistic it's the portrait like because it looked like the person and later on obviously we have you know photos we have other technologies what they compete in art becomes like is this artistic is this like you know expressionism things like that came to be so there's like a net new aesthetics shift when new technology comes out you know like users prefer different things as they can do more things with the new you know technology coming out. I guess the question for Martin would be because you've you know seeing all the different trends and platforms shifts what do you think would be the net new user behavior here and I work yeah. So as a founder you kind of have to see the future and to lead people there which is what you're doing and I play that role for 11 years. The great thing about being an investor is you kind of get to be a spectator in a way that you just kind of watch it unfold but there are these general trends that you can you you know we're going to happen when you have a shift like this and so that to me is what's so exciting. So for example what's going on with this generative AI stuff well you're bringing the marginal cost of creation to zero and so areas that weren't markets or had all behaviors blow up like nobody would have thought to the image was a market. It's just been a totally failed market forever. Textes meet has basically been a failed market with generative AI because it's so cheap and the quality is so good that these have become these massive markets totally new user behaviors great companies and like the web was the same way by the way like remember the early websites do remember like hamster dance. Oh I don't know. I don't remember answers. I remember geocities. Like do you know what the first video was online? A cat something was a cat. Nope. No. It was a coffee pot. So there's like some guy in Cambridge that wanted to know whether there was like the coffee pot had coffee and it yeah seriously and it was so popular to go look at this coffee pot that did nothing and like so it sounds really silly because you're like it was a coffee pot but like if there's a coffee pot then you get YouTube and you get Netflix and so every once in a while you get these dislocations where there's this new technology and the economics are different and the capabilities are totally different and you know it's going to be amazing and you see glimmers of it but you don't know quite what it's going to be and there's always like these kind of early versions that are kind of funny like hamster dance which is just dancing hamster and so it feels like well honestly when I saw my picture in that it was like one of these moments like this is the first time I've been like wow I've been transported into so I actually don't have a specific answer I don't know what the user behaviors and thank goodness like I'm not in the position to do that I don't know what specific products or use cases I do know this is a platform shift I do know like you know these areas will blow up into big large markets you know and that's generative AI generally and for sure on photography I'm just really selective on the consumer side again I'm just I wouldn't even call myself do photography I work on photography but I don't do photography I do I take photos and there's I think there's so many people out there that I like like me just take photos every day and in the past you know they have they have to try really hard you know and for me personally I want to be being that moment and I want to have something to look at remind me that moment but I don't necessarily need to spend all the time to try to capture that moment and I feel like that's something that can be changed with gener I you know you you can be present in that moment you can enjoy that moment and still afterwards you have something very good to look at to remind yourself about that moment or to share with others about that moment and I think that could change I actually you know I actually think you can kind of roughly articulate like the the quantity of of of change that's going to happen so I'll just give you an example so um um how much would it cost 10 years ago to have take a picture of me with a dog on the top of the Matterhorn I mean like you know top of the Matterhorn yeah yeah and so let's say like it's $50,000 to like airlift to like me and the dog you know whatever whatever um and then you know if if you have a model that's good enough like what does the inference cost of this model a hundredth of a penny or something like that you're talking about like five orders of magnitude of difference right and so like you just know that the laws of physics on generating images we have an emotional connection to about the real world is shifting in a dramatic way and historically every time we have a four orders of magnitude difference whether it's like the difference between like somebody computing numbers by hand in any act or sending an email in the internet like anytime you have that you just get these kind of super amazing new areas grow up and I I think this is definitely one of them pictures with emotional connection to this but it's not it's not it's not like it like whatever going to mid-journey and creating some fantasy image it's like it's like I would you know it's like me on the Matterhorn right like it's like you have an emotional connection that it's you and you're there in some environment so yeah and something is just like it's priceless like like because that's the thing about the thing that's guess there was zero saying the thing that gets fixed when you press the shutter and versus something they can do afterwards like for our wedding use case like we already pay like thousands of dollars to their photo to just take photos for us but for us to think about how much we pay to get more photos after that wedding like three years after their wedding that's that's huge right like there's just so many moments in your life that they're priceless and yeah one way I've been looking at is as I was trying out your product I felt like the early days when I was trying out Instagram I was you know happy to be there when Instagram launched yeah and back then I mean filters were not that common what's the future of filter yeah no filters you know it was a lot of one to one analogy a lot of these aesthetics that we said on these social medias I think that these are what like others said like maybe like it's something that the word legs or like it's something that's trendy it's like on social media it's like because it's popular but there's I think people are craving for something that's more personal and more unique and also more like just authentic it's a so I think what we're building now is that like what looks good doesn't necessarily mean that it's idealized it's perfect it could be but it doesn't necessarily need to be because we enable this personalization aspect so you can actually I think what looks good means like totally different things to different people it could be more expressive it could be more authentic it could be very perfect but what we enable is to just open up this possibilities to everyone so that they can choose what means good looks good to them it's not like a preset like filter that's like decided by others or like decided by these platforms like what we're enabling is something that's decided by you it's like you can create your appearance or your fault like your way of like remembering things in a way that's best like represent you I think that's the unique part I do like filters in a sense that simple to use and the unlocks creativity for consumers like with one button click you can you can quickly change or how this photo looks like different styles it was great back then but when you look at it today so limited right like only you change the color and tone and even that people are amazed by that right like now think about what you can do with jennie right like we want to bring that simplicity and like so it's still very easy to use and it's still something they can do after capturing but it can unlock a lot of things they can do to your photo like different angles different lighting and even change the expression slightly open your eyes or all these kind of different things right like we still want to make that easy to use and but and imagine how much creativity that can unlock I'm from the consumer yeah in the info land I think maybe you're thinking about the same thing we think about levels of abstraction my filter is one level of abstraction where you kind of have this you know color correction or like relating on top of the photos the next level of abstraction is that can I change the subject can I change different more of a parameters on the photo the other level of abstraction now you know with jennie eyes that you can change the background you can change like the context a little bit but still preserve what feels true to the person taking the pictures what's the spectrum of the knobs that you guys have seen that you feel like would be the most important ones to enable for the users I think to to go back to the core of photography is to I think still record or like document something that's authentic that actually happened in your life I think there's definitely the content that we want to preserve the content is really the who where and when but the how I think that's the most interesting knob and also that's the most difficult part in photography like how to take a good picture just you think deeper what do you mean by how yeah so how is basically that all the things like all these notoriously difficult things I think we couldn't get right the perspective the angle the composition and the moment and also the lighting I think those are the three key aspects of that that's fundamentally difficult to do post capture that we can enable like right now with jennie AI and I think that's the type of control where it would make it will enable faithful enhancement on photographs what about the styles I don't know like as a fellow user martian you know like when we generally some pictures of the subject there's a different styles people can choose from and then people have very drastically different reaction to the style I don't know what was your reaction on different styles of your you know retake up your pictures but it is interesting how people have like preference what constitutes you a style like you know like what that looks like the way like we define kind of currently said at photo abs is like we have these different um photographers styles which for example one of them is called faithful which is almost like it was the photo that taken by the same device same almost the similar same composition but just like in a slightly better angle just doing more subtle changes but it can um it almost like no style no filter but it's like better in a in a more faithful way there's also like more like portrait photographer who really likes to use like like really long lenses to take a shallow depth of field separate foreground from the background make the subject a very center in the scene there's definitely also the film look that everyone really loves like timeless like a very vintage style so I think there's the style really encompasses um from the angle of the type of device that is taking it also the so basically what lenses you're using the camera the type of camera you're using and also the the palace where the color where the tone of the color and tone uh that this photo is being retouched in that direction quick question for you like so um on one side of the continuum there's like you took a picture that exists and you you minorly tweak it and the other end of the continuum you generate something that's never happened right like like where do you think at least for photo labs like it like it starts to leave what you're focused on like do you expect the majority of use cases can be around things that have definitely happened or could happen or like a subset of like how do you think about that hmm yeah I really like this mental picture of this thing on the left and this is on the on the right microsoft slider button yeah exactly exactly and I feel like what what users want is something in the middle and different what different people expect different things right some people they expect to be very creative and some people expect to be very authentic um and and once you know I was talking about these different knobs that we can control and just like I I just remind myself like how good the technology has been today like think about what lightwork did in the past you know just to change one parameter focus how hard it is to do that yeah and now we're talking about changing that angle, changing the composition, changing the focus, changing the lighting, changing everything right like so that's and that speaks to you like this knob that this spectrum thing right so um and that's where we are taking a fundamentally different approach from all the uh genetic models out there they're creating something that's totally generative all the way all the way to design and for a lot of this image editing apps a lot of this like bi-room Photoshop they're starting from this something from all the way from the left and try to bring it to the middle and that's really really really hard I just think about how many things they need to change yeah so what we're doing is what we want to directly generate this thing in the middle that's great now where it is in the middle we want the user to to define that but we're definitely right now our focus definitely is to generate something that's grounded in reality so we won't try to fab like could that somewhere in this thing even though they're not there um but but like when you when you're taking that picture your eyes was closed and you wish that your eyes were open yeah we could totally do that and so that's that's where where we are on the spectrum about changing backgrounds now right now um because I mean we can't totally do that um but we're still in the photography sense but what we can do though is that say you're at this restaurant and you take a photo of something and uh uh just just that restaurant and then you have a photo of yourself sitting from the table but the background isn't that good so you want to change that but still it remains to be the same restaurant we can totally do that still grounded in reality um love that um I guess just to wrap up I want to ask everyone uh the same question and just brainstorm on it like what's the most exciting thing that you think will happen in competition of photography in the next 10 years uh to me is like um I want jenny I want day to be as um popular ubiquitous as camera and I think that the reason people like camera is that they they're deeply connected to our life right like that's why why we use camera and uh um I don't think jenny is there yet today and uh so that's what we're in at reimagining you know where we want to bring jenny into photography something that people of just taking photos that some people videos that some people think that people really like and uh we don't change that and I hope one day uh everybody will start using jenny i instead of using a camera there will be using a model to generate a picture of themselves yeah the interface still needs to be there but what it is we'll see yeah yeah yeah well there's a lens or some sensors yeah I think that's uh something to be discovered yeah i think um maybe even taking a step back to me it's more like it's like I do hope um that people can just make memories in a very effortless way and it's i think right now we bet we take the bet that jenny i is the next shift to enable this and we genuinely believe that uh there's like a very long way to go uh to to unlock this uh possibility but this is like really an exciting starting point where we for the first time be able to create things that you can you kind of can relate and you find meaningful to you and your life however i think we are at the core like we want to build something that uh can allow people to just uh connect themselves with the memory in a very beautiful way i can make it a little personal for me because i take a lot of photos as i described i think that for me a um could see in the future where uh i don't feel something exact to describe earlier i don't feel like i need to be uh in a moment taking the pictures i'm the family photographer so i do that right and if i don't then there's a sense of loss like this moment you know we're going to enjoy it as opposed to uh record it or and make a memory of it and i think that in the future uh this uh technology is gonna enable us to have the concept of being able to send a photographer back in time right right it could be a professional in my case maybe i'd want to send myself you know back in time to be able to take pictures uh of those moments when i wanted to just be in the moment and enjoy the moment or maybe i could send it back to any moment in my past where with all the skills that i have now and the gear that i have now i could send myself back and then refotograph those moments uh or in the future i could pick from any number of real photographers whose work that i really admire because of the way they as human beings bring out the most beautiful pictures which is both a social you know skill as well as a technical skill and maybe be able to hire them and send them back into my past with my photograph catalog of or the things that i've done and places i've been and have them refotograph moments for me that i had had before and i think uh the possibility of all those things i think is just really going to transform photography where we don't take so many pictures or have to take so many pictures in the moment uh and our whole lives of images are going to be enriched yeah sending yourself back to uh as a person i probably want to do that uh some of the people may want to pick you know their favorite photographer or the work of that photographer and figure out how to about i think there's a really big opportunity for photographers into the future too it's going to change photography right as a profession i think there's very interesting uh um you know opportunities in there i think people uh that our photographers can continue to be central to that but if you can transport photographers through time and space to be able to take pictures yeah i think that's a really big opportunity uh potentially um and last but not least but this is 10 years 10 years 3dVR like i want to go back to my bedroom when i was like like take a picture reconstruct this i really really want yeah i really think that VR is gated on content yeah 100% and i think that this is like a fival path for me to go back to like my childhood and i think that'd be pretty amazing oh that's beautiful do you have pictures from your childhood? 100% amazing they need to be at it thanks for listening to got more great conversations coming your way see you next time as a reminder the content here is for informational purposes only should not be taken as legal business tax or investment advice or be used to evaluate any investment or security and is not directed at any investors or potential investors in any a-16z fund please note that a-16z and its affiliates may also maintain investments in the companies discussed in this podcast for more details including a link to our investments please see a-16z.com forward slash disclosures

Podcast Summary

Key Points:

  1. Discussion on the fleeting nature of capturing moments in photography.
  2. Exploration of computational photography's evolution from Stanford camera arrays to personalized generative retakes.
  3. Focus on the shift from hardware-based photography to software-based and generative models.
  4. Personalized visual genealogy in everyday photography at Photo Lab.
  5. The impact of generative models in creating compelling and realistic images.

Summary:

The transcription discusses the fleeting nature of capturing moments in photography and the evolution of computational photography from hardware-based to software-based models. It delves into the concept of personalized visual genealogy at Photo Lab, aiming to create images that reflect the individual's life experiences. The impact of generative models in generating compelling and realistic images is highlighted, with a focus on bridging the gap between capturing moments and enhancing them through technology.

The conversation also touches on the future of photography, discussing the shift towards generative models and the merging of photography and graphics in the digital age.

FAQs

The idea of Lightro came about as a way to bring more computation into photography by marrying computer graphics with photography.

The inspiration behind the development of Lightro was to focus on computational photography and bring all of graphics into the core of photographic imaging.

Computational photography has evolved from hardware-based solutions to more software-based solutions, transitioning from dedicated cameras to features integrated into smartphones.

Photo Lab focuses on bringing personalized visual generative models into everyday photography to create images that reflect the individual's life experiences.

Photo Lab's approach personalizes AI models to create images that are relevant to the individual's life experiences, moving beyond generic generative models.

Users, including the first test subject Marti, were impressed by the output of Photo Lab's generative models, finding that the images produced felt more like photographs than traditional generative art.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.