The transcription discusses the fleeting nature of capturing moments in photography and the evolution of computational photography from hardware-based to software-based models. It delves into the concept of personalized visual genealogy at Photo Lab, aiming to create images that reflect the individual's life experiences. The impact of generative models in generating compelling and realistic images is highlighted, with a focus on bridging the gap between capturing moments and enhancing them through technology. The conversation also touches on the future of photography, discussing the shift towards generative models and the merging of photography and graphics in the digital age.
Transcription
9566 Words, 51865 Characters
The critical moment, the decisive moment in photography, which is, you know, this thing that
as photographers we seek to capture, that moment in space and time is so fleeting.
A moment is no longer fixed when the shot is pressed. It's a very interesting like a new
dimension of what photography like is. If you believe that premise, which I deeply do,
then we are the land shift moment for imaging of photography. What if you could retake your favorite
memories, fix the lighting, catch the smile, or even open your eyes years after the moment past.
In this episode, A16D Journal Partner, Martín Casado, and partner Yoko Li
sit down with scientists and Lightro Founder Red and Ing, plus photo labs co-founders Cecilia Zhang
and Zach Shao to trace the wild arc of computational photography. From Stanford camera arrays and
lightfield refocus to today's personalized generative retakes. Together, the unpack why identity
and context matter more than filters and ask whether the future of photography even needs a lens.
Let's get into it. I'm super excited to have the group here together because we share a lot
of personal history and then chatting about evolution of computational photography.
One fun fact is Cecilia and we went to college together and we actually know each other for
more than 15 years. I kind of watched her working a first day of college. I don't know if you're
remembered. You kind of carried a camera. I'm watching you. And I just so great to kind of see you
turning that into like a top-notch research career too. Adobe and where you met Zach. Another
connection here is Cecilia that her PhD was Ren, who was the founder and CEO of Lightro camera,
which is a lightfield camera, very revolutionary. And another fun fact is Ren and Martín 20 years ago,
20 years ago, 15 years ago, maybe a 20 years ago. We were actually grad school classmates and
when did grad schools together. We were actually portfolio founders in the first fund of
Andrewson Horowitz. Wow. And I was his reference. You know this Ren guy. The best entrepreneur
ever knew so to say something nice about me. So you know, Ben decided to take a look.
Yeah, maybe the right place to start is kind of turning back to history, like thinking about
historically how the computational photography has the field evolve. How did you come up with Lightro?
Like the story behind us. Founding Lightro 2. And I'll let you start Ren.
Exciting topic. Exciting times. I think for computational photography, photography at large.
And I'd say that I think computational photography has been in the mix, you know,
at the since the beginning of digital photography, right? I mean, you can't have a digital picture
without doing an immense amount of processing and very basic things or things we would think of,
very basic like making it make the right colors involves, you know, some of the most studied algorithms
early on in computational imaging. And so I'd say there's been a ton of computational imaging
where I arrived on the scene was that I was really into photography. Like Cecilia, I carried a camera.
Actually, I got into photography because all my best friends were rock climbers. And one of them
was really into photography and he said, oh, you should, you know, like try my camera. And it's a
big climb at the top of a root and then try to take pictures of the nixers and climbing up and make
ourselves look cool, you know, so that was my introduction of photography. And I got hooked.
And then I decided for my PhD that the stuff I was working on, which I thought was great,
was not going to go where I wanted to go. And I was like, I want to do photography stuff. So I tried
to marry that stuff, which is computer graphics with photography. And so for me, it was really, you know,
an attempt to bring more computation into photography by bringing all of graphics,
how we make pictures with the computer into the core of the question of like, you know, how do we,
how are we going to do photographic imaging? And so, and that was the start for. And Marti, you were
there when we made the investment to your lightro and what was it like for computation of photography
back then? So I'm a systems guy. And so here's what I remember. What I remember is you'd go into
gates and like, because Mark LaVoy had this camera, which is not one camera, but a whole bunch of
camera. 100 cameras in an array. 100 cameras in array. And then it created a, I was like, I think
a gigapixel image and like we actually blew it up and we like put it on the wall. And you could see
very little details of like the, the quad. And so from there, I think now like you can just have
this in a camera, you know, like that you get off the shelf or a lot of features of that. Yes,
I mean, a lot of a lot of ways I think that kind of computational photography is accessible now.
But my entry point into life view photography started from there that array was built by Ben
Wilburton, right? Who did this PhD around our time too? And I said, well, look at that. I was like,
what this is really cool and it's really complicated system to build. I think it'd be great if it was
inside my camera. So I can use it. And miniaturizing that, you know, 100 cameras integrate into
the body of a single camera was essentially my entry point into life view photography. And then
a long time later, Ben it, you know, worked at Lightrow, cave on fat alien. Also, did you work at Lightroom?
He was at an IEEE for a year before you went out to start his presentation. I studied with
Qual for calls. He taught me the graphic stuff. I taught him the system stuff. He line is one of my
favorite people in the world. You know, we we roomed together in grad school. He was the best man in my
wedding. So lots of, lots of connections. Critically too, Benji came. He was also finishing his
PhD around that time too. He came back to finish his PhD after, you know, doing that whole SGI thing
and being the most engineering company in the valley decided to finish his PhD. So we're all there
around the same time. You may want to describe actually what Lightrow did. Yeah. I remember when you
first showed it to me, it just absolutely blew my mind. So I just a lot of fun because it started with
a very simple premise, which was at the time. I was like, well, if you try to take pictures of your
friends climbing up a climbing road and you're hanging on the end of a row, it is very hard to
hold stable enough to focus accurately. So a lot of pictures, if you're doing that kind of
photography or out of focus. So I was like, you know, do you have to focus a picture before you take it
because in computer graphics, with ray tracing in particular, we taught a class where we have
students simulate a camera system, the optics of a camera system and make them synthesize
photographs, focus at different depths. So if you can do ray tracing, it's sort of very core,
you know, instinctually that you can focus after the fact. So the idea is to bring graphics into the
body of the camera. You have to capture more data. You capture the rays of light in a photograph instead
of just the pixels in a photograph. And then you apply ray tracing, computer graphics ray tracing
in order to synthesize photographs after the fact and can focus in different ways. But once you
got that and we have this higher dimensional data set we call life field. So life field photography.
Once you've got that, you can do much richer computational photography and turn significant parts
of the physical work of the lens and the camera into pure software. So when I map the camera
system onto more is law and more. That's amazing. Yeah, from that point on, like how did it evolve?
Like how did the computation of photography as the field evolve from like hardware based to more
software from camera to phone or yeah, we'll have to hear more. I would say the main theme I would say
is like mass speciation. It has gone in so many rich different ways. Computational photography has
enriched our lives. We're all rich and really with it because in our phones, you know, for many, many
generations of smartphones, the core marketing, right, decisive marketing feature set was the camera.
Oh, it's got this new feature. It's got that new feature sometimes. It says a bigger zoom. But
oftentimes it had additional features like portrait mode or cinematic mode or you know high-speed
stuff. We were probably lumping their two for for competition photography HDR, which is like it's
so standard now. They don't even tell you if they should do it anymore because every photo is HDR,
right? Burst photography, you know, under the hood, all these kinds of things. But it's not just in
photography. It's in scientific fields, black hole imaging, you know, taking this data that really
doesn't look like an image at all from the event horizon telescope. This interferometric data
and then figure out how to solve the problem like what does a black hole look like that would produce
this kind of data. We think of that as computational imaging, I guess, as well. So speciation. So many
different things have happened. Yeah. So I guess like tie it back to, you know, pulling on the
personal side of the story, you and Cecilia met when Cecilia was in grad school and you all worked on
so many things together. How did the idea of photo lab came to be? How did you find it, you know,
rinse lab under rinse supervision? Well, I think I just want to add more, one more thing to light
roll. I think the, there's a mental model that makes life very, like, looks really visionary at
a time is because the couples that capture from, like, what you get at the output. So it's like,
moment is no longer fixed when the shutter is pressed is like, can be tweaked afterwards.
Like, so it's a, it's a very interesting, like, a new dimension of, like, how, what, even like,
what a photography, like, is. Right. So I think that kind of a definitely effect, like,
influence that what we are doing at photo lab is that I think this involvement from, like,
focusing a lot on cameras and hardware and now to maybe software, like, just for image manipulation
and editing. It also inspired us, like, oh, what we can do with, like, this era of, like, Genie I,
where there's like, a lot of, like, yeah, like, generative models are becoming really, really good.
Like, they can just produce compelling images. They, a lot of time, like, those images are just
perfect in composition lighting and everything. That's like, fundamentally, difficult to do in
photography. Like, it's a, so when Zach and I were, when we were, when we were at Adobe, like,
we worked on, we were editing towards like Photoshop Lightroom. And also, we touched like,
computational camera, like, building a capture, capture software as well. But, you know, like,
editing, capture, we just realized people are still struggling with, like, capturing what
they meant to. Because a lot of the issues are not really in the, how you edit afterwards,
or like, how, what's the, whether you have the best, best camera in the world is really,
when you have the camera, and when you want to capture some moment, like, how do you do that?
I think it's really, like, just evolving from hardware to software to Genie I,
it's really, we're just using a different tool to ask the question of what, how we can capture this
moment in the best way possible. So, I can see the connections between, like, all these
developments. And I just think we kind of fully, but Genie, I could be the next shift in this area.
Yeah. Just from, from my education, is there still a big distinction between, like, photography and
graphics? I think so. Is there? Yeah. Yeah. I mean, graphics is a, you know, a very broad term. But,
I'd say it's sort of making pictures with a computer, but usually pictures that are almost like
not taking with a camera. So, if you did a photo-real generation using a foundation model that, like, looks
something, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, and then there
was like a 3D engine and they were different. But like, now I can, yeah, no, I think you're asking
the right question, which is that I don't think today that people would agree with a statement
yet that photography is graphics in a generative sense. Okay. Okay. I think many people feel
rightly that, you know, this is in the air, you know, and there's a lot of, you know, complex ways to
feel about that or think about that. But to me, it's just really clear, you know, that the future
of photography is going to be heavily centered in, if not, the core of it's going to be a generation.
Technical distinction between generating a realistic photo and generating a non-one.
There's, there's, honestly, this is just really clear. Yeah, yeah. To me, like, I feel them very
similar. Yeah, there's a comment I saw on Reddit, which is like, I like it a lot. He says that, you know,
if something is purely created from someone sitting in front of a keyboard, I wouldn't call that a photo
of a photography. But if something is created from something based in reality, like, so you have
something where there's a light world camera or just a smartphone camera capture some
information about what happened there the moment. And then from that, you create a beautiful
representation of that thing. That's what I was talking about. So in the past, photography was like
optics in a camera and computer graphics was a computer. And so the technical stack was entirely
different. But your definition right now has nothing to do with technology stack. It's like the use
of it. But it seems to me like more and more of the technology is aligned now, right? Is that fair?
Yeah, it is fair. And I think from a technology perspective, it's like, you still need the optics
part just so that you can capture information about what's there, right? But then a lot of the thing
happens when you're in front of the keyboard to create that image or that photo. Yeah, I really,
sorry, somebody like you, I really like this definition. I think the future photography is going to
include a lot of stuff that is actually just a keyboard or not even a keyboard that doesn't have optics
that's capturing it directly. And I think that's going to be very meaningful too. I think for things
that people will want to care about looking at, I think how we think of photography is often very
intimate. Things we care about, people we care about, shown to other people that we care about.
And so I think the person that's creating the photograph there and their thoughts and intentions
in the generation of that image will still remain at the core of a lot of the most authentic and
meaningful photography, but I'm not sure if it's necessary for the lens to be there in the future,
myself. Yeah, on that philosophical question, which by the way, I love this philosophy. I feel
what is photography? Yeah, yeah, I don't know if you want to go there, but
how does that feed into what you're building at photo labs? At photo labs, we are bringing the
visual genealogy into everyday photography for everyone. So I think the key here is everyday
photography, and also everyone. People are taking photo all the time. And so many times, we just find
a lot of things that just go wrong. It could be a bad lighting, it could be an awkward angle,
it could be a missed expression. So we just have this trouble, frustration of not capturing what
we really, really want it. So what we are building at photo labs is to bring personalized visual
genealogy into your, like, basically, it understands you and tries to understand you and your life,
such that it create and understands like photos or like images. If we do not want to call it photos,
because it's generation, like generated contents that actually reflect you and your life. So it's,
we kind of see this as, so we can do this because we, we have, we're able to personalize these AI
models and existing foundation models, like they are built to be generic and a bit very powerful,
but they are just, they do not understand personal content. So that's why, like, they can generate
a compelling photograph, but like compelling image, but completely hallucinated, it's not relevant
to what you experienced. So we're trying to bring that gap. This totally talking to the fourth law,
but this is where we can, you know, show example of what it's like to have the retake. And Marti was
actually the first test subject. What are the reactions based on, you know, the beta product you
already released? Like, what was the first reaction? Like you had when you saw it, and then curious about
your, you know, other user reactions? So, I mean, just, just to like, like, the foundation of this,
I've worked with a lot of generative models, right? So I was a very early user of mid-chirney,
I'm on the board of ideogram, like, of course, when investors in BFL. And so like, I know what it's
like to do character consistency, where you take your image and you put it in, and then like,
you generate something using a traditional model, like this was entirely different. Like, I can always
tell when I use, like, ideogram and consistency, which is very, very good. Like, it's a likeness of me,
or if I, you know, before I was like, you know, I have like some open source, you work flow,
where I do it with a bunch of images of myself, and it kind of looked like me, but it was, like,
there's almost like this, like, style, you know, that came from the images. It was almost kind of
felt like art. When I saw the output of this, I'm like, "Hey, I'm old. That is totally me." I mean,
it was like, it looked like a photograph. So it actually, it actually feels different than
standard generative models. It doesn't feel like art. It felt like a photograph.
Yeah. So I guess like, what, what did it take to make this? Like, a question for both of you,
like, what's the technology unlock here? That made it possible that, you know,
something like a foundational model couldn't do, which we use all day every day.
So for me, like, I have this anecdote of like, the first time, the first time I actually
know Cecilia a little bit more is when I was looking at her, the recording of her PhD defense.
And I remember in that defense, you really shared a story about how Cecilia's photography skills
has grown from, like, not so good to very good, during her PhD. I just knew where that story.
And I was the only thing that come into it that they just barely have made.
Cecilia's an amazing photographer. Just for the record.
And then I was like, "Why did my advisor tell me that during my PhD?"
I was doing a bad photographer. And then I wonder, Adobe, I had the hope, you know,
we're building like an AI camera there. And, you know, the idea is, you know,
you can build a camera that's so good that everybody can use it and they can take good pictures.
That wasn't the case. Like, we were using the same camera as the other people.
I was using the same camera as Cecilia. She still takes better pictures than me.
I know, we're working on editing features. Afterwards, even using them, I still couldn't
make a better picture out of it. And that just had always been a thing.
And, you know, the people that the person that complained the most about my photography skills
is my wife. Of course. She asked me to take photos of her and I just couldn't do it.
Again, she's always, always on me about, you know, how bad a photo turned out to be.
And there was this one time and luckily it wasn't me taking the photo, but, you know, this
is our wedding. So, we got married. How I spontaneously, like, we didn't plan it at all.
So, we decided to get married like two days before.
That's incredible. Yeah. Yeah. And so, we like, funer location,
beautiful location on the beach. We managed to hire a photographer and asked the photographer
to take one to photos of us on that day. We waited for three months to get the photos back.
And it was, it was good. I mean, it was decent, but there's just so few of them. And, you know,
my wife didn't like, like, the angle is, it was on the beach. So, it's all backlit.
The angle wasn't that good. Just because of when you have one person there with one camera there.
And, you know, we regret that, right? Like, we really want to share it out. Like, so,
and that, that gets changed when we develop our technologies at photo labs. You know, that we were
able to use our product and technology to retake a lot of photos on that day. And to the different
angles, much better lighting, a lot of different things. And, you know, when I showed the photo to my
wife, you know, it's not like, it's like the reaction is totally different from when she sees like a
cool AI image. She's like, this is so great. Okay, print it so that I can hang it on the wall.
Like, it's, I think that's really the power of photography is when you connect all these beautiful
images to something that you experienced and you want to share with others. And it's so powerful
that, you know, when printed out, hang it on the wall and we can point to our friends and family
when they're on your house. Look at this photo. There's something so beautiful there. I just want to,
you know, just talk more about, which is there's a moment where you experience that's the truth of it.
Right. You know, you take a photography, but then how much of a photo, what a photo capture,
what a light capture was actually real. But maybe what matters is how you remember the moment.
Exactly. And the question becomes, technically, how do you reconstruct the moment? What do you
need to get there? Yeah. So there are two things that Martin sort of touched a little bit on that
is the identity part of it. You know, you wouldn't, I think Martin described it in the best way that
I can describe it is, you know, when you look at the photo, you don't think it looks like me. You
think, oh, that's me. Yeah, totally. The imperfections, imperfections, all of it, that's me.
So that's a problem that we're trying really hard to solve is just to get the identity right.
And the other part about the moment is the context and the actual moment, right. It's not
enough just to make sure that's a person in that photo. It actually needs to be the moment that you
how you remember it. Right. And that's fundamentally different from identity because that's evolving and
changing all the time. Well, identity while it was, it was slowly. Right. So these are two things that we
we tackle at photo labs and we take two very different approaches for that. Right. The first one
to get the identity right is about to make all these genetic models, large powerful genetic models
personal so that they know you, they know how you look like. And so then can generate photos of you.
Yeah. So there's so much nuance there, just not looking at ourselves, but on like people we know,
pets we know, I guess like from a technology perspective from Lightrow all the way to photo labs,
like, what was the technology, the technical trajectory that lead us to this point? What are the new
net new on logs that you know, you have to innovate to make it work? Yeah. I say, I say there's a,
there's a big leap. So from Lightrow, other ways than there's hardware and there's a
communication of powerful other software they can do behind us in like to do basic stuff to improve
a photo, like improve the lighting or improve the noise, improve the revolution. And then it gets
a little bit more creative with like chain and lighting. And that's what a lot of CCTG work was on.
And so the real, the reals, the big improvement that we have seen in the past few years is Genei.
You know, Genei is able to generate all these very good images that look so realistic they cannot
tell is real or not, but that's not connected to our life. Right. That's not that's very different from
photography. Photography is about documenting, recording something and like, or like something
they can look at and think about your memory from the past. So, and the problem with the existing
technologies, these existing models are so generic, like they train all the data on the internet,
but they don't know you specifically. They don't know about your life. So that's where our breakthrough
was in we figured out the best way to personalize these powerful generated models on your context,
on your identity. So they can generate images for you. Yeah, among the group, I think there's
one constant person who's always been there in the history of computational photography. It's like
when I'm 18. So I guess like throughout the history, like based on the trajectory you have seen
with the new knobs, like now we can generate photos that capture the historical moment that we're
eating like felt very personal. What are the net new use cases? Like everyone's most excited about
and then based on what you've seen the past and what you expect in the future. I'm super excited
about it. So, you know, it bit by me like I, you know, I described how I got into this for photography.
And I have just always taken, you know, an unimaginable number of photos. Okay. I'm very
serious about my photography. My family is a very, you know, photography driven family. My
youngest kid is like really into photography now. So I love photography. I love taking pictures. And
I'm a student of the composition of photography, you know, the expression of within a still frame,
how you capture something that's really, you know, meaningful to you, which can mean so many
different things. Whereas like the humanities all the way intersecting with tech right into like the
frame that you see in a moment and that we all love to look at. So, so I consider myself a photographer
that cares about the light. That cares about the authenticity of that moment. I shot a lot of
slide film before I got into digital photography. And I was the person that will wax on about how
it's so, this is such a meaningful photo because a slide photo for those that have not taken a
slide photo before is you take it with the camera, right? But it's a the piece of film that eventually
you show is not printed. It's you put in a projector, right? And light flows back through that
slide. You went on to a screen. And so the film that saw the light of the world at that moment
is the same film that you're putting light through on a wall to tell someone, hey, look at this thing,
right? It's the same thing. And there's like some deep authenticity to that, I think, right? Which
doesn't go away in the presence of digital photography, doesn't go away into the future of photography.
But I'm that person, okay? So when I talk about like the future of photography, we're generative
and not all photographers feel about this the same way I do, but some photographers I know that
a few are very serious about this as well. Do feel similarly is that that future photography does
not necessarily have to be bound to the authenticity of the light at that moment in a
technical or physical sense, in order to have meaning as photography. And if you believe that premise,
which I deeply do, then we are at a land shift moment for, you know, for imaging of photography.
Because generation or AI driven generation is this completely surprising thing. I mean,
I was just been floored by the beginning of this and each of the subsequent steps in this,
you know, very rapid evolution of AI technology, visual generation, AI technology. And now we can
with words describe something and have a picture come out that it just looks remarkably like the picture
that we were expect to see, right? Which has been impossible in graphics. And to be really clear,
this has never been possible in photography. Right. Because the critical moment, the decisive
moment in photography, which is, you know, this thing that as photographers, we seek to capture,
is so fleeting in the space of, as the photographer, in the space of where I have to position my camera
exactly, the focal length I have to choose, the focus that I have to achieve, the arrangement of my
subjects that I need to achieve without arranging them. Because I'm apparently, you know, in a
photogelistic sense, I'm just there to document the scene. So if it were a decisive moment of picture,
I wouldn't be messing around with that. That moment in space and time to be able to capture a photograph
that has all of that essence is, is always impossible. Right. We can imagine it and we've seen
examples of it because those are the ones that have prominence in, you know, in our minds and our
memory and in the culture. But to achieve that in our own lives is impossible. But if we can generate
pictures and if we have access to technology that can have identity and knows about us and can
make images that we feel like our representations, authentic representations of us and are the places
we've been and of the things that have happened. Then we can imagine in a completely software sense
trying to identify these moments in space and time, right, in this generation mode and achieve,
you know, these phenomenal photography photography in our most mundane moments. That to me is what
going to change. The future photography is going to be centered there and we can expect photographs to
have this kind of, you know, a look for all of us for our most mundane moments that we've never been
able to get before. I totally agree with the statement that photography or like how we see photography
is not about how it's made or it's taken. It's really about what's the emotion that it provokes in us.
So I really see, I totally see what Rennie Scott described this. I think the relationship between
like us and the photos that we make or like even the cameras that we it's going to shift or change
like in a way that so kind of tracing back a little bit to back to my PhD thesis which is actually
making casual videos and photos more like cinematic. So one of the motivations for that was I think
maybe Renn told me this story like there was a journalist who made the most most authentic
story using an iPhone. I think a lot of previously it was like when you do like interviews I go
in journalism like you use like large cameras like you set up the lighting and make sure everything
is correct or like it's correct in a sense of like it looks good but when you only have an iPhone
which people are so familiar with in front of them and then they're you are asking them to tell your
story like the I think the emotion that they express what like story they told just becomes more authentic.
So I kind of see this so like yeah I'm just seeing this coming like how it's fizz into the story
that I thought I was trying to tell is that maybe if imagine we have these the genitive models as
it's like understands us so well and can generally like photos or like moments that we actually
experienced then we don't need to spend so much time on how to capture. The capture becomes like a
way to collect information to collect the context and so in that sense we may even have
wear book cameras or like even multimodality sensors that just try to capture as rich
context as possible not only the light but also maybe third more or even like different ways
of like the signals so that gathering all this information allow us to produce the best possible
photos or images that can really represent how we felt at that moment. I think that authenticity
is like a very interesting like thing that I think we're hoping to enable in the future.
That's just interesting because it reminds me of you know before photography became a thing
there's portrait artists who go to rich people's homes and then have someone pose in front
of them draw a portrait and then back then what they compete on the skill set is like how realistic
it's the portrait like because it looked like the person and later on obviously we have you know
photos we have other technologies what they compete in art becomes like is this artistic is this
like you know expressionism things like that came to be so there's like a net new aesthetics shift
when new technology comes out you know like users prefer different things as they can do more things
with the new you know technology coming out. I guess the question for Martin would be because you've
you know seeing all the different trends and platforms shifts what do you think would be the net
new user behavior here and I work yeah. So as a founder you kind of have to see the future and to
lead people there which is what you're doing and I play that role for 11 years. The great thing about
being an investor is you kind of get to be a spectator in a way that you just kind of watch it unfold
but there are these general trends that you can you you know we're going to happen when you have
a shift like this and so that to me is what's so exciting. So for example what's going on with this
generative AI stuff well you're bringing the marginal cost of creation to zero and so areas that weren't
markets or had all behaviors blow up like nobody would have thought to the image was a market.
It's just been a totally failed market forever. Textes meet has basically been a failed market with
generative AI because it's so cheap and the quality is so good that these have become these massive
markets totally new user behaviors great companies and like the web was the same way by the way
like remember the early websites do remember like hamster dance. Oh I don't know. I don't remember
answers. I remember geocities. Like do you know what the first video was online? A cat something
was a cat. Nope. No. It was a coffee pot. So there's like some guy in Cambridge that wanted to know
whether there was like the coffee pot had coffee and it yeah seriously and it was so popular to go
look at this coffee pot that did nothing and like so it sounds really silly because you're like
it was a coffee pot but like if there's a coffee pot then you get YouTube and you get Netflix and so
every once in a while you get these dislocations where there's this new technology and the economics
are different and the capabilities are totally different and you know it's going to be amazing
and you see glimmers of it but you don't know quite what it's going to be and there's always like
these kind of early versions that are kind of funny like hamster dance which is just dancing hamster
and so it feels like well honestly when I saw my picture in that it was like one of these moments
like this is the first time I've been like wow I've been transported into so I actually don't have
a specific answer I don't know what the user behaviors and thank goodness like I'm not in the
position to do that I don't know what specific products or use cases I do know this is a platform
shift I do know like you know these areas will blow up into big large markets you know and that's
generative AI generally and for sure on photography I'm just really selective on the consumer side
again I'm just I wouldn't even call myself do photography I work on photography but I don't do
photography I do I take photos and there's I think there's so many people out there that I like
like me just take photos every day and in the past you know they have they have to try really hard
you know and for me personally I want to be being that moment and I want to have something to
look at remind me that moment but I don't necessarily need to spend all the time to try to capture
that moment and I feel like that's something that can be changed with gener I you know you you can
be present in that moment you can enjoy that moment and still afterwards you have something very
good to look at to remind yourself about that moment or to share with others about that moment
and I think that could change I actually you know I actually think you can kind of roughly articulate
like the the quantity of of of change that's going to happen so I'll just give you an example so um
um how much would it cost 10 years ago to have take a picture of me with a dog on the top of
the Matterhorn I mean like you know top of the Matterhorn yeah yeah and so let's say like it's
$50,000 to like airlift to like me and the dog you know whatever whatever um and then you know if
if you have a model that's good enough like what does the inference cost of this model a hundredth
of a penny or something like that you're talking about like five orders of magnitude of difference
right and so like you just know that the laws of physics on generating images we have an
emotional connection to about the real world is shifting in a dramatic way and historically every
time we have a four orders of magnitude difference whether it's like the difference between like somebody
computing numbers by hand in any act or sending an email in the internet like anytime you have that
you just get these kind of super amazing new areas grow up and I I think this is definitely one of them
pictures with emotional connection to this but it's not it's not it's not like it like
whatever going to mid-journey and creating some fantasy image it's like it's like I would
you know it's like me on the Matterhorn right like it's like you have an emotional connection
that it's you and you're there in some environment so yeah and something is just like it's priceless
like like because that's the thing about the thing that's guess there was zero saying the thing that
gets fixed when you press the shutter and versus something they can do afterwards like for our
wedding use case like we already pay like thousands of dollars to their photo to just take photos
for us but for us to think about how much we pay to get more photos after that wedding like three
years after their wedding that's that's huge right like there's just so many moments in your life
that they're priceless and yeah one way I've been looking at is as I was trying out your product
I felt like the early days when I was trying out Instagram I was you know happy to be there when
Instagram launched yeah and back then I mean filters were not that common what's the future
of filter yeah no filters you know it was a lot of one to one analogy a lot of these aesthetics that
we said on these social medias I think that these are what like others said like maybe like it's
something that the word legs or like it's something that's trendy it's like on social media it's like
because it's popular but there's I think people are craving for something that's more personal
and more unique and also more like just authentic it's a so I think what we're building now is that
like what looks good doesn't necessarily mean that it's idealized it's perfect it could be but it
doesn't necessarily need to be because we enable this personalization aspect so you can actually
I think what looks good means like totally different things to different people it could be
more expressive it could be more authentic it could be very perfect but what we enable is to just
open up this possibilities to everyone so that they can choose what means good looks good to them
it's not like a preset like filter that's like decided by others or like decided by these platforms
like what we're enabling is something that's decided by you it's like you can create your
appearance or your fault like your way of like remembering things in a way that's best like
represent you I think that's the unique part I do like filters in a sense that simple to use
and the unlocks creativity for consumers like with one button click you can you can quickly change
or how this photo looks like different styles it was great back then but when you look at it today
so limited right like only you change the color and tone and even that people are amazed by that
right like now think about what you can do with jennie right like we want to bring that simplicity
and like so it's still very easy to use and it's still something they can do after capturing
but it can unlock a lot of things they can do to your photo like different angles different
lighting and even change the expression slightly open your eyes or all these kind of different things
right like we still want to make that easy to use and but and imagine how much creativity that can
unlock I'm from the consumer yeah in the info land I think maybe you're thinking about the same thing
we think about levels of abstraction my filter is one level of abstraction where you kind of have
this you know color correction or like relating on top of the photos the next level of abstraction
is that can I change the subject can I change different more of a parameters on the photo the other
level of abstraction now you know with jennie eyes that you can change the background you can change
like the context a little bit but still preserve what feels true to the person taking the pictures
what's the spectrum of the knobs that you guys have seen that you feel like would be the most
important ones to enable for the users I think to to go back to the core of photography is to
I think still record or like document something that's authentic that actually happened in your
life I think there's definitely the content that we want to preserve the content is really the
who where and when but the how I think that's the most interesting knob and also that's the most
difficult part in photography like how to take a good picture just you think deeper what do you
mean by how yeah so how is basically that all the things like all these notoriously difficult things
I think we couldn't get right the perspective the angle the composition and the moment
and also the lighting I think those are the three key aspects of that that's fundamentally
difficult to do post capture that we can enable like right now with jennie AI and I think that's
the type of control where it would make it will enable faithful enhancement on photographs what
about the styles I don't know like as a fellow user martian you know like when we generally some
pictures of the subject there's a different styles people can choose from and then people have
very drastically different reaction to the style I don't know what was your reaction on different
styles of your you know retake up your pictures but it is interesting how people have like preference
what constitutes you a style like you know like what that looks like the way like we define kind
of currently said at photo abs is like we have these different um photographers styles which
for example one of them is called faithful which is almost like it was the photo that
taken by the same device same almost the similar same composition but just like in a slightly
better angle just doing more subtle changes but it can um it almost like no style no filter but it's
like better in a in a more faithful way there's also like more like portrait photographer who really
likes to use like like really long lenses to take a shallow depth of field separate foreground
from the background make the subject a very center in the scene there's definitely also the film
look that everyone really loves like timeless like a very vintage style so I think there's the style
really encompasses um from the angle of the type of device that is taking it also the so basically
what lenses you're using the camera the type of camera you're using and also the the palace where
the color where the tone of the color and tone uh that this photo is being retouched in that direction
quick question for you like so um on one side of the continuum there's like you took a picture that
exists and you you minorly tweak it and the other end of the continuum you generate something that's
never happened right like like where do you think at least for photo labs like it like it starts
to leave what you're focused on like do you expect the majority of use cases can be around things
that have definitely happened or could happen or like a subset of like how do you think about that
hmm yeah I really like this mental picture of this thing on the left and this is on the on the right
microsoft slider button yeah exactly exactly and I feel like what what users want is something in the
middle and different what different people expect different things right some people they expect
to be very creative and some people expect to be very authentic um and and once you know I was
talking about these different knobs that we can control and just like I I just remind myself like
how good the technology has been today like think about what lightwork did in the past you know
just to change one parameter focus how hard it is to do that yeah and now we're talking about
changing that angle, changing the composition, changing the focus, changing the lighting, changing
everything right like so that's and that speaks to you like this knob that this spectrum thing right
so um and that's where we are taking a fundamentally different approach from all the uh
genetic models out there they're creating something that's totally generative all the way all the way
to design and for a lot of this image editing apps a lot of this like bi-room Photoshop they're starting
from this something from all the way from the left and try to bring it to the middle and that's really
really really hard I just think about how many things they need to change yeah so what we're doing is
what we want to directly generate this thing in the middle that's great now where it is in the middle
we want the user to to define that but we're definitely right now our focus definitely is to generate
something that's grounded in reality so we won't try to fab like could that somewhere in this thing
even though they're not there um but but like when you when you're taking that picture your eyes was
closed and you wish that your eyes were open yeah we could totally do that and so that's that's where
where we are on the spectrum about changing backgrounds now right now um because I mean we can't
totally do that um but we're still in the photography sense but what we can do though is that
say you're at this restaurant and you take a photo of something and uh uh just just that restaurant
and then you have a photo of yourself sitting from the table but the background isn't that good
so you want to change that but still it remains to be the same restaurant we can totally do that
still grounded in reality um love that um I guess just to wrap up I want to ask everyone uh the same
question and just brainstorm on it like what's the most exciting thing that you think will happen
in competition of photography in the next 10 years uh to me is like um I want jenny I want day to be
as um popular ubiquitous as camera and I think that the reason people like camera is that
they they're deeply connected to our life right like that's why why we use camera and uh um I don't
think jenny is there yet today and uh so that's what we're in at reimagining you know where we want to
bring jenny into photography something that people of just taking photos that some people videos that
some people think that people really like and uh we don't change that and I hope one day uh everybody
will start using jenny i instead of using a camera there will be using a model to generate a picture
of themselves yeah the interface still needs to be there but what it is we'll see
yeah yeah yeah well there's a lens or some sensors yeah I think that's uh something
to be discovered yeah i think um maybe even taking a step back to me it's more like it's like I do
hope um that people can just make memories in a very effortless way and
it's i think right now we bet we take the bet that jenny i is the next shift to enable this
and we genuinely believe that uh there's like a very long way to go uh to to unlock this
uh possibility but this is like really an exciting starting point where we for the first time
be able to create things that you can you kind of can relate and you find meaningful to you and
your life however i think we are at the core like we want to build something that uh can allow
people to just uh connect themselves with the memory in a very beautiful way i can make it a
little personal for me because i take a lot of photos as i described i think that for me a um
could see in the future where uh i don't feel something exact to describe earlier i don't feel like
i need to be uh in a moment taking the pictures i'm the family photographer so i do that right
and if i don't then there's a sense of loss like this moment you know we're going to enjoy it as
opposed to uh record it or and make a memory of it and i think that in the future uh this
uh technology is gonna enable us to have the concept of being able to send a photographer back in time
right right it could be a professional in my case maybe i'd want to send myself you know back
in time to be able to take pictures uh of those moments when i wanted to just be in the moment
and enjoy the moment or maybe i could send it back to any moment in my past where with all the skills
that i have now and the gear that i have now i could send myself back and then refotograph those moments
uh or in the future i could pick from any number of real photographers whose work that i really admire
because of the way they as human beings bring out the most beautiful pictures which is both a
social you know skill as well as a technical skill and maybe be able to hire them and send them
back into my past with my photograph catalog of or the things that i've done and places i've been
and have them refotograph moments for me that i had had before and i think uh the possibility of all
those things i think is just really going to transform photography where we don't take so many
pictures or have to take so many pictures in the moment uh and our whole lives of images are
going to be enriched yeah sending yourself back to uh as a person i probably want to do that uh
some of the people may want to pick you know their favorite photographer or the work of that
photographer and figure out how to about i think there's a really big opportunity for
photographers into the future too it's going to change photography right as a profession i think
there's very interesting uh um you know opportunities in there i think people uh that our
photographers can continue to be central to that but if you can transport photographers
through time and space to be able to take pictures yeah i think that's a really big opportunity
uh potentially um and last but not least but this is 10 years 10 years 3dVR like i want to go back to
my bedroom when i was like like take a picture reconstruct this i really really want yeah i really
think that VR is gated on content yeah 100% and i think that this is like a fival path for me to go
back to like my childhood and i think that'd be pretty amazing oh that's beautiful do you have pictures
from your childhood? 100% amazing they need to be at it
thanks for listening to got more great conversations coming your way see you next time
as a reminder the content here is for informational purposes only
should not be taken as legal business tax or investment advice or be used to evaluate any
investment or security and is not directed at any investors or potential investors in any a-16z
fund please note that a-16z and its affiliates may also maintain investments in the companies
discussed in this podcast for more details including a link to our investments please see a-16z.com
forward slash disclosures
Podcast Summary
Key Points:
Discussion on the fleeting nature of capturing moments in photography.
Exploration of computational photography's evolution from Stanford camera arrays to personalized generative retakes.
Focus on the shift from hardware-based photography to software-based and generative models.
Personalized visual genealogy in everyday photography at Photo Lab.
The impact of generative models in creating compelling and realistic images.
Summary:
The transcription discusses the fleeting nature of capturing moments in photography and the evolution of computational photography from hardware-based to software-based models. It delves into the concept of personalized visual genealogy at Photo Lab, aiming to create images that reflect the individual's life experiences. The impact of generative models in generating compelling and realistic images is highlighted, with a focus on bridging the gap between capturing moments and enhancing them through technology.
The conversation also touches on the future of photography, discussing the shift towards generative models and the merging of photography and graphics in the digital age.
FAQs
The idea of Lightro came about as a way to bring more computation into photography by marrying computer graphics with photography.
The inspiration behind the development of Lightro was to focus on computational photography and bring all of graphics into the core of photographic imaging.
Computational photography has evolved from hardware-based solutions to more software-based solutions, transitioning from dedicated cameras to features integrated into smartphones.
Photo Lab focuses on bringing personalized visual generative models into everyday photography to create images that reflect the individual's life experiences.
Photo Lab's approach personalizes AI models to create images that are relevant to the individual's life experiences, moving beyond generic generative models.
Users, including the first test subject Marti, were impressed by the output of Photo Lab's generative models, finding that the images produced felt more like photographs than traditional generative art.
Chat with AI
Loading...
Pro features
Go deeper with this episode
Unlock creator-grade tools that turn any transcript into show notes and subtitle files.