AI, Digital Twins & eCommerce 3D: Why 3D Artists Are Still Essential | Tobias Nientiedt

[MUSIC] Welcome to Virtual for Real Derabit pipeline podcast. I'm Max your host and with me today is Toby Nittit. Toby, welcome to the show. I'm really happy to have you here. Super excited. Hi, Max. I don't know how to introduce you properly. I would just say Toby's an amazing 3D and CGI generalist. Thank you. I'm really excited to hear your learnings from many years of 3D digital twins and working with CGI at various companies, big companies, and also studios. Just discussing about 3D digital twins and AI and the future of that. So maybe we can start with a quick introduction. How did your 3D journey start Toby and how did it develop? I studied communication and media design back in the days in Dortmund. There I've been in touch with CGI for the first time in one of our courses and was fascinated by the technology. Because I was able to generate images without the need of having a camera, a location, an independent of weather and so on. And that was super fascinating for me. And from that point on, I dove deeper into that technology and used it all the time within my, in the beginning of my career, when I was working as an art director and different advertising agencies just to visualise my ideas. But then yeah, I recognised within that journey that yeah, I had a lot of fun with creating those images and then went more and more into the direction of producing not only the visualization of the ideas, but also the final content. Yeah, amazing. And that brought you to working with various companies in different sectors. So I remember like we met when you were at Otto back then and a lot of people who don't know Otto's was one of the biggest, yeah, now online businesses in Germany. Back then was a catalog. We could order things so they needed really good visuals so that people could understand what they order. But I think the journey needs even way back. Yeah, exactly. That's a little bit more about that. Because I started advertising and then Otto knocked on my door and asked if, because I have found it in the CGI agency at Scholz and France within the Scholz and France network in the beginning for Opel. But then we also worked for different other clients and then Otto asked me if I would like to do the same within Otto for the home and living department. And yeah, sounds very interesting for me moving from advertising to the other side of the table. And yeah, it has been an amazing time. I was six years over there was responsible for leveraging CGI for product images for the PDP, the product detail page. And yeah, that was interesting because at that time Otto was on their journey from becoming or from a retailer to becoming a platform so that also other retailers were able to sell their products on that platform. And with that we were able to also to found production service so that other platform retailers were able to produce content through Otto's pipeline for Otto for the Otto platform. And yeah, that there was like a completely different part of the process and of the industry, right? There in the first place at Otto we were only responsible for creating content for our own needs. And yeah, you needed other workflows, other colleagues, other skills to really implement those kind of services. Very interesting time. Yeah, that's actually quite exciting because I know of similar cases like when you have, let's say retailers like in the US, Target or so, Target has a house brand, the Target brand, can buy stuff that is made by Target for the Target stores. But then you can also buy a lot of other articles from other brands, right? And all these national brands from the US that sell to Target. And I think it's similar with Otto, right? Let's say I don't know so fast or so that are made by Otto, but you can also buy other articles from other brands, right? It's a little Otto platform. And so I mean to standardize CGI basically, ideally. Yeah, absolutely, absolutely. And it's always those platform ecosystems, right? The ideas to have more multiple services for retailers on their platform, yeah, to support them selling their products. And one of them was the CGI production service. Great. And then I noticed that you went to 747, which was a studio, like a virtual photography. And now you're at Accenture. Yeah, how did that work? What are your activities nowadays? Yeah, when I left Otto, I have never worked in a smaller companies. Like, Ogilvin made a Liubinette, Schwarzenfranc, Otto, all of them worked quite a large companies. And I wanted to see how it is to work in a, yeah, in a kind of smaller company and in a different kind of environment. And the 747 Studios has been one of the CGI vendors for Otto. And I've been in close contact with them. Even before I went to Otto, I've been in contact with them already when I was working at Schwarzenfranc. And they have very nice spirit within that studio. So every, yeah, the working together is, yeah, it's different there. And then Christian, the founder asked me if I, yeah, would love to join the 747 Studios to, on the one hand, help them scale the digital production department, but also help them to digitize the entire studio and processes and workflows and so on. And then back in the days, AI came into place. And then of course, it was also my responsibility to integrate AI workflows into existing, into existing production workflows. And yeah, that was super interesting. But at the same time, I recognized when you're working in the smallest studio, you have multiple jobs on your table that probably are not like in the super focus of your expertise. And then I recognized that this is probably not what I'm aiming for. So focusing more on what I'm destined is more what I wanted to do. And that's why I left the 747 Studios after 1.5 years or something. And yeah, I went to extension. And now I'm a consultant for scaling content productions with CGI with AI. Yeah. I'm part of the business development department over there. Well, yeah, and I would love to talk more about the AI part also later. And that sounds really interesting. And what you did at 747, it was great. Some years back now, everyone is talking about everyone's doing AI projects. I think you were among the first ones to actually execute there. And that was really cool to see. And also with Accenture, if I can say so, I saw this case study that you guys did with using Nvidia ARM Niverse and with Nestle, right? Like for Nestle. Yeah. Maybe you can share a bit more about that. I think there's some information out there about it already, but it would be great to just learn a little more like what did Nestle want and what did you guys help them achieve? Yeah, they worked with CGI or leverage the technology some years ago, but they always had the challenges that there were multiple CGI vendors using CGI for creating images, but they never really made it on one DCC and Brenda engine pipeline. And then Nvidia ARM Niverse came into place. And yeah, with the only way we were able to produce kind of a production platform where through OpenUSD, all the different vendors were able to publish their digital twins or the public, the digital twins they've created into the centralized production engine to automate their the production of the first place basic images. But yeah, now we're also leveraging digital twins for creating in the Nestle context. It's called advanced imagery. And now we're also looking into videos and see how we can use digital twins to also create videos or also to adapt content that is already existing and then using CGI and AI to adapt that to different market needs, bring them into different language classes and so on. And if I may ask because the term is new for me also like when they say advanced imagery, what is that? It's like basic and what is advanced imagery. Basic are those cut out or white studio images that you know from e-commerce or the P and advanced images is then a product in context. Hi, okay. And that's where I can help a lot with making an advanced image right, giving it some context, putting it on a desk or on a beach or wherever. Yeah, and Nestle has so many different brands. Also the quality expectation from the brands are quite different. There are some who just need an image and there are others like Nespresso, for example, or Star Wars or whatever, who really have the expectation of high end super-addirected imagery and there it's not that easy with AI just to produce any kind of good looking image, but then it should fit to the campaign look or the defined visual appearance of a brand and so on that makes it a little bit more challenging let's say. Okay, I would love to talk about that more later when we look at maybe also what's the future of 3D and CGI versus AI, how do they all play together, maybe or not, but then maybe just a step back. So if we look at CGI, what we need to create a nice rendering of a product is a digital twin of that product and we talked about it recently in the podcast basically trying to understand okay, how can I get to a digital twin in the first place, we understood that it are often even multiple digital twins, but they're side-old like in, let's say, product design or an engineering or then in marketing we have the problem at the end of the pipeline and we know that there is some kind of digital twin, but it's not in the form that we need it and we don't have expertise and the tooling to leverage it, etc. and then we basically outsource things to agencies, but just taking a step back like looking at digital twins for design and marketing, you work quite a bit in the industry right and I think you saw the trend there and what happened the past 20 years let's say in the design departments and there is this term I wanted to ask you about which I recently learned is called sample less production, so I have production without samples and what is it for product design, what is the old way basically and why could it make sense to use digital twin? When I imagine and this is already six years ago, seven years ago something where we made our first steps in exactly what you mentioned, sample less production for fashion at Otto where we had the departments that were involved in the design process for creating garments and they were started using tools like browser and flow 3D and so on, but only for making sure that the patterns were created and back in the days before they used browser or flow, they did that manually and then those patterns has been sent to China or wherever, Bangladesh, whatever or the production location was and there they did the first samples and then sends the stuff back to Germany, then some changes has been done on those patterns, then send them back to Beijing to China or wherever and this went back and forth for let's say three four five six times whatever until you have the final patterns on those patterns then the final product has been created and this process took eight, 12, 20 weeks or something and then after the final product has been manufactured, created, they have been sent to Germany and then you started with creating imagery, so this process 24 weeks, 30 weeks or whatever and now when you're doing everything digitally, you can start creating the marketing material, the imagery, the videos and that moment when you make your design decision and say okay this is the way how the shirt, the hoodie, whatever should look like and then you can immediately start creating images and also start selling your product on your website platform or whatever, probably the delivery time is a little bit longer but in that moment when the container arrives in Europe you already have sold everything that has been produced and yeah which makes time to market incredibly faster. So it makes so much sense. Yeah, absolutely. So basically in other words because you are designing in 3D you can start selling like you said before you even have the physical product. Exactly. But also and I think that's what you just said is basically doing the design phase, all these design iterations, I can shorten that drastically by just doing digitally, right? Yeah. Crazy. Yeah, this example, it makes sense. So what they were doing before if I get a practice really you have a suing pattern and you but you don't sue it in Germany which is like another topic, right? But then you send that to the factory which is far away and maybe like then by ship you get delivered the garment when it basically is together. Yeah, yeah, I can understand how that can take long and expensive, not good for the environment either. Okay, that's doing the design phase but you also set in for marketing purposes so, samtless production the opposite of that would be basically production where you have a physical sample. So let's say, or pornography, right? So you take pictures of an object to market it. But then when you use a digital twin, I think there's even like a mixed form, right? Where you have a card model or so or you have something but it's not final and then you still send either pictures of the real thing or the real thing, right? And people model it. Yeah, true. But the great thing about that is that those images doesn't need to look super glossy fine. You can do that wherever in the world with a cell phone in a storage or whatever or within the production assembly or yeah, doesn't make really sense. It's just for the artist to have some kind of visual reference and to see how probably details or things that are not integrated in the caps files look like or are yeah, just as a reference, which makes it like way more easy to produce those images. And you still don't send a physical sample that you just send the picture. So that makes sense. But nevertheless, when it comes to other products, where you have the separation between let's say the geometry and CG, the geometry and the label on top, which makes it quite difficult as well because those different pillars are in most of the time in different board pillars, because on the one hand, you have the design phase of the labels and then you have the marketing pillar where they need the caps files. But to create a digital twin, you need both the caps files for the geometry and the label files for the printing that comes out of the geometry. And that makes it hard because then you normally would need the printing in a certain standardized way, following some production standards that you've defined. And you don't get them because those people are at the beginning and the design says. So this is quite challenging also to get them in the right way and at the right time to be able to then for marketing purposes, automate production and get the images you need. That makes so much sense. So basically, there's still a lot of work to be done when you want to create a digital twin, especially when you have not been able to maintain it throughout the pipeline, right, from the design phase, which might be different in fashion where you may always look at the materials, right? But then when I don't know, produce a speaker or something like that, something that has a cat model and then the water bottle. Yeah, there's no digital twin throughout the pipeline, right? Yeah. But then, okay, let's assume we get there in their ways to get there, right? And like people investigate so many different approaches and yeah, often they work through agencies, but often they work through in-house, use really scanning, material scanning and so on. So when you have your digital twin, then there's this trend that I think you mentioned at some point is we see more and more enterprise leverage 3D also for content adaptation and localization. So basically not just having one version of a product and rendering it as is, how it would be with the photo, but actually doing more. And that is the really exciting part where we can actually play out its trends. So we spend a lot of time creating that 3D twin, but then what do we get for all of it? Yeah, that's true. That's true. There are so many brands or so many enterprises having a ton of images or videos laying in their dump and they would love to leverage that for different use cases for different countries for different markets, but they're not able because, for example, the label is in the wrong language, for example. And therefore it's super interesting to take a deeper look on how can we create systems that are able to exchange the product in those images or videos for different language classes, for example, or different flavors because there's a different flavor in the Mexican market, for example, and that's super interesting how to organize the production than those things can be can average, but also very challenging to find solutions that enable to that. I can see that it's very advanced. We were basically saying like the industry overall is still trying to figure out step one, which is how to get the digital twin, but instead, of course, still it takes more effort, but it's quite encouraging to see that there are so many options what you can do when you are able to dynamically adapt content based on a digital twin. I heard this term once about CAD, so I'm not a CAD engineer, but I know that there is this term of living CAD models versus dead CAD models. Maybe it's a little bit like that with the digital twins between that because it doesn't know that it has the label on it. It's just all being into a single texture, atlas, whatever. Or is it like a living digital twin where you can actually use it to customize your content real quick and localize it, adapt it and then generate millions of images and variations with the help of a 3D model? Emma, and there what we're talking about now is just the appearance, the visual appearance of a product, but not functionality. I think about the functionality comes into place where you have to make sure that the digital twin is movable, but only in the in the right way. And then you probably have dependencies between different objects within that product. If you move one part, the other moves as well to a certain degree and so on. There's still a lot to explore for us within the next years. Is that maybe why so fast are such a popular thing to be done with CGI? Because there are so many options. I don't know, I want to have the 3C or the 2C, then with the auto man or not and like backrests, headrest or not. That may one reason. Because there's so much you like that you can't impossibly resolve with photography so easily. Yeah, and it's quite expensive to get them or to produce them in the first place when you think about having a collection of a sofa series. And as you mentioned, it's available as a one-seater, two-seater, three-seater on the left side and so on. And this with multiple materials and you have to produce every of those sofas, ship them into the photo studio to make an image. This is, yeah, quite expensive and therefore it makes completely sense to do that. Interesting, yeah. So you mentioned basically also the life-cycle value of one product. I guess if you have a lot of small products, it's worth less than if you have one big product like a sofa, pace of more quickly. Cool. And so then looking a bit more at the AI world, you mentioned some work at Accenture already around what you do with Nestle there, for example. But then I also, other use cases where CGI and AI somehow are interconnected to play together. Yeah, there are because until now it's often the case when you use AI to create an image that involves a product that certain parts or even typography of the product will not be displayed in the correct manner. So the AI is hallucinating in a way and is changing those parts or things. And there it makes sense to have a combination of CGI digital twins and AI to leverage best of both worlds. Because then you can make sure that you have the creative freedom of using AI, but in combination with the rendering of a digital twin that makes sure that the product looks correct in the right way. And this is like a feasible way to have best of both worlds. Right, I think that makes perfect sense and we talked about that even in the last episode here on the show. So basically if I get it correctly using CGI or high-d precise representations of your product, we basically also have a built-in QA mechanism, right? You know that the model looks good and then you know that the rendering will look accurate like the renderer, a classical renderer will never hallucinate anything in the product, right? We can know that we're comfortable with that. So we're comfortable automating the rendering. But then like the other topic is do you what would that even be needed for if you're having a new product? So let's move back to sample as production and designing a new product. Why would I have generative AI? Like why would it even be able to generate pictures of that that doesn't make sense? It's designed in 3D unless the AI does my whole cat and that's another story. But absolutely. We are right now. Yeah. And also you want to have the product consistent through different touch points and different marketing materials. It should always look the same and doesn't vary in in a certain way. Yeah, so basically you get that accuracy and I get that this is needed because you want to avoid returns where people send back the product that's always super expensive, right? Like when you get returns. There are different kind of products, right? If you think about having a car or a sofa or a can of tuna, you want to have the sofa and the car probably to look correct. But when you talk about a can of tuna, then it's probably more the tuna that you're interested in and not the can and therefore it doesn't really make a difference if it looks correct or not. It's just the image is just yeah, if we equal to transport the information that it's about tuna and not that one for example. Yeah, that makes perfect sense. Yeah, and I think that's a really great example. The car is so different not in terms of price only but maybe we can find another example that costs a bit less but I don't know let's say. Yeah, so far. This is something so far sneaker exactly. The sneakers they have a huge aesthetic value, right? Of course, we need to have something on our feet usually that we don't catch a cold or whatever or run into a broken glass and then heard our says. But apart from this functional part, I think put where is a highly visual like fashion product, right? Then that's why it's super important that we get the details right and the aesthetics and in contrary, if they're not right, people say this doesn't do the job that it's supposed to do. It's supposed to look good. It doesn't have the exact color I want it. I'm sending it back. I don't like this color. It's too pinkish or too orangey for me or whatever. This can happen but with the tuna, it's more like whatever. If the box online looked like it was a white can and now it's actually like a tin can. If it's still tuna and it still costs the same and it's still the same amount inside it tastes good and whatever. I put the can away anyway. So yeah, like CPG versus anything that has an aesthetic value. That's already huge difference. Yeah, absolutely. Great, great. Yes, thanks. That was super insightful already. But then it gives us a little bit more of an angle, right? Where AI can also be a little more aggressive, let's say, with generating things versus where we need to be a little more careful and rely more on the digital twins and the accuracy. But then for AI in general, maybe that's question. Just thinking about the future, we're hearing a lot about AI bubble right now, people questioning the projections from some odd menu. It comes to open AI's business and like we see that there's more and more like a cool down that we can see on the horizon at least. I think everyone says AI is here to stay. Sure, no question. But is there still some kind of bubble? Is there some kind of hangover that you can see coming? Or how do you see AI in the future, especially in CGI and these kind of contact us? When we, you mentioned this like a minute ago that the AI normally doesn't have any clue of how the cat model looks. Imagine you would be able to ingest a cat model into an AI, cat model into an AI model and then the AI model knows exactly how the product looks like and how materials behave in terms of light shadows, reflections and so on and with that you would be able to create more or less anything content wise with that model. We're going that direction with Loras and so on. But yeah, it's not finally there and we're still seeing improvements in so many different ways with AI that I'm pretty sure that we're not final there. We will like within the next years still see a lot of large improvements on how AI will help us create content. When you take look on real video production for larger brands like we do for automotive industry, for example, we're still not there that we really are able to use AI for creating those videos because we're not able to adjust it in a way that we want then the client has some corrections and we're not able to really adjust those corrections like the client is used to. And I'm pretty sure there are some starting slides. We have control nets and all those things that makes it easier to adjust or control the content that is created. But I'm pretty sure we're still not there where we're going to be in the future. And they are super interesting. What will come next? Yeah. Yeah, that makes sense. Yeah, I'm also quite excited to see where everything's heading. I think we're understanding a lot more than we understood a couple years ago about how the giant AI work together and will maybe work together or like very long. But in the end, when I was working at 747 Studios and AI entered the game, there were some photographers who really went crazy because they thought in half a year, I do not have a job anymore because they saw all those fancy good looking images that has been created by 13 years old using AI. But there are still photo studios out there. There are still CG agencies creating images and I'm pretty sure that would stay also for the next 10 years. Maybe not in a way that like it has been before, but at the end, it's just another tool that we have in our toolbox right next to photography and CGI and whatever. That's a really great and I think very true statement. So with that last question, what did really also imply that when you're 3D artists and you're running a studio, when you learn AI tools right now, that's something you'd recommend to basically stay up to date and just stay relevant. And then as you said, just have another tool in your toolbox and then do that for the next decades. Yeah, because you can shortcut ways, right? When you normally would create a whole environment in CG that took you, I don't know, five days or whatever and now you can shortcut that and now it takes you half a day or even shorter, then it's always good to know how to handle those things. Perfect. Yeah, I think it makes perfect sense and just coming from an education background, we learned programming and then in the same topic, being discussed, hey, do we still need to learn programming? Oh, AI is going to replace all the programmers. Yeah, not exactly right. It's a context knowledge that makes it valuable, right? If you know what you're looking for to create good images, you know what a good image can look like, you know how lighting works and these kinds of things, then you're much better AI artists than the average chando, right? Yeah, and you learn those things in reality best even in a studio or on set and not in CG or in AI. So the old school way is a photographer basically. Yeah, that's also going to take away. Yeah, good. Yeah, thank you so much to be for a really insightful and very pleasant exciting conversation. Looking forward to our staying in touch and seeing what you guys are up to at Accenture. Cool. Thanks, Marks.