When Is Shot Designer 3D Being Released?

[andretittle]

Per,

You’ve been posting little snippets of information about Shot Designer 3D on this forum, and on the previous iteration of this forum, for quite a while now. Over two years.

It truly sounds amazing.

I am very much looking forward to using it.

However, when you also keep posting something like “coming late this year,” and the calendar keeps turning, that is frustrating.

I imagine you’re trying to squeeze in every possible bit of capability before you release it.

But I don’t think AI’s ever going to be standing still.

Do you have a realistic release date?

If yes, what is it?

If not, I suggest you stop telling this community we’re going to see this amazing app late this year.

Maybe this is good marketing to generate interest.

I find it frustrating.

[perholmes]

Hi,

I’m glad that you find it frustrating, actually. Much better than indifferent.

It’s not about squeezing as much AI into as possible, it’s about the fact that once you have 3D (and/or AI), your demand on what you want these characters to do goes up dramatically. You think you don’t care, but once you’re filming in a real environment, it will take you exactly 10 seconds to demand much more.

At the very least, you need characters worth filming, so this is why they have to walk, talk, gesture, etc. But we’re doing some major compromises in terms of e.g. a character designer, that will come later. And in terms of the richness of walking and gesturing, 1.0 will be only survivable. There’s an endless list of things humans can do and that you will want them to do in front of a camera, and we’ll only support 5% of it at the start, and then grow into it. We’ll also launch with only barebones locations and props. It will be embarrassing.

But in terms of the AI rendering, this is a massively important feature, because it brings it home completely. With the 3D guidance, you can start making something that looks like a real movie. From our perspective, this is actually a rather lightweight feature. It’s more about having the data collection directly in the app (character sheets, AI location looks as seen from different angles), and we already have the jobs running that take any given shot over the AI for rendering and brings it back into the app for playback. This can be infinitely tuned later, we just want a barebones version of it from the beginning, or it’s not clear what the app is.

But making the actors move is the very hard part, and where AI video generation gets a lot for free, we’re all on our own if we want to control it, and that means human motion synthesis. I wasted a full 5 months on chasing the wrong path. We’re back to using Unreal’s motion matching for all locomotion, an online AI model (NVidia Audio2Face) for lipsync, and our own LLM driven gesture system. Then you can make the characters walk and talk in a way that you’ll actually want to film, and I don’t think the app can launch without it.

Here’s screenshots to show progress: