Unreal camera w dynamic lens distort

https://vimeo.com/886032310?share=copy

Reliable workflow to produce fast, interactive results.
Dynamic distortion based on focal distance.
Converted the ST maps to K1, K2, K3, P1, P2 values so it runs fast.
Also saves needing to reference 4.7 GB contained in 43 32 bit EXRs for this one lens.
Focus pull mapped in lens object as well.

Next up: applying to first set of 14 lenses with good sets of grids at multiple focal distances.

Here’s a test lens object to try out in Unreal 5.3 or later.

5 Likes

Not sure how I’ve managed to do Flame for 25+ years and lots of different kind of comps, and I’ve never dealt with or thought about lens distortion science or ST maps.

1 Like

Word up man.
Hope all is well.

Tracking team was solid and kept you in the dark?
If using cam tracks, deliverables include STmaps generated from solve.

Point of my pointless exercise is push interactive tools like Unreal that can translate to flame quality.
Drawn to Flame back in the Long Long Ago for similar reasons. Color too. Interactive tools are a lot of fun.

This is a by-product of my RnD into datasets, cinema lenses in this case. Large set of verifiable data with a handful of set variables.

Data workflow btw unreal, flame and houdini based from the real world lenses. Map the rotation response curve and the tool not only looks right, but feels right when used.

CSV data out of Shotgun and into Houdini.

Cheers-
Andy D. (the other one)


4 Likes

Nice stuff! I put in a flame FR for something like this a while back in the hopes that action’s camera could natively apply lens distortion to all the objects inside (instead of a downstream ST map).

Unreal is, as always, very impressive.

Think what’s great about how unreal implemented the lenses are as an object that attaches and over-rides the settings on the camera. Distortion, sensor size, focus and iris pulls are all embedded in one small package. Wish list would include adding aperture blades, focus min lock, bokeh kernel and sone way of addressing flares/glares which is usually heavily influenced by coatings.

Hope you and yer peeps are well pal.

Andy D. (West -side)

3 Likes

If you’re just doing projection cleanups you can get away without them most of the time, but if you want to put cg into a tracked scene that resembles the real world, then dealing with lens distortion is a must.

1 Like

@imag4media Sorry to necro this thread, but I’m super lost even after taking a look at the demo object you generously provided.

  • How specifically did you convert the ST maps into distortion parameters? I see you mentioned that you used Unreal, Flame, and Houdini, but how?
  • What value are you changing in the video you posted? I see that Eval Inputs → Focus is changing, but it doesn’t look like your CineCameraActor’s focus is changing?
  • You’re using ST maps you generated yourself, rather than the ones supplied by Zeiss through the CinCraft Mapper application, correct? If so, why not just generate the distortion parameters directly instead of creating an ST map?

Ok.

Zeiss app allowed me to call for st maps manually and I could capture 50+ distances per lens in many cases.

The stmaps I could verify on the vfx side for accuracy.

3dEqualizer has the ability to convert to raw distortion values which in the case of SPH lenses, usually boil down to k1 and k2 values. (Distort and quartic). The other 3 values (k3, p1 and p2) are near zero most of the time unless the mount isn’t aligned correctly.

The raw values can be entered into unreal to be mapped to distance. So each lens is mapped from focal min to infinity. (Open to close)
This is much more interactive than relying on 50 32bit ST maps to blend between in real time.

Houdini is my ground truth for cg. Unreal is a great app but it’s very Wild West. Placing the same data into Houdini allowed me to map points along a curve that was generated by calculating the focal distance / (focal distance - focal length)
This gives the falloff to infinity for any lens.

Houdini and flame can also use the raw distortion values instead of ST maps. Interactivity is much improved this way. I could bring in my raw values into flame then verify that it was working correctly by using the original st map to check it.

Flame doesn’t currently have a way to create a data stream like a lens model so I used timelines to map distortion and vignette as well as metadata over the distance throw of the lens.

Flame 2025 adopted the 3de model to create their new lens distortion tool which made it easier to apply the same data cleanly.

Depending on the lens, the dynamic distortion may be more/less obvious.
My recent post with the flame 2025 setups might not be so obvious with the lens breathing because they were longer lenses in this quick demo.

https://vimeo.com/933572088

The distortion is still present and dynamic as you can see w the values in the UI.

The Dropbox link on that clip includes a couple of wider lenses which should show that effect more prominently.

Hope that helps?
A

1 Like

the dynamic distortion in unreal is more apparent in this clip:
https://vimeo.com/914920379

All that’s being adjusted is the focus distance.

1 Like

Fantastic, thanks. That makes a ton more sense (even if I’m definitely still out of my depth :sweat_smile:)

  • Was there a specific guide you followed for using 3DEqualizer to convert the ST maps to distortion values? I didn’t see anything immediately in the docs (though I’m sure I’m just missing it.)
  • Do you happen to already have .ulens files for any of the Zeiss Supreme Prime Radiance lenses, particularly the 25mm, the 29mm, the 35mm, and/or the 50mm? I saw the ST maps you posted on your website, but if you happen to already have .ulens files, that would save me a ton of effort (particularly if they’re already set up for the FX6/FX9).