EXR "Data window" nuke discrepancy in resolution

Greetings. Anyone have any idea why importing a st map with the Data Window selected in Pixel space window would give a different resolution than the bounding box of the same st map in nuke. its between 2 and 4 pixels different. not a deal breaker, just curious. thanks

1 Like

I’ve wondered the same thing before. Hopefully someone has an answer.

I feel like if anyone might know it’d be @milanesa . ?

Hard to know without a test image. A quick test reveals not difference in data window values with the same image in Nuke and Flame:

In my experience you find it with STMaps from the lens distortion node.

1 Like

I only just found out that Flame could see this Data Window recently and we have been going back into our distort workflow and taking advantage of it.

Display Window vs Data Window :scream: Who knew!

I can’t say I have noticed the difference but I will take another look.

It’s an absolute fuck on. From memory the data window can change in Nuke but in flame it will get set by the size of the data window at the first frame.

1 Like

Good to know.

Yeah @miles I just checked back and both seem to match for a single frame STmap

These were provided by a vendor:
Dropbox - stmaps - Simplify your life

nuke and the vendor undistort to 2404x1654
flame undistorts to 2408x1658

also round tripping these have proved a bit of a puzzle, I’ve found a hack but if I undistort using the stmap to an oversized image and redistort back to the working rez 2224x1548 they don’t match. Nuke has no issues as retains it’s bounding box throughout. Curious to see what folks can find.

Wondering what’s the STMap undistortion/distortion workflow in Flame. Never done before.

The UV node is just a shader I wrote, but Lewis has one, and there’s a couple more. You can also use action. This is what has worked for me every time until now. mystery.

What I have seen as a good practice is to avoid STMaps with Data windows at resolutions other than the display window. Nuke handles this discrepancy nicely, but other apps don’t.
For extreme pin-cushion type of distortion may be also good to work on a higher un-distorted resolution to keep the filtering to a minimum when re-distorting.

Taking a closer look at your re-distort stamp. It has negative values around the edges of the image, which means that is trying to pull information from outside the 0-1 UV range. For this to work pixels information have to be present outside the boundaries of the Display Window. The mystery may boil down to Flame not been able to handle those out of bounds ranges.

Thanks for checkin it out. I noticed those values as well, how would nuke treat those values, perhaps I can rewrite my shader to do the same thing. Thanks again.

If Flame doesn’t allow access to pixels outside the resolution format (negative UV space) you may be out of luck. Your new shader may need to normalize (and maybe apply a couple other transformations) the STMap values before applying the distortion.
Alternatively you can regenerate new 0-1 range STMaps in Nuke using the ones your posted.

Not sure if it will work in your case, but If you have Nuke try this:

Read {
 inputs 0
 file_type exr
 file redistort.1001.exr
 format "2224 1548 0 0 2224 1548 2 "
 origset true
 name Read2
 selected true
 xpos 2
 ypos -288
}
push $cut_paste_input
Reformat {
 format "2224 1548 0 0 2224 1548 2 "
 name Reformat1
 selected true
 xpos -108
 ypos -346
}
Expression {
 expr0 (x+0.5)/width
 expr1 (y+0.5)/height
 name Expression1
 selected true
 xpos -108
 ypos -314
}
Transform {
 scale {{parent.Transform2.scale}}
 center {1112 774}
 invert_matrix true
 name Transform4
 selected true
 xpos -108
 ypos -290
}
STMap {
 inputs 2
 uv rgb
 name STMap2
 selected true
 xpos -108
 ypos -259
}
set N38261c00 [stack 0]
Dot {
 name Dot1
 selected true
 xpos -74
 ypos 101
}
Read {
 inputs 0
 file_type exr
 file undistort.1001.exr
 format "2224 1548 0 0 2224 1548 2 "
 origset true
 name Read1
 selected true
 xpos -347
 ypos -450
}
Shuffle2 {
 fromInput1 {{0} B}
 fromInput2 {{0} B}
 mappings "4 rgba.red 0 0 rgba.red 0 0 rgba.green 0 1 rgba.green 0 1 rgba.blue 0 2 rgba.blue 0 2 white -1 -1 rgba.alpha 0 3"
 name Shuffle1
 selected true
 xpos -347
 ypos -313
}
Transform {
 scale 0.922
 center {1112 774}
 name Transform2
 selected true
 xpos -347
 ypos -289
}
Crop {
 box {0 0 2224 1548}
 reformat true
 crop false
 name Crop1
 selected true
 xpos -347
 ypos -245
}
set N38263800 [stack 0]
CheckerBoard2 {
 inputs 0
 format "2224 1548 0 0 2224 1548 2 "
 name CheckerBoard1
 selected true
 xpos -514
 ypos -26
}
STMap {
 inputs 2
 uv rgb
 blur_scale 0
 name STMap3
 selected true
 xpos -347
 ypos -6
}
STMap {
 inputs 2
 uv rgb
 name STMap1
 selected true
 xpos -347
 ypos 94
}
push $N38263800
Write {
 file undistort_flame.1001.exr
 file_type exr
 datatype "32 bit float"
 metadata "no metadata"
 first_part rgba
 version 1
 name Write1
 selected true
 xpos -242
 ypos -95
}
push $N38261c00
Write {
 file redistort_flame.1001.exr
 file_type exr
 datatype "32 bit float"
 metadata "no metadata"
 first_part rgba
 version 1
 name Write2
 selected true
 xpos -32
 ypos -95
}

Again, the undistortion/redistortion will apply softening to your comp, so maybe you can try the STMaps at a higher resolution.

Someone shared the trick to getting this to work. I forget who, sorry. Anyways, there’s a way to get it work which I thought was pretty clever. I’m not in front of a box for a while so I’ll just have to try to explain.

You have to do the re-distortion in Action.

  • Input 1: Color frame or whatever that is the original resolution
  • Input 2: Undistorted plate (higher resolution)
  • Input 3: Redistortion STMap

You then use input 1 as the main surface and then use input 2 as a diffuse and input 3 as a UV Map. This then allows Flame to recognize the negative values and map them appropriately.

Works a treat. I wish I could remember exactly who came up with it because it’s ingenious.

4 Likes

I’d like to thank the academy… :joy:

2 Likes

Got it!


Thank you @kyleobley

@milanesa it looks like @kyleobley gave you the gist but in case I made a 2022 arch a while back for @Brooks when he was having similar issues…

1 Like