# EXR "Data window" nuke discrepancy in resolution

**URL:** <https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152>\
**Category:** Flame Questions\
**Created:** [October 26, 2021, 8:55pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152 "2021-10-26T20:55:37Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![miles](https://avatars.discourse-cdn.com/v4/letter/m/47e85d/32.png) [@miles](https://forum.logik.tv/u/miles)\
**Post date:** [October 26, 2021, 8:55pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/1 "2021-10-26T20:55:37Z")

</div>

Greetings. Anyone have any idea why importing a st map with the Data Window selected in Pixel space window would give a different resolution than the bounding box of the same st map in nuke. its between 2 and 4 pixels different. not a deal breaker, just curious. thanks

---

<div class="post-metadata">

**Author:** ![kyleobley](https://avatars.discourse-cdn.com/v4/letter/k/df788c/32.png) [@kyleobley](https://forum.logik.tv/u/kyleobley)\
**Post date:** [October 27, 2021, 1:12pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/2 "2021-10-27T13:12:16Z")

</div>

I’ve wondered the same thing before. Hopefully someone has an answer.

---

<div class="post-metadata">

**Author:** ![randy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/randy/32/6122_2.png) [@randy](https://forum.logik.tv/u/randy)\
**Post date:** [October 27, 2021, 1:26pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/3 "2021-10-27T13:26:33Z")

</div>

I feel like if anyone might know it’d be @milanesa . ?

---

<div class="post-metadata">

**Author:** ![milanesa](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@milanesa](https://forum.logik.tv/u/milanesa)\
**Post date:** [October 27, 2021, 2:44pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/4 "2021-10-27T14:44:50Z")

</div>

Hard to know without a test image. A quick test reveals not difference in data window values with the same image in Nuke and Flame:

 ![image](https://us1.discourse-cdn.com/flex020/uploads/thedepartmentofexternalservices/original/2X/3/34525a59bbf8cda33de0087c1a57674a09bfbd55.jpeg)

---

<div class="post-metadata">

**Author:** ![kyleobley](https://avatars.discourse-cdn.com/v4/letter/k/df788c/32.png) [@kyleobley](https://forum.logik.tv/u/kyleobley)\
**Post date:** [October 28, 2021, 4:23am UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/5 "2021-10-28T04:23:45Z")

</div>

In my experience you find it with STMaps from the lens distortion node.

---

<div class="post-metadata">

**Author:** ![PlaceYourBetts](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/placeyourbetts/32/6852_2.png) [@PlaceYourBetts](https://forum.logik.tv/u/PlaceYourBetts)\
**Post date:** [October 28, 2021, 4:25am UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/6 "2021-10-28T04:25:18Z")

</div>

I only just found out that Flame could see this **Data Window** recently and we have been going back into our distort workflow and taking advantage of it.

**Display Window** vs **Data Window** 😱 Who knew!

I can’t say I have noticed the difference but I will take another look.

---

<div class="post-metadata">

**Author:** ![johnt](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/johnt/32/14040_2.png) [@johnt](https://forum.logik.tv/u/johnt)\
**Post date:** [October 28, 2021, 6:56am UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/7 "2021-10-28T06:56:52Z")

</div>

It’s an absolute fuck on. From memory the data window can change in Nuke but in flame it will get set by the size of the data window at the first frame.

---

<div class="post-metadata">

**Author:** ![PlaceYourBetts](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/placeyourbetts/32/6852_2.png) [@PlaceYourBetts](https://forum.logik.tv/u/PlaceYourBetts)\
**Post date:** [October 28, 2021, 7:00am UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/8 "2021-10-28T07:00:19Z")

</div>

Good to know.

Yeah @miles I just checked back and both seem to match for a single frame STmap

---

<div class="post-metadata">

**Author:** ![miles](https://avatars.discourse-cdn.com/v4/letter/m/47e85d/32.png) [@miles](https://forum.logik.tv/u/miles)\
**Post date:** [October 28, 2021, 2:31pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/9 "2021-10-28T14:31:43Z")

</div>

These were provided by a vendor:  
“[Dropbox - stmaps - Simplify your life](https://www.dropbox.com/sh/9p304yfc0qvhg0p/AAB5kgX-5p0MR42y5UQR62KBa?dl=0)”

nuke and the vendor undistort to 2404x1654  
flame undistorts to 2408x1658

also round tripping these have proved a bit of a puzzle, I’ve found a hack but if I undistort using the stmap to an oversized image and redistort back to the working rez 2224x1548 they don’t match. Nuke has no issues as retains it’s bounding box throughout. Curious to see what folks can find.

---

<div class="post-metadata">

**Author:** ![milanesa](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@milanesa](https://forum.logik.tv/u/milanesa)\
**Post date:** [October 28, 2021, 3:27pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/10 "2021-10-28T15:27:57Z")

</div>

Wondering what’s the STMap undistortion/distortion workflow in Flame. Never done before.

---

<div class="post-metadata">

**Author:** ![miles](https://avatars.discourse-cdn.com/v4/letter/m/47e85d/32.png) [@miles](https://forum.logik.tv/u/miles)\
**Post date:** [October 28, 2021, 3:37pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/11 "2021-10-28T15:37:46Z")

</div>

The UV node is just a shader I wrote, but Lewis has one, and there’s a couple more. You can also use action. This is what has worked for me every time until now. mystery.

 ![Screen Shot 2021-10-28 at 8.33.44 AM](https://us1.discourse-cdn.com/flex020/uploads/thedepartmentofexternalservices/original/2X/e/e0510342ea051c821e87d78c6c27fa4ca3622b97.png)

---

<div class="post-metadata">

**Author:** ![milanesa](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@milanesa](https://forum.logik.tv/u/milanesa)\
**Post date:** [October 28, 2021, 5:58pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/12 "2021-10-28T17:58:53Z")

</div>

What I have seen as a good practice is to avoid STMaps with Data windows at resolutions other than the display window. Nuke handles this discrepancy nicely, but other apps don’t.  
For extreme pin-cushion type of distortion may be also good to work on a higher un-distorted resolution to keep the filtering to a minimum when re-distorting.

---

<div class="post-metadata">

**Author:** ![milanesa](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@milanesa](https://forum.logik.tv/u/milanesa)\
**Post date:** [October 28, 2021, 7:35pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/13 "2021-10-28T19:35:27Z")

</div>

> [@miles](#):
>
> This is what has worked for me every time until now. mystery.

Taking a closer look at your re-distort stamp. It has negative values around the edges of the image, which means that is trying to pull information from outside the 0-1 UV range. For this to work pixels information have to be present outside the boundaries of the Display Window. The mystery may boil down to Flame not been able to handle those out of bounds ranges.

---

<div class="post-metadata">

**Author:** ![miles](https://avatars.discourse-cdn.com/v4/letter/m/47e85d/32.png) [@miles](https://forum.logik.tv/u/miles)\
**Post date:** [October 28, 2021, 7:40pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/14 "2021-10-28T19:40:59Z")

</div>

Thanks for checkin it out. I noticed those values as well, how would nuke treat those values, perhaps I can rewrite my shader to do the same thing. Thanks again.

---

<div class="post-metadata">

**Author:** ![milanesa](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@milanesa](https://forum.logik.tv/u/milanesa)\
**Post date:** [October 28, 2021, 8:09pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/15 "2021-10-28T20:09:48Z")

</div>

If Flame doesn’t allow access to pixels outside the resolution format (negative UV space) you may be out of luck. Your new shader may need to normalize (and maybe apply a couple other transformations) the STMap values before applying the distortion.  
Alternatively you can regenerate new 0-1 range STMaps in Nuke using the ones your posted.

---

<div class="post-metadata">

**Author:** ![milanesa](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@milanesa](https://forum.logik.tv/u/milanesa)\
**Post date:** [October 28, 2021, 8:24pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/16 "2021-10-28T20:24:44Z")

</div>

Not sure if it will work in your case, but If you have Nuke try this:

```auto
Read {
 inputs 0
 file_type exr
 file redistort.1001.exr
 format "2224 1548 0 0 2224 1548 2 "
 origset true
 name Read2
 selected true
 xpos 2
 ypos -288
}
push $cut_paste_input
Reformat {
 format "2224 1548 0 0 2224 1548 2 "
 name Reformat1
 selected true
 xpos -108
 ypos -346
}
Expression {
 expr0 (x+0.5)/width
 expr1 (y+0.5)/height
 name Expression1
 selected true
 xpos -108
 ypos -314
}
Transform {
 scale {{parent.Transform2.scale}}
 center {1112 774}
 invert_matrix true
 name Transform4
 selected true
 xpos -108
 ypos -290
}
STMap {
 inputs 2
 uv rgb
 name STMap2
 selected true
 xpos -108
 ypos -259
}
set N38261c00 [stack 0]
Dot {
 name Dot1
 selected true
 xpos -74
 ypos 101
}
Read {
 inputs 0
 file_type exr
 file undistort.1001.exr
 format "2224 1548 0 0 2224 1548 2 "
 origset true
 name Read1
 selected true
 xpos -347
 ypos -450
}
Shuffle2 {
 fromInput1 {{0} B}
 fromInput2 {{0} B}
 mappings "4 rgba.red 0 0 rgba.red 0 0 rgba.green 0 1 rgba.green 0 1 rgba.blue 0 2 rgba.blue 0 2 white -1 -1 rgba.alpha 0 3"
 name Shuffle1
 selected true
 xpos -347
 ypos -313
}
Transform {
 scale 0.922
 center {1112 774}
 name Transform2
 selected true
 xpos -347
 ypos -289
}
Crop {
 box {0 0 2224 1548}
 reformat true
 crop false
 name Crop1
 selected true
 xpos -347
 ypos -245
}
set N38263800 [stack 0]
CheckerBoard2 {
 inputs 0
 format "2224 1548 0 0 2224 1548 2 "
 name CheckerBoard1
 selected true
 xpos -514
 ypos -26
}
STMap {
 inputs 2
 uv rgb
 blur_scale 0
 name STMap3
 selected true
 xpos -347
 ypos -6
}
STMap {
 inputs 2
 uv rgb
 name STMap1
 selected true
 xpos -347
 ypos 94
}
push $N38263800
Write {
 file undistort_flame.1001.exr
 file_type exr
 datatype "32 bit float"
 metadata "no metadata"
 first_part rgba
 version 1
 name Write1
 selected true
 xpos -242
 ypos -95
}
push $N38261c00
Write {
 file redistort_flame.1001.exr
 file_type exr
 datatype "32 bit float"
 metadata "no metadata"
 first_part rgba
 version 1
 name Write2
 selected true
 xpos -32
 ypos -95
}

```

Again, the undistortion/redistortion will apply softening to your comp, so maybe you can try the STMaps at a higher resolution.

---

<div class="post-metadata">

**Author:** ![kyleobley](https://avatars.discourse-cdn.com/v4/letter/k/df788c/32.png) [@kyleobley](https://forum.logik.tv/u/kyleobley)\
**Post date:** [October 29, 2021, 3:38am UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/17 "2021-10-29T03:38:25Z")

</div>

Someone shared the trick to getting this to work. I forget who, sorry. Anyways, there’s a way to get it work which I thought was pretty clever. I’m not in front of a box for a while so I’ll just have to try to explain.

You have to do the re-distortion in Action.

- Input 1: Color frame or whatever that is the original resolution
- Input 2: Undistorted plate (higher resolution)
- Input 3: Redistortion STMap

You then use input 1 as the main surface and then use input 2 as a diffuse and input 3 as a UV Map. This then allows Flame to recognize the negative values and map them appropriately.

Works a treat. I wish I could remember exactly who came up with it because it’s ingenious.

---

<div class="post-metadata">

**Author:** ![cnoellert](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/cnoellert/32/559_2.png) [@cnoellert](https://forum.logik.tv/u/cnoellert)\
**Post date:** [October 29, 2021, 4:38am UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/18 "2021-10-29T04:38:18Z")

</div>

I’d like to thank the academy… 😂

---

<div class="post-metadata">

**Author:** ![milanesa](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@milanesa](https://forum.logik.tv/u/milanesa)\
**Post date:** [October 29, 2021, 6:14am UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/19 "2021-10-29T06:14:55Z")

</div>

Got it!

 ![image](https://us1.discourse-cdn.com/flex020/uploads/thedepartmentofexternalservices/original/2X/6/68a0dc52960a4fa1817d90dac27eb24b7ed15fe5.jpeg)  
Thank you @kyleobley

---

<div class="post-metadata">

**Author:** ![cnoellert](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.logik.tv/cnoellert/32/559_2.png) [@cnoellert](https://forum.logik.tv/u/cnoellert)\
**Post date:** [October 29, 2021, 2:09pm UTC](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152/20 "2021-10-29T14:09:15Z")

</div>

@milanesa it looks like @kyleobley gave you the gist but in case I made a 2022 arch a while back for @Brooks when he was having similar issues…

> [@Creating Lens Distort Presets](https://forum.logik.tv/t/creating-lens-distort-presets/1325/19):
>
> Hey All, [cnoellert](https://forum.logik.tv/u/cnoellert) Christopher Noellert sent me this archive, that has a little more instructions on the workflow! It’s a huge help! and works with ST maps too! Flame2022 only at the moment, sorry, it’s all. I have.

[Next page](https://forum.logik.tv/t/exr-data-window-nuke-discrepancy-in-resolution/4152.md?page=2)