Was working on some updates to the shaders to make them usable in the Image node. Which lead me down to understanding the designation of the SelectiveFX shader category.
Turns out a SelectiveFX is a shader that can be used inside the Selective Node in Batch, or inside the Image Node in TL-FX as a child of a selective. The selective matte is passed in as an additional and optional attribute.
The documentation does indicate that this additional selective matte should be treated as a strength modulator, not just a matte multiplier. So on the blur effect, the selective matte should actually affect the blur radius, not just limit the effect to the non-black parts of the selective. Good to know, and also good to keep in mind when writing shaders.
I ran into one problem, is that there must have been a change in the API, as even the example SelectiveFX shader by ADSK has a runtime error on texture binding. I filed an issue to see if we can clarity of what changed. It only affects multi-pass shaders, it work fine for single-pass shaders.
You can use non-SelectiveFX shaders inside Image as well, but they will treat the selective matte only as a matte, not a strength modulator. And vice versa, SelectiveFX can be used outside of Image, and the selective matte will just default to white as no-op.
Slightly OT, but I just discovered that you can attach a selective to the camera in Action and it affects all surfaces in the comp. Kind of like what Camera FX was trying to do.
Real shame they never brought camera fx to a higher level of development. There is still no way to apply an effect to the scene in Action, but easily include or exclude objects. That would be a really big deal.
@fredwarren, is there any appetite to look into this?
Essentially the solve here would be that you attach a selective to the camera, then can gesturally include or exclude objects like gmask-linking.
That last piece of the puzzle would transform the capabilities of Action.
With Camera fx, to include or exclude something you had to go through some weird Object ID menu which was difficult and not instinctual. Camera FX was never realized into something that people use, similar to Lightbox. Great potential; just one round of development then forgotten.