Stencil params getting out of sync.

[DVLP]

Description

Calls to stencil functions like gl.stencilFunc, gl.stencilOp or gl.stencilMask are ignored when gl.STENCIL_TEST is not enabled. That's how it works in webgl (Edit: no it doesn't, I was mislead by AI which lead me to a long state cache modifying misadventure and the real issue is in the comments). Regardless of that WebGLState.js is caching these values as current. preventing them from being set correctly when the stencil buffer is enabled afterwards and the value is the same as cached.

Reproduction steps

Rendering an empty scene should be enough to reproduce the issue.
Renderer clears the frame and calls this.state.buffers.stencil.setMask( 0xffffffff );

At this point the cached currentStencilMask will be set to 0xffffffff but the native call to gl.stencilMask is ignored because gl.STENCIL_TEST is not enabled. The state is now out of sync and subsequent calls with the same value will be ignored because currentStencilMask will match them.

1. in WebglState.js under line currentStencilMask = stencilMask; add a new line console.log('Stencil mask cached correctly', gl.getParameter(gl.STENCIL_WRITEMASK) === currentStencilMask)
2. Create a basic scene, run and look in the console

Code

Live example

No live example, requires core modification

Screenshots

No response

Version

164

Device

Desktop

Browser

Chrome

OS

Windows

[Mugen87]

Calls to stencil functions like gl.stencilFunc, gl.stencilOp or gl.stencilMask are ignored when gl.STENCIL_TEST is not enabled.

Why does the following fiddle work then though? Stencil test is disabled but using gl.stencilMask() is not ignored.

https://jsfiddle.net/aw8c3d2j/

[DVLP]

I tried this code and indeed it worked. This is very interesting as I've been testing this with a debugger in a larger environment and gl.stencilMask was not persisting the setting. I'm working on a library to synchronize webgl context state with an ability to save and restore a snapshot. For one of the blocks of code GPT explained it requires enabling stencil test to set stencil parameters. Then I checked if that's the case, and it appeared it was. If I'll be able to reproduce this issue with gl.stencilMask not persisting the state and finding what's the cause I'll reopen.

  var stencilTestEnabled = gl.state.feats[gl.STENCIL_TEST]
  if (!stencilTestEnabled) gl.enable(gl.STENCIL_TEST)
  gl.stencilFunc(s.stencilFunc[0], s.stencilFunc[1], s.stencilFunc[2])
  gl.stencilOp(s.stencilOp[0], s.stencilOp[1], s.stencilOp[2])
  gl.stencilMask(s.stencilMask)
// ...
  if (!stencilTestEnabled) gl.disable(gl.STENCIL_TEST)

[DVLP]

Ok I managed to reproduce the issue. It's not about stencil test being on or off but but Chrome not settnig the value as 0xFFFFFFFF (max 32 bit value) but changing it to 0x7FFFFFFF (max 31 bit value). It seems to be the highest value it accepts. When setting the value as 0x80000000 it also changes it to 0x7FFFFFFF. Could be a bug or limitation in Chrome or just how it works. Try in Firefox and then Chrome on Windows.
https://jsfiddle.net/d6fqme4t/

[Mugen87]

I can reproduce on macOS as well. When logging the values to the console, Firefox shows 4294967295 (decimal representation of 0xFFFFFFFF). Chrome (and Safari) log 2147483647 (decimal representation of 0x7FFFFFFF).

According to the spec, mask is a GLuint which has 32 bit. It's max value is 4294967295 so my first impression is the behavior in Chrome isn't spec conform.

Since we use 0xffffffff as a clear value in WebGLRenderer.clear(), that explains the root cause.

[DVLP]

In this case it's not a Three issue but Chrome. Interesting stuff.

[Mugen87]

I have filed a Chromium bug: https://issues.chromium.org/issues/338634235

[Mugen87]

As pointed out in the Chromium bug, using 0xffffffff in context of gl.stencilMask() is safe but relying on the value returned by gl.getParameter() is not due to the different browser behavior. However, the engine does not use gl.getParameter() to evaluate the stencil mask so there should not be any side effects.

There are no plans to apply any changes to three.js because of the Chromium bug. However, developers who work with the raw GL context and use gl.getParameter() to evalute the stencil mask must be aware of this issue. This thread is going to be closed though. Please continue the discussion at the Chromium bug tracker if necessary.

[DVLP]

Makes sense. Thanks for investigating and submitting a Chrome bug.