Fix write stencil ops when writeMask == 0
We were checking whether we need to apply the write mask by erroneously
looking at whether the writeMask was non-zero. This meant that when the
write mask was zero, we would compute the stencil value, and store
it, rather than store 0. Instead, what we mean is to apply the write
mask if masking does something: when it's not FF (or 2^n-1, with
n = number of stencil bits).
Bug: b/141484021
Change-Id: I6134289ac8bb13a3502884e4e408f71961814adf
Reviewed-on: https://swiftshader-review.googlesource.com/c/SwiftShader/+/36848
Reviewed-by: Nicolas Capens <nicolascapens@google.com>
Kokoro-Presubmit: kokoro <noreply+kokoro@google.com>
Tested-by: Antonio Maiorano <amaiorano@google.com>
diff --git a/src/Pipeline/PixelRoutine.cpp b/src/Pipeline/PixelRoutine.cpp
index ce85f3b..9cdfc1f 100644
--- a/src/Pipeline/PixelRoutine.cpp
+++ b/src/Pipeline/PixelRoutine.cpp
@@ -718,7 +718,7 @@
Byte8 newValue;
stencilOperation(newValue, bufferValue, state.frontStencil, false, zMask, sMask);
- if(state.frontStencil.writeMask != 0)
+ if((state.frontStencil.writeMask & 0xFF) != 0xFF) // Assume 8-bit stencil buffer
{
Byte8 maskedValue = bufferValue;
newValue &= *Pointer<Byte8>(data + OFFSET(DrawData,stencil[0].writeMaskQ));
@@ -730,7 +730,7 @@
stencilOperation(newValueBack, bufferValue, state.backStencil, true, zMask, sMask);
- if(state.backStencil.writeMask != 0)
+ if((state.backStencil.writeMask & 0xFF) != 0xFF) // Assume 8-bit stencil buffer
{
Byte8 maskedValue = bufferValue;
newValueBack &= *Pointer<Byte8>(data + OFFSET(DrawData,stencil[1].writeMaskQ));
