Fix resolvable depth difference calculation for triangle near 0
When all three vertices of a triangle have depth 0, the maximum exponent
e is 0. We can't just use (e - n) << n as the bit pattern for 2^(e - n),
because the exponent field underflows and creates a negative value.
All triangles where all depth values have a bias-adjusted exponent less
than 23 have this issue (note 2^(23 - 127) = 4.9e-32).
The fix is to perform 2^(e - n) as 2^e * 2^(-n). The multiplication
handles the underflow of the exponent either by creating a subnormal, or
zero.
Bug: b/139341727
Change-Id: I44b87feb55f61c5fa18ba235e9ec211926de3b3e
Fixes: b/174051829
Reviewed-on: https://swiftshader-review.googlesource.com/c/SwiftShader/+/51148
Kokoro-Result: kokoro <noreply+kokoro@google.com>
Tested-by: Nicolas Capens <nicolascapens@google.com>
Reviewed-by: Alexis Hétu <sugoi@google.com>
Commit-Queue: Nicolas Capens <nicolascapens@google.com>
diff --git a/src/Pipeline/SetupRoutine.cpp b/src/Pipeline/SetupRoutine.cpp
index 5ed9b9c..a360184 100644
--- a/src/Pipeline/SetupRoutine.cpp
+++ b/src/Pipeline/SetupRoutine.cpp
@@ -466,7 +466,7 @@
Int e = Max(Max(e0, e1), e2);
- r = As<Float>(e - (23 << 23));
+ r = As<Float>(e) * Float(1.0f / (1 << 23));
}
bias = r * *Pointer<Float>(data + OFFSET(DrawData, constantDepthBias));
