tree 3ccb66a071ec9767df14730045cd12530ff9e622
parent 0a64a974755521dbb2b16332002a927fb830a0d4
author Nicolas Capens <capn@google.com> 1607628613 -0500
committer swiftshader-scoped@luci-project-accounts.iam.gserviceaccount.com <swiftshader-scoped@luci-project-accounts.iam.gserviceaccount.com> 1607701390 +0000

Fix resolvable depth difference calculation for triangle near 0

When all three vertices of a triangle have depth 0, the maximum exponent
e is 0. We can't just use (e - n) << n as the bit pattern for 2^(e - n),
because the exponent field underflows and creates a negative value.

All triangles where all depth values have a bias-adjusted exponent less
than 23 have this issue (note 2^(23 - 127) = 4.9e-32).

The fix is to perform 2^(e - n) as 2^e * 2^(-n). The multiplication
handles the underflow of the exponent either by creating a subnormal, or
zero.

Bug: b/139341727
Change-Id: I44b87feb55f61c5fa18ba235e9ec211926de3b3e
Fixes: b/174051829
Reviewed-on: https://swiftshader-review.googlesource.com/c/SwiftShader/+/51148
Kokoro-Result: kokoro <noreply+kokoro@google.com>
Tested-by: Nicolas Capens <nicolascapens@google.com>
Reviewed-by: Alexis Hétu <sugoi@google.com>
Commit-Queue: Nicolas Capens <nicolascapens@google.com>