Fix float to r11g11b10 conversion
Floating point negative values were first converted to half values,
followed with a conversion to 11 bit or 10 bit mini floats by
chopping the mantissa and removing the sign bit. Negative values
were converted to the same values as positive values since removing
the sign bit ended up doing the equivalent of an Abs() call.
Clamped the value to 0 before the conversion to solve the issue.
Bug: b/146223877 b/147900455
Change-Id: I97decae66dc57a68f175b06902eb6725fc7d5794
Reviewed-on: https://swiftshader-review.googlesource.com/c/SwiftShader/+/42548
Kokoro-Presubmit: kokoro <noreply+kokoro@google.com>
Tested-by: Alexis Hétu <sugoi@google.com>
Reviewed-by: Nicolas Capens <nicolascapens@google.com>
diff --git a/src/Pipeline/ShaderCore.cpp b/src/Pipeline/ShaderCore.cpp
index fff67ad..83f8126 100644
--- a/src/Pipeline/ShaderCore.cpp
+++ b/src/Pipeline/ShaderCore.cpp
@@ -611,7 +611,8 @@
UInt r11g11b10Pack(const Float4 &value)
{
- auto halfBits = floatToHalfBits(As<UInt4>(value), true);
+ // 10 and 11 bit floats are unsigned, so their minimal value is 0
+ auto halfBits = floatToHalfBits(As<UInt4>(Max(value, Float4(0.0f))), true);
// Truncates instead of rounding. See b/147900455
UInt4 truncBits = halfBits & UInt4(0x7FF00000, 0x7FF00000, 0x7FE00000, 0);
return (UInt(truncBits.x) >> 20) | (UInt(truncBits.y) >> 9) | (UInt(truncBits.z) << 1);
