Rasterize polygon edges from the real endpoint

Previously the outline rasterizer would start at the y coordinate of the
first endpoint clamped to the scissor rectangle bound. This caused
potential overflow in a 32-bit multiplication used as part of computing
the initial x coordinate, when increasing sub-pixel precision.

This change makes us start rasterization at the endpoint's coordinate,
rounded up to the next integer. This makes the difference between this
integer value and the fixed-point endpoint coordinate small and avoid
the overflow.

One consequence of this is that we're potentially edge-stepping through
rows outside of the scissor rectangle. While that could be avoided with
some extra math to jump over the gap in one go, that would come at an
extra cost for every polygon, instead of just the ones with parts
outside of the scissor rectangle.

That cost could be reduced by detecting when there's a gap larger than
one pixel, but this is unwarranted complexity since we should work
towards using a coverage-based rasterizer instead, which wouldn't suffer
from this issue in the first place.

Bug: b/257337434
Change-Id: I6ac8a50f2d27ca81b09855ee863423b1b88dbb4e
Reviewed-on: https://swiftshader-review.googlesource.com/c/SwiftShader/+/69668
Kokoro-Result: kokoro <noreply+kokoro@google.com>
Tested-by: Nicolas Capens <nicolascapens@google.com>
Reviewed-by: Alexis Hétu <sugoi@google.com>
diff --git a/src/Pipeline/SetupRoutine.cpp b/src/Pipeline/SetupRoutine.cpp
index 825ab17..821d35f 100644
--- a/src/Pipeline/SetupRoutine.cpp
+++ b/src/Pipeline/SetupRoutine.cpp
@@ -561,10 +561,12 @@
 		constexpr int subPixB = vk::SUBPIXEL_PRECISION_BITS;
 		constexpr int subPixM = vk::SUBPIXEL_PRECISION_MASK;
 
-		Int y1 = Max((Y1 + subPixM) >> subPixB, *Pointer<Int>(data + OFFSET(DrawData, scissorY0)));
-		Int y2 = Min((Y2 + subPixM) >> subPixB, *Pointer<Int>(data + OFFSET(DrawData, scissorY1)));
+		Int y1 = (Y1 + subPixM) >> subPixB;
+		Int y2 = (Y2 + subPixM) >> subPixB;
+		Int yMin = Max(y1, *Pointer<Int>(data + OFFSET(DrawData, scissorY0)));
+		Int yMax = Min(y2, *Pointer<Int>(data + OFFSET(DrawData, scissorY1)));
 
-		If(y1 < y2)
+		If(yMin < yMax)
 		{
 			Int xMin = *Pointer<Int>(data + OFFSET(DrawData, scissorX0));
 			Int xMax = *Pointer<Int>(data + OFFSET(DrawData, scissorX1));
@@ -598,7 +600,10 @@
 
 			Do
 			{
-				*Pointer<Short>(edge + y * sizeof(Primitive::Span)) = Short(Clamp(x, xMin, xMax));
+				If(y >= yMin)
+				{
+					*Pointer<Short>(edge + y * sizeof(Primitive::Span)) = Short(Clamp(x, xMin, xMax));
+				}
 
 				x += Q;
 				d += R;
@@ -610,7 +615,7 @@
 
 				y++;
 			}
-			Until(y >= y2);
+			Until(y >= yMax);
 		}
 	}
 }