diff --git a/.gitignore b/.gitignore
index a19ccde..0d1d959 100644
--- a/.gitignore
+++ b/.gitignore
@@ -10,3 +10,4 @@
 # Ignore specific patterns at the top-level directory
 /llvm2ice
 /build/
+/crosstest/Output/
diff --git a/LOWERING.rst b/LOWERING.rst
new file mode 100644
index 0000000..251e25c
--- /dev/null
+++ b/LOWERING.rst
@@ -0,0 +1,310 @@
+Target-specific lowering in ICE
+===============================
+
+This document discusses several issues around generating target-specific ICE
+instructions from high-level ICE instructions.
+
+Meeting register address mode constraints
+-----------------------------------------
+
+Target-specific instructions often require specific operands to be in physical
+registers.  Sometimes one specific register is required, but usually any
+register in a particular register class will suffice, and that register class is
+defined by the instruction/operand type.
+
+The challenge is that ``Variable`` represents an operand that is either a stack
+location in the current frame, or a physical register.  Register allocation
+happens after target-specific lowering, so during lowering we generally don't
+know whether a ``Variable`` operand will meet a target instruction's physical
+register requirement.
+
+To this end, ICE allows certain hints/directives:
+
+    * ``Variable::setWeightInfinite()`` forces a ``Variable`` to get some
+      physical register (without specifying which particular one) from a
+      register class.
+
+    * ``Variable::setRegNum()`` forces a ``Variable`` to be assigned a specific
+      physical register.
+
+    * ``Variable::setPreferredRegister()`` registers a preference for a physical
+      register based on another ``Variable``'s physical register assignment.
+
+These hints/directives are described below in more detail.  In most cases,
+though, they don't need to be explicity used, as the routines that create
+lowered instructions have reasonable defaults and simple options that control
+these hints/directives.
+
+The recommended ICE lowering strategy is to generate extra assignment
+instructions involving extra ``Variable`` temporaries, using the
+hints/directives to force suitable register assignments for the temporaries, and
+then let the global register allocator clean things up.
+
+Note: There is a spectrum of *implementation complexity* versus *translation
+speed* versus *code quality*.  This recommended strategy picks a point on the
+spectrum representing very low complexity ("splat-isel"), pretty good code
+quality in terms of frame size and register shuffling/spilling, but perhaps not
+the fastest translation speed since extra instructions and operands are created
+up front and cleaned up at the end.
+
+Ensuring some physical register
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The x86 instruction::
+
+    mov dst, src
+
+needs at least one of its operands in a physical register (ignoring the case
+where ``src`` is a constant).  This can be done as follows::
+
+    mov reg, src
+    mov dst, reg
+
+so long as ``reg`` is guaranteed to have a physical register assignment.  The
+low-level lowering code that accomplishes this looks something like::
+
+    Variable *Reg;
+    Reg = Func->makeVariable(Dst->getType());
+    Reg->setWeightInfinite();
+    NewInst = InstX8632Mov::create(Func, Reg, Src);
+    NewInst = InstX8632Mov::create(Func, Dst, Reg);
+
+``Cfg::makeVariable()`` generates a new temporary, and
+``Variable::setWeightInfinite()`` gives it infinite weight for the purpose of
+register allocation, thus guaranteeing it a physical register.
+
+The ``_mov(Dest, Src)`` method in the ``TargetX8632`` class is sufficiently
+powerful to handle these details in most situations.  Its ``Dest`` argument is
+an in/out parameter.  If its input value is ``NULL``, then a new temporary
+variable is created, its type is set to the same type as the ``Src`` operand, it
+is given infinite register weight, and the new ``Variable`` is returned through
+the in/out parameter.  (This is in addition to the new temporary being the dest
+operand of the ``mov`` instruction.)  The simpler version of the above example
+is::
+
+    Variable *Reg = NULL;
+    _mov(Reg, Src);
+    _mov(Dst, Reg);
+
+Preferring another ``Variable``'s physical register
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+One problem with this example is that the register allocator usually just
+assigns the first available register to a live range.  If this instruction ends
+the live range of ``src``, this may lead to code like the following::
+
+    mov reg:eax, src:esi
+    mov dst:edi, reg:eax
+
+Since the first instruction happens to end the live range of ``src:esi``, it
+would be better to assign ``esi`` to ``reg``::
+
+    mov reg:esi, src:esi
+    mov dst:edi, reg:esi
+
+The first instruction, ``mov esi, esi``, is a redundant assignment and will
+ultimately be elided, leaving just ``mov edi, esi``.
+
+We can tell the register allocator to prefer the register assigned to a
+different ``Variable``, using ``Variable::setPreferredRegister()``::
+
+    Variable *Reg;
+    Reg = Func->makeVariable(Dst->getType());
+    Reg->setWeightInfinite();
+    Reg->setPreferredRegister(Src);
+    NewInst = InstX8632Mov::create(Func, Reg, Src);
+    NewInst = InstX8632Mov::create(Func, Dst, Reg);
+
+Or more simply::
+
+    Variable *Reg = NULL;
+    _mov(Reg, Src);
+    _mov(Dst, Reg);
+    Reg->setPreferredRegister(llvm::dyn_cast<Variable>(Src));
+
+The usefulness of ``setPreferredRegister()`` is tied into the implementation of
+the register allocator.  ICE uses linear-scan register allocation, which sorts
+live ranges by starting point and assigns registers in that order.  Using
+``B->setPreferredRegister(A)`` only helps when ``A`` has already been assigned a
+register by the time ``B`` is being considered.  For an assignment ``B=A``, this
+is usually a safe assumption because ``B``'s live range begins at this
+instruction but ``A``'s live range must have started earlier.  (There may be
+exceptions for variables that are no longer in SSA form.)  But
+``A->setPreferredRegister(B)`` is unlikely to help unless ``B`` has been
+precolored.  In summary, generally the best practice is to use a pattern like::
+
+    NewInst = InstX8632Mov::create(Func, Dst, Src);
+    Dst->setPreferredRegister(Src);
+    //Src->setPreferredRegister(Dst); -- unlikely to have any effect
+
+Ensuring a specific physical register
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Some instructions require operands in specific physical registers, or produce
+results in specific physical registers.  For example, the 32-bit ``ret``
+instruction needs its operand in ``eax``.  This can be done with
+``Variable::setRegNum()``::
+
+    Variable *Reg;
+    Reg = Func->makeVariable(Src->getType());
+    Reg->setWeightInfinite();
+    Reg->setRegNum(Reg_eax);
+    NewInst = InstX8632Mov::create(Func, Reg, Src);
+    NewInst = InstX8632Ret::create(Func, Reg);
+
+Precoloring with ``Variable::setRegNum()`` effectively gives it infinite weight
+for register allocation, so the call to ``Variable::setWeightInfinite()`` is
+technically unnecessary, but perhaps documents the intention a bit more
+strongly.
+
+The ``_mov(Dest, Src, RegNum)`` method in the ``TargetX8632`` class has an
+optional ``RegNum`` argument to force a specific register assignment when the
+input ``Dest`` is ``NULL``.  As described above, passing in ``Dest=NULL`` causes
+a new temporary variable to be created with infinite register weight, and in
+addition the specific register is chosen.  The simpler version of the above
+example is::
+
+    Variable *Reg = NULL;
+    _mov(Reg, Src, Reg_eax);
+    _ret(Reg);
+
+Disabling live-range interference
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Another problem with the "``mov reg,src; mov dst,reg``" example happens when
+the instructions do *not* end the live range of ``src``.  In this case, the live
+ranges of ``reg`` and ``src`` interfere, so they can't get the same physical
+register despite the explicit preference.  However, ``reg`` is meant to be an
+alias of ``src`` so they needn't be considered to interfere with each other.
+This can be expressed via the second (bool) argument of
+``setPreferredRegister()``::
+
+    Variable *Reg;
+    Reg = Func->makeVariable(Dst->getType());
+    Reg->setWeightInfinite();
+    Reg->setPreferredRegister(Src, true);
+    NewInst = InstX8632Mov::create(Func, Reg, Src);
+    NewInst = InstX8632Mov::create(Func, Dst, Reg);
+
+This should be used with caution and probably only for these short-live-range
+temporaries, otherwise the classic "lost copy" or "lost swap" problem may be
+encountered.
+
+Instructions with register side effects
+---------------------------------------
+
+Some instructions produce unwanted results in other registers, or otherwise kill
+preexisting values in other registers.  For example, a ``call`` kills the
+scratch registers.  Also, the x86-32 ``idiv`` instruction produces the quotient
+in ``eax`` and the remainder in ``edx``, but generally only one of those is
+needed in the lowering.  It's important that the register allocator doesn't
+allocate that register to a live range that spans the instruction.
+
+ICE provides the ``InstFakeKill`` pseudo-instruction to mark such register
+kills.  For each of the instruction's source variables, a fake trivial live
+range is created that begins and ends in that instruction.  The ``InstFakeKill``
+instruction is inserted after the ``call`` instruction.  For example::
+
+    CallInst = InstX8632Call::create(Func, ... );
+    VarList KilledRegs;
+    KilledRegs.push_back(eax);
+    KilledRegs.push_back(ecx);
+    KilledRegs.push_back(edx);
+    NewInst = InstFakeKill::create(Func, KilledRegs, CallInst);
+
+The last argument to the ``InstFakeKill`` constructor links it to the previous
+call instruction, such that if its linked instruction is dead-code eliminated,
+the ``InstFakeKill`` instruction is eliminated as well.
+
+The killed register arguments need to be assigned a physical register via
+``Variable::setRegNum()`` for this to be effective.  To avoid a massive
+proliferation of ``Variable`` temporaries, the ``TargetLowering`` object caches
+one precolored ``Variable`` for each physical register::
+
+    CallInst = InstX8632Call::create(Func, ... );
+    VarList KilledRegs;
+    Variable *eax = Func->getTarget()->getPhysicalRegister(Reg_eax);
+    Variable *ecx = Func->getTarget()->getPhysicalRegister(Reg_ecx);
+    Variable *edx = Func->getTarget()->getPhysicalRegister(Reg_edx);
+    KilledRegs.push_back(eax);
+    KilledRegs.push_back(ecx);
+    KilledRegs.push_back(edx);
+    NewInst = InstFakeKill::create(Func, KilledRegs, CallInst);
+
+On first glance, it may seem unnecessary to explicitly kill the register that
+returns the ``call`` return value.  However, if for some reason the ``call``
+result ends up being unused, dead-code elimination could remove dead assignments
+and incorrectly expose the return value register to a register allocation
+assignment spanning the call, which would be incorrect.
+
+Instructions producing multiple values
+--------------------------------------
+
+ICE instructions allow at most one destination ``Variable``.  Some machine
+instructions produce more than one usable result.  For example, the x86-32
+``call`` ABI returns a 64-bit integer result in the ``edx:eax`` register pair.
+Also, x86-32 has a version of the ``imul`` instruction that produces a 64-bit
+result in the ``edx:eax`` register pair.
+
+To support multi-dest instructions, ICE provides the ``InstFakeDef``
+pseudo-instruction, whose destination can be precolored to the appropriate
+physical register.  For example, a ``call`` returning a 64-bit result in
+``edx:eax``::
+
+    CallInst = InstX8632Call::create(Func, RegLow, ... );
+    ...
+    NewInst = InstFakeKill::create(Func, KilledRegs, CallInst);
+    Variable *RegHigh = Func->makeVariable(IceType_i32);
+    RegHigh->setRegNum(Reg_edx);
+    NewInst = InstFakeDef::create(Func, RegHigh);
+
+``RegHigh`` is then assigned into the desired ``Variable``.  If that assignment
+ends up being dead-code eliminated, the ``InstFakeDef`` instruction may be
+eliminated as well.
+
+Preventing dead-code elimination
+--------------------------------
+
+ICE instructions with a non-NULL ``Dest`` are subject to dead-code elimination.
+However, some instructions must not be eliminated in order to preserve side
+effects.  This applies to most function calls, volatile loads, and loads and
+integer divisions where the underlying language and runtime are relying on
+hardware exception handling.
+
+ICE facilitates this with the ``InstFakeUse`` pseudo-instruction.  This forces a
+use of its source ``Variable`` to keep that variable's definition alive.  Since
+the ``InstFakeUse`` instruction has no ``Dest``, it will not be eliminated.
+
+Here is the full example of the x86-32 ``call`` returning a 32-bit integer
+result::
+
+    Variable *Reg = Func->makeVariable(IceType_i32);
+    Reg->setRegNum(Reg_eax);
+    CallInst = InstX8632Call::create(Func, Reg, ... );
+    VarList KilledRegs;
+    Variable *eax = Func->getTarget()->getPhysicalRegister(Reg_eax);
+    Variable *ecx = Func->getTarget()->getPhysicalRegister(Reg_ecx);
+    Variable *edx = Func->getTarget()->getPhysicalRegister(Reg_edx);
+    KilledRegs.push_back(eax);
+    KilledRegs.push_back(ecx);
+    KilledRegs.push_back(edx);
+    NewInst = InstFakeKill::create(Func, KilledRegs, CallInst);
+    NewInst = InstFakeUse::create(Func, Reg);
+    NewInst = InstX8632Mov::create(Func, Result, Reg);
+
+Without the ``InstFakeUse``, the entire call sequence could be dead-code
+eliminated if its result were unused.
+
+One more note on this topic.  These tools can be used to allow a multi-dest
+instruction to be dead-code eliminated only when none of its results is live.
+The key is to use the optional source parameter of the ``InstFakeDef``
+instruction.  Using pseudocode::
+
+    t1:eax = call foo(arg1, ...)
+    InstFakeKill(eax, ecx, edx)
+    t2:edx = InstFakeDef(t1)
+    v_result_low = t1
+    v_result_high = t2
+
+If ``v_result_high`` is live but ``v_result_low`` is dead, adding ``t1`` as an
+argument to ``InstFakeDef`` suffices to keep the ``call`` instruction live.
diff --git a/Makefile b/Makefile
index e32ae89..a1584f5 100644
--- a/Makefile
+++ b/Makefile
@@ -29,7 +29,7 @@
 # It's recommended that CXX matches the compiler you used to build LLVM itself.
 OPTLEVEL := -O0
 CXX := g++
-CXXFLAGS := -Wall -Werror -fno-rtti -fno-exceptions \
+CXXFLAGS := -Wall -Wextra -Werror -fno-rtti -fno-exceptions \
 	$(OPTLEVEL) -g $(LLVM_CXXFLAGS) -m32
 LDFLAGS := -m32
 
@@ -38,7 +38,10 @@
 	IceCfgNode.cpp \
 	IceGlobalContext.cpp \
 	IceInst.cpp \
+	IceInstX8632.cpp \
 	IceOperand.cpp \
+	IceTargetLowering.cpp \
+	IceTargetLoweringX8632.cpp \
 	IceTypes.cpp \
 	llvm2ice.cpp
 
@@ -64,6 +67,7 @@
 check: llvm2ice
 	LLVM_BIN_PATH=$(LLVM_BIN_PATH) \
 	$(LLVM_SRC_PATH)/utils/lit/lit.py -sv tests_lit
+	(cd crosstest; LLVM_BIN_PATH=$(LLVM_BIN_PATH) ./runtests.sh)
 
 # TODO: Fix the use of wildcards.
 format:
diff --git a/README.rst b/README.rst
index 9de4602..21d7418 100644
--- a/README.rst
+++ b/README.rst
@@ -34,15 +34,24 @@
     ``-notranslate`` -- Suppress the ICE translation phase, which is useful if
     ICE is missing some support.
 
-    ``-target=<TARGET>`` -- Set the target architecture.  The default is x8632,
-    and x8632fast (generate x8632 code as fast as possible at the cost of code
-    quality) is also available.  Future targets include x8664, arm32, and arm64.
+    ``-target=<TARGET>`` -- Set the target architecture.  The default is x8632.
+    Future targets include x8664, arm32, and arm64.
+
+    ``-O<LEVEL>`` -- Set the optimization level.  Valid levels are ``2``, ``1``,
+    ``0``, ``-1``, and ``m1``.  Levels ``-1`` and ``m1`` are synonyms, and
+    represent the minimum optimization and worst code quality, but fastest code
+    generation.
 
     ``-verbose=<list>`` -- Set verbosity flags.  This argument allows a
     comma-separated list of values.  The default is ``none``, and the value
     ``inst,pred`` will roughly match the .ll bitcode file.  Of particular use
     are ``all`` and ``none``.
 
+    ``-o <FILE>`` -- Set the assembly output file name.  Default is stdout.
+
+    ``-log <FILE>`` -- Set the file name for diagnostic output (whose level is
+    controlled by ``-verbose``).  Default is stdout.
+
 See ir_samples/README.rst for more details.
 
 Running the test suite
diff --git a/crosstest/crosstest.py b/crosstest/crosstest.py
new file mode 100755
index 0000000..e835193
--- /dev/null
+++ b/crosstest/crosstest.py
@@ -0,0 +1,127 @@
+#!/usr/bin/env python2
+
+import argparse
+import os
+import re
+import subprocess
+import sys
+import tempfile
+
+sys.path.insert(0, '../pydir')
+from utils import shellcmd
+
+if __name__ == '__main__':
+    """Builds a cross-test binary that allows functions translated by
+    Subzero and llc to be compared.
+
+    Each --test argument is compiled once by llc and once by Subzero.
+    C/C++ tests are first compiled down to PNaCl bitcode by the
+    build-pnacl-ir.py script.  The --prefix argument ensures that
+    symbol names are different between the two object files, to avoid
+    linking errors.
+
+    There is also a --driver argument that specifies the C/C++ file
+    that calls the test functions with a variety of interesting inputs
+    and compares their results.
+    """
+    # arch_map maps a Subzero target string to an llvm-mc -arch string.
+    arch_map = { 'x8632':'x86', 'x8664':'x86-64', 'arm':'arm' }
+    desc = 'Build a cross-test that compares Subzero and llc translation.'
+    argparser = argparse.ArgumentParser(description=desc)
+    argparser.add_argument('--test', required=True, action='append',
+                           metavar='TESTFILE_LIST',
+                           help='List of C/C++/.ll files with test functions')
+    argparser.add_argument('--driver', required=True,
+                           metavar='DRIVER',
+                           help='Driver program')
+    argparser.add_argument('--target', required=False, default='x8632',
+                           choices=arch_map.keys(),
+                           metavar='TARGET',
+                           help='Translation target architecture')
+    argparser.add_argument('-O', required=False, default='2', dest='optlevel',
+                           choices=['m1', '-1', '0', '1', '2'],
+                           metavar='OPTLEVEL',
+                           help='Optimization level ' +
+                                '(m1 and -1 are equivalent)')
+    argparser.add_argument('--prefix', required=True,
+                           metavar='SZ_PREFIX',
+                           help='String prepended to Subzero symbol names')
+    argparser.add_argument('--output', '-o', required=True,
+                           metavar='EXECUTABLE',
+                           help='Executable to produce')
+    argparser.add_argument('--dir', required=False, default='.',
+                           metavar='OUTPUT_DIR',
+                           help='Output directory for all files')
+    argparser.add_argument('--llvm-bin-path', required=False,
+                           default=os.environ.get('LLVM_BIN_PATH'),
+                           metavar='PATH',
+                           help='Path to LLVM executables like llc ' +
+                                '(defaults to $LLVM_BIN_PATH)')
+    args = argparser.parse_args()
+
+    objs = []
+    remove_internal = re.compile('^define internal ')
+    fix_target = re.compile('le32-unknown-nacl')
+    llvm_bin_path = args.llvm_bin_path
+    for arg in args.test:
+        base, ext = os.path.splitext(arg)
+        if ext == '.ll':
+            bitcode = arg
+        else:
+            bitcode = os.path.join(args.dir, base + '.pnacl.ll')
+            shellcmd(['../pydir/build-pnacl-ir.py', '--disable-verify',
+                      '--dir', args.dir, arg])
+            # Read in the bitcode file, fix it up, and rewrite the file.
+            f = open(bitcode)
+            ll_lines = f.readlines()
+            f.close()
+            f = open(bitcode, 'w')
+            for line in ll_lines:
+                line = remove_internal.sub('define ', line)
+                line = fix_target.sub('i686-pc-linux-gnu', line)
+                f.write(line)
+            f.close()
+
+        asm_sz = os.path.join(args.dir, base + '.sz.s')
+        obj_sz = os.path.join(args.dir, base + '.sz.o')
+        obj_llc = os.path.join(args.dir, base + '.llc.o')
+        shellcmd(['../llvm2ice',
+                  '-O' + args.optlevel,
+                  '--target=' + args.target,
+                  '--prefix=' + args.prefix,
+                  '-o=' + asm_sz,
+                  bitcode])
+        shellcmd([os.path.join(llvm_bin_path, 'llvm-mc'),
+                  '-arch=' + arch_map[args.target],
+                  '-x86-asm-syntax=intel',
+                  '-filetype=obj',
+                  '-o=' + obj_sz,
+                  asm_sz])
+        objs.append(obj_sz)
+        # Each original bitcode file needs to be translated by the
+        # LLVM toolchain and have its object file linked in.  There
+        # are two ways to do this: explicitly use llc, or include the
+        # .ll file in the link command.  It turns out that these two
+        # approaches can produce different semantics on some undefined
+        # bitcode behavior.  Specifically, LLVM produces different
+        # results for overflowing fptoui instructions for i32 and i64
+        # on x86-32.  As it turns out, Subzero lowering was based on
+        # inspecting the object code produced by the direct llc
+        # command, so we need to directly run llc on the bitcode, even
+        # though it makes this script longer, to avoid spurious
+        # failures.  This behavior can be inspected by switching
+        # use_llc between True and False.
+        use_llc = False
+        if use_llc:
+            shellcmd([os.path.join(llvm_bin_path, 'llc'),
+                      '-filetype=obj',
+                      '-o=' + obj_llc,
+                      bitcode])
+            objs.append(obj_llc)
+        else:
+            objs.append(bitcode)
+
+    linker = 'clang' if os.path.splitext(args.driver)[1] == '.c' else 'clang++'
+    shellcmd([os.path.join(llvm_bin_path, linker), '-g', '-m32', args.driver] +
+             objs +
+             ['-lm', '-o', os.path.join(args.dir, args.output)])
diff --git a/crosstest/runtests.sh b/crosstest/runtests.sh
new file mode 100755
index 0000000..400e7a4
--- /dev/null
+++ b/crosstest/runtests.sh
@@ -0,0 +1,59 @@
+#!/bin/sh
+
+# TODO: Retire this script and move the individual tests into the lit
+# framework, to leverage parallel testing and other lit goodness.
+
+set -eux
+
+OPTLEVELS="m1"
+OUTDIR=Output
+# Clean the output directory to avoid reusing stale results.
+rm -rf "${OUTDIR}"
+mkdir -p "${OUTDIR}"
+
+for optlevel in ${OPTLEVELS} ; do
+
+    ./crosstest.py -O${optlevel} --prefix=Subzero_ --target=x8632 \
+        --dir="${OUTDIR}" \
+        --llvm-bin-path="${LLVM_BIN_PATH}" \
+        --test=simple_loop.c \
+        --driver=simple_loop_main.c \
+        --output=simple_loop_O${optlevel}
+
+    ./crosstest.py -O${optlevel} --prefix=Subzero_ --target=x8632 \
+        --dir="${OUTDIR}" \
+        --llvm-bin-path="${LLVM_BIN_PATH}" \
+        --test=test_cast.cpp --test=test_cast_to_u1.ll \
+        --driver=test_cast_main.cpp \
+        --output=test_cast_O${optlevel}
+
+    ./crosstest.py -O${optlevel} --prefix=Subzero_ --target=x8632 \
+        --dir="${OUTDIR}" \
+        --llvm-bin-path="${LLVM_BIN_PATH}" \
+        --test=test_fcmp.pnacl.ll \
+        --driver=test_fcmp_main.cpp \
+        --output=test_fcmp_O${optlevel}
+
+    ./crosstest.py -O${optlevel} --prefix=Subzero_ --target=x8632 \
+        --dir="${OUTDIR}" \
+        --llvm-bin-path="${LLVM_BIN_PATH}" \
+        --test=test_icmp.cpp \
+        --driver=test_icmp_main.cpp \
+        --output=test_icmp_O${optlevel}
+
+    ./crosstest.py -O${optlevel} --prefix=Subzero_ --target=x8632 \
+        --dir="${OUTDIR}" \
+        --llvm-bin-path="${LLVM_BIN_PATH}" \
+        --test=test_arith.cpp --test=test_arith_frem.ll \
+        --driver=test_arith_main.cpp \
+        --output=test_arith_O${optlevel}
+
+done
+
+for optlevel in ${OPTLEVELS} ; do
+    "${OUTDIR}"/simple_loop_O${optlevel}
+    "${OUTDIR}"/test_cast_O${optlevel}
+    "${OUTDIR}"/test_fcmp_O${optlevel}
+    "${OUTDIR}"/test_icmp_O${optlevel}
+    "${OUTDIR}"/test_arith_O${optlevel}
+done
diff --git a/crosstest/simple_loop.c b/crosstest/simple_loop.c
new file mode 100644
index 0000000..6568380
--- /dev/null
+++ b/crosstest/simple_loop.c
@@ -0,0 +1,10 @@
+// This is a simple loop that sums elements of an input array and
+// returns the result.  It's here mainly because it's one of the
+// simple examples guiding the early Subzero design.
+
+int simple_loop(int *a, int n) {
+  int sum = 0;
+  for (int i = 0; i < n; ++i)
+    sum += a[i];
+  return sum;
+}
diff --git a/crosstest/simple_loop_main.c b/crosstest/simple_loop_main.c
new file mode 100644
index 0000000..5ff36b8
--- /dev/null
+++ b/crosstest/simple_loop_main.c
@@ -0,0 +1,29 @@
+/* crosstest.py --test=simple_loop.c --driver=simple_loop_main.c \
+   --prefix=Subzero_ --output=simple_loop */
+
+#include <stdio.h>
+
+int simple_loop(int *a, int n);
+int Subzero_simple_loop(int *a, int n);
+
+int main(int argc, char **argv) {
+  unsigned TotalTests = 0;
+  unsigned Passes = 0;
+  unsigned Failures = 0;
+  int a[100];
+  for (int i = 0; i < 100; ++i)
+    a[i] = i * 2 - 100;
+  for (int i = -2; i < 100; ++i) {
+    ++TotalTests;
+    int llc_result = simple_loop(a, i);
+    int sz_result = Subzero_simple_loop(a, i);
+    if (llc_result == sz_result) {
+      ++Passes;
+    } else {
+      ++Failures;
+      printf("Failure: i=%d, llc=%d, sz=%d\n", i, llc_result, sz_result);
+    }
+  }
+  printf("TotalTests=%u Passes=%u Failures=%u\n", TotalTests, Passes, Failures);
+  return Failures;
+}
diff --git a/crosstest/test_arith.cpp b/crosstest/test_arith.cpp
new file mode 100644
index 0000000..18b4b57
--- /dev/null
+++ b/crosstest/test_arith.cpp
@@ -0,0 +1,30 @@
+// This aims to test all the arithmetic bitcode instructions across
+// all PNaCl primitive data types.
+
+#include <stdint.h>
+
+#include "test_arith.h"
+
+#define X(inst, op, isdiv)                                                     \
+  bool test##inst(bool a, bool b) { return a op b; }                           \
+  uint8_t test##inst(uint8_t a, uint8_t b) { return a op b; }                  \
+  uint16_t test##inst(uint16_t a, uint16_t b) { return a op b; }               \
+  uint32_t test##inst(uint32_t a, uint32_t b) { return a op b; }               \
+  uint64_t test##inst(uint64_t a, uint64_t b) { return a op b; }
+UINTOP_TABLE
+#undef X
+
+#define X(inst, op, isdiv)                                                     \
+  bool test##inst(bool a, bool b) { return a op b; }                           \
+  int8_t test##inst(int8_t a, int8_t b) { return a op b; }                     \
+  int16_t test##inst(int16_t a, int16_t b) { return a op b; }                  \
+  int32_t test##inst(int32_t a, int32_t b) { return a op b; }                  \
+  int64_t test##inst(int64_t a, int64_t b) { return a op b; }
+SINTOP_TABLE
+#undef X
+
+#define X(inst, op, func)                                                      \
+  float test##inst(float a, float b) { return func(a op b); }                  \
+  double test##inst(double a, double b) { return func(a op b); }
+FPOP_TABLE
+#undef X
diff --git a/crosstest/test_arith.def b/crosstest/test_arith.def
new file mode 100644
index 0000000..4cf4596
--- /dev/null
+++ b/crosstest/test_arith.def
@@ -0,0 +1,45 @@
+#ifndef TEST_ARITH_DEF
+#define TEST_ARITH_DEF
+
+#define XSTR(s) STR(s)
+#define STR(s) #s
+
+#define UINTOP_TABLE \
+  /* inst, operator, div */ \
+  X(Add,   +,        0 )    \
+  X(Sub,   -,        0 )    \
+  X(Mul,   *,        0 )    \
+  X(Udiv,  /,        1 )    \
+  X(Urem,  %,        1 )    \
+  X(Shl,   <<,       0)     \
+  X(Lshr,  >>,       0)     \
+  X(And,   &,        0 )    \
+  X(Or,    |,        0 )    \
+  X(Xor,   ^,        0 )    \
+//#define X(inst, op, isdiv)
+
+#define SINTOP_TABLE \
+  /* inst, operator, div */ \
+  X(Sdiv,  /,        1)     \
+  X(Srem,  %,        1)     \
+  X(Ashr,  >>,       0)     \
+//#define X(inst, op, isdiv)
+
+#define COMMA ,
+#define FPOP_TABLE \
+  /* inst, infix_op, func */ \
+  X(Fadd,  +,              ) \
+  X(Fsub,  -,              ) \
+  X(Fmul,  *,              ) \
+  X(Fdiv,  /,              ) \
+  X(Frem,  COMMA,    myFrem) \
+//#define X(inst, op, func)
+
+// Note: The above definition of COMMA, plus the "func" argument to
+// the X macro, are because C++ does not allow the % operator on
+// floating-point primitive types.  To work around this, the expansion
+// is "func(a infix_op b)", which becomes "myFrem(a , b)" for the Frem
+// instruction and "(a + b)" for the Fadd instruction.  The two
+// versions of myFrem() are defined in a separate bitcode file.
+
+#endif // TEST_ARITH_DEF
diff --git a/crosstest/test_arith.h b/crosstest/test_arith.h
new file mode 100644
index 0000000..996d962
--- /dev/null
+++ b/crosstest/test_arith.h
@@ -0,0 +1,29 @@
+#include <stdint.h>
+#include "test_arith.def"
+
+#define X(inst, op, isdiv)                                                     \
+  bool test##inst(bool a, bool b);                                             \
+  uint8_t test##inst(uint8_t a, uint8_t b);                                    \
+  uint16_t test##inst(uint16_t a, uint16_t b);                                 \
+  uint32_t test##inst(uint32_t a, uint32_t b);                                 \
+  uint64_t test##inst(uint64_t a, uint64_t b);
+UINTOP_TABLE
+#undef X
+
+#define X(inst, op, isdiv)                                                     \
+  bool test##inst(bool a, bool b);                                             \
+  int8_t test##inst(int8_t a, int8_t b);                                       \
+  int16_t test##inst(int16_t a, int16_t b);                                    \
+  int32_t test##inst(int32_t a, int32_t b);                                    \
+  int64_t test##inst(int64_t a, int64_t b);
+SINTOP_TABLE
+#undef X
+
+float myFrem(float a, float b);
+double myFrem(double a, double b);
+
+#define X(inst, op, func)                                                      \
+  float test##inst(float a, float b);                                          \
+  double test##inst(double a, double b);
+FPOP_TABLE
+#undef X
diff --git a/crosstest/test_arith_frem.ll b/crosstest/test_arith_frem.ll
new file mode 100644
index 0000000..34b7156
--- /dev/null
+++ b/crosstest/test_arith_frem.ll
@@ -0,0 +1,11 @@
+target triple = "i686-pc-linux-gnu"
+
+define float @_Z6myFremff(float %a, float %b) {
+  %rem = frem float %a, %b
+  ret float %rem
+}
+
+define double @_Z6myFremdd(double %a, double %b) {
+  %rem = frem double %a, %b
+  ret double %rem
+}
diff --git a/crosstest/test_arith_main.cpp b/crosstest/test_arith_main.cpp
new file mode 100644
index 0000000..745da61
--- /dev/null
+++ b/crosstest/test_arith_main.cpp
@@ -0,0 +1,193 @@
+/* crosstest.py --test=test_arith.cpp --test=test_arith_frem.ll \
+   --driver=test_arith_main.cpp --prefix=Subzero_ --output=test_arith */
+
+#include <stdint.h>
+
+#include <cfloat>
+#include <cstring> // memcmp
+#include <iostream>
+
+// Include test_arith.h twice - once normally, and once within the
+// Subzero_ namespace, corresponding to the llc and Subzero translated
+// object files, respectively.
+#include "test_arith.h"
+namespace Subzero_ {
+#include "test_arith.h"
+}
+
+volatile unsigned Values[] = { 0x0,        0x1,        0x7ffffffe, 0x7fffffff,
+                               0x80000000, 0x80000001, 0xfffffffe, 0xffffffff,
+                               0x7e,       0x7f,       0x80,       0x81,
+                               0xfe,       0xff,       0x100,      0x101,
+                               0x7ffe,     0x7fff,     0x8000,     0x8001,
+                               0xfffe,     0xffff,     0x10000,    0x10001, };
+const static size_t NumValues = sizeof(Values) / sizeof(*Values);
+
+template <typename TypeUnsigned, typename TypeSigned>
+void testsInt(size_t &TotalTests, size_t &Passes, size_t &Failures) {
+  typedef TypeUnsigned (*FuncTypeUnsigned)(TypeUnsigned, TypeUnsigned);
+  typedef TypeSigned (*FuncTypeSigned)(TypeSigned, TypeSigned);
+  static struct {
+    const char *Name;
+    FuncTypeUnsigned FuncLlc;
+    FuncTypeUnsigned FuncSz;
+    bool ExcludeDivExceptions; // for divide related tests
+  } Funcs[] = {
+#define X(inst, op, isdiv)                                                     \
+  {                                                                            \
+    STR(inst), (FuncTypeUnsigned)test##inst,                                   \
+        (FuncTypeUnsigned)Subzero_::test##inst, isdiv                          \
+  }                                                                            \
+  ,
+      UINTOP_TABLE
+#undef X
+#define X(inst, op, isdiv)                                                     \
+  {                                                                            \
+    STR(inst), (FuncTypeUnsigned)(FuncTypeSigned)test##inst,                   \
+        (FuncTypeUnsigned)(FuncTypeSigned)Subzero_::test##inst, isdiv          \
+  }                                                                            \
+  ,
+          SINTOP_TABLE
+#undef X
+    };
+  const static size_t NumFuncs = sizeof(Funcs) / sizeof(*Funcs);
+
+  if (sizeof(TypeUnsigned) <= sizeof(uint32_t)) {
+    // This is the "normal" version of the loop nest, for 32-bit or
+    // narrower types.
+    for (size_t f = 0; f < NumFuncs; ++f) {
+      for (size_t i = 0; i < NumValues; ++i) {
+        for (size_t j = 0; j < NumValues; ++j) {
+          TypeUnsigned Value1 = Values[i];
+          TypeUnsigned Value2 = Values[j];
+          // Avoid HW divide-by-zero exception.
+          if (Funcs[f].ExcludeDivExceptions && Value2 == 0)
+            continue;
+          // Avoid HW overflow exception (on x86-32).  TODO: adjust
+          // for other architectures.
+          if (Funcs[f].ExcludeDivExceptions && Value1 == 0x80000000 &&
+              Value2 == 0xffffffff)
+            continue;
+          ++TotalTests;
+          TypeUnsigned ResultSz = Funcs[f].FuncSz(Value1, Value2);
+          TypeUnsigned ResultLlc = Funcs[f].FuncLlc(Value1, Value2);
+          if (ResultSz == ResultLlc) {
+            ++Passes;
+          } else {
+            ++Failures;
+            std::cout << "test" << Funcs[f].Name << (8 * sizeof(TypeUnsigned))
+                      << "(" << Value1 << ", " << Value2
+                      << "): sz=" << (unsigned)ResultSz
+                      << " llc=" << (unsigned)ResultLlc << std::endl;
+          }
+        }
+      }
+    }
+  } else {
+    // This is the 64-bit version.  Test values are synthesized from
+    // the 32-bit values in Values[].
+    for (size_t f = 0; f < NumFuncs; ++f) {
+      for (size_t iLo = 0; iLo < NumValues; ++iLo) {
+        for (size_t iHi = 0; iHi < NumValues; ++iHi) {
+          for (size_t jLo = 0; jLo < NumValues; ++jLo) {
+            for (size_t jHi = 0; jHi < NumValues; ++jHi) {
+              TypeUnsigned Value1 =
+                  (((TypeUnsigned)Values[iHi]) << 32) + Values[iLo];
+              TypeUnsigned Value2 =
+                  (((TypeUnsigned)Values[jHi]) << 32) + Values[jLo];
+              // Avoid HW divide-by-zero exception.
+              if (Funcs[f].ExcludeDivExceptions && Value2 == 0)
+                continue;
+              ++TotalTests;
+              TypeUnsigned ResultSz = Funcs[f].FuncSz(Value1, Value2);
+              TypeUnsigned ResultLlc = Funcs[f].FuncLlc(Value1, Value2);
+              if (ResultSz == ResultLlc) {
+                ++Passes;
+              } else {
+                ++Failures;
+                std::cout << "test" << Funcs[f].Name
+                          << (8 * sizeof(TypeUnsigned)) << "(" << Value1 << ", "
+                          << Value2 << "): sz=" << (unsigned)ResultSz
+                          << " llc=" << (unsigned)ResultLlc << std::endl;
+              }
+            }
+          }
+        }
+      }
+    }
+  }
+}
+
+template <typename Type>
+void testsFp(size_t &TotalTests, size_t &Passes, size_t &Failures) {
+  static const Type NegInf = -1.0 / 0.0;
+  static const Type PosInf = 1.0 / 0.0;
+  static const Type Nan = 0.0 / 0.0;
+  volatile Type Values[] = {
+    0,                    1,                    0x7e,
+    0x7f,                 0x80,                 0x81,
+    0xfe,                 0xff,                 0x7ffe,
+    0x7fff,               0x8000,               0x8001,
+    0xfffe,               0xffff,               0x7ffffffe,
+    0x7fffffff,           0x80000000,           0x80000001,
+    0xfffffffe,           0xffffffff,           0x100000000ll,
+    0x100000001ll,        0x7ffffffffffffffell, 0x7fffffffffffffffll,
+    0x8000000000000000ll, 0x8000000000000001ll, 0xfffffffffffffffell,
+    0xffffffffffffffffll, NegInf,               PosInf,
+    Nan,                  FLT_MIN,              FLT_MAX,
+    DBL_MIN,              DBL_MAX
+  };
+  const static size_t NumValues = sizeof(Values) / sizeof(*Values);
+  typedef Type (*FuncType)(Type, Type);
+  static struct {
+    const char *Name;
+    FuncType FuncLlc;
+    FuncType FuncSz;
+  } Funcs[] = {
+#define X(inst, op, func)                                                      \
+  { STR(inst), (FuncType)test##inst, (FuncType)Subzero_::test##inst }          \
+  ,
+      FPOP_TABLE
+#undef X
+    };
+  const static size_t NumFuncs = sizeof(Funcs) / sizeof(*Funcs);
+
+  for (size_t f = 0; f < NumFuncs; ++f) {
+    for (size_t i = 0; i < NumValues; ++i) {
+      for (size_t j = 0; j < NumValues; ++j) {
+        Type Value1 = Values[i];
+        Type Value2 = Values[j];
+        ++TotalTests;
+        Type ResultSz = Funcs[f].FuncSz(Value1, Value2);
+        Type ResultLlc = Funcs[f].FuncLlc(Value1, Value2);
+        // Compare results using memcmp() in case they are both NaN.
+        if (!memcmp(&ResultSz, &ResultLlc, sizeof(Type))) {
+          ++Passes;
+        } else {
+          ++Failures;
+          std::cout << std::fixed << "test" << Funcs[f].Name
+                    << (8 * sizeof(Type)) << "(" << Value1 << ", " << Value2
+                    << "): sz=" << ResultSz << " llc=" << ResultLlc
+                    << std::endl;
+        }
+      }
+    }
+  }
+}
+
+int main(int argc, char **argv) {
+  size_t TotalTests = 0;
+  size_t Passes = 0;
+  size_t Failures = 0;
+
+  testsInt<uint8_t, int8_t>(TotalTests, Passes, Failures);
+  testsInt<uint16_t, int16_t>(TotalTests, Passes, Failures);
+  testsInt<uint32_t, int32_t>(TotalTests, Passes, Failures);
+  testsInt<uint64_t, int64_t>(TotalTests, Passes, Failures);
+  testsFp<float>(TotalTests, Passes, Failures);
+  testsFp<double>(TotalTests, Passes, Failures);
+
+  std::cout << "TotalTests=" << TotalTests << " Passes=" << Passes
+            << " Failures=" << Failures << "\n";
+  return Failures;
+}
diff --git a/crosstest/test_cast.cpp b/crosstest/test_cast.cpp
new file mode 100644
index 0000000..1035109
--- /dev/null
+++ b/crosstest/test_cast.cpp
@@ -0,0 +1,61 @@
+// This aims to test all the conversion bitcode instructions across
+// all PNaCl primitive data types.
+
+#include <stdint.h>
+#include "test_cast.h"
+
+template <typename FromType, typename ToType>
+ToType __attribute__((noinline)) cast(FromType a) {
+  return (ToType)a;
+}
+
+template <typename FromType, typename ToType>
+ToType __attribute__((noinline)) castBits(FromType a) {
+  return *(ToType *)&a;
+}
+
+// The purpose of the following sets of templates is to force
+// cast<A,B>() to be instantiated in the resulting bitcode file for
+// all <A,B>, so that they can be called from the driver.
+template <typename ToType> class Caster {
+  static ToType f(bool a) { return cast<bool, ToType>(a); }
+  static ToType f(int8_t a) { return cast<int8_t, ToType>(a); }
+  static ToType f(uint8_t a) { return cast<uint8_t, ToType>(a); }
+  static ToType f(int16_t a) { return cast<int16_t, ToType>(a); }
+  static ToType f(uint16_t a) { return cast<uint16_t, ToType>(a); }
+  static ToType f(int32_t a) { return cast<int32_t, ToType>(a); }
+  static ToType f(uint32_t a) { return cast<uint32_t, ToType>(a); }
+  static ToType f(int64_t a) { return cast<int64_t, ToType>(a); }
+  static ToType f(uint64_t a) { return cast<uint64_t, ToType>(a); }
+  static ToType f(float a) { return cast<float, ToType>(a); }
+  static ToType f(double a) { return cast<double, ToType>(a); }
+};
+
+// Comment out the definition of Caster<bool> because clang compiles
+// casts to bool using icmp instead of the desired cast instruction.
+// The corrected definitions are in test_cast_to_u1.ll.
+
+// template class Caster<bool>;
+
+template class Caster<int8_t>;
+template class Caster<uint8_t>;
+template class Caster<int16_t>;
+template class Caster<uint16_t>;
+template class Caster<int32_t>;
+template class Caster<uint32_t>;
+template class Caster<int64_t>;
+template class Caster<uint64_t>;
+template class Caster<float>;
+template class Caster<double>;
+
+// This function definition forces castBits<A,B>() to be instantiated
+// in the resulting bitcode file for the 4 relevant <A,B>
+// combinations, so that they can be called from the driver.
+double makeBitCasters() {
+  double Result = 0;
+  Result += castBits<uint32_t, float>(0);
+  Result += castBits<uint64_t, double>(0);
+  Result += castBits<float, uint32_t>(0);
+  Result += castBits<double, uint64_t>(0);
+  return Result;
+}
diff --git a/crosstest/test_cast.h b/crosstest/test_cast.h
new file mode 100644
index 0000000..bf59cd9
--- /dev/null
+++ b/crosstest/test_cast.h
@@ -0,0 +1,2 @@
+template <typename FromType, typename ToType> ToType cast(FromType a);
+template <typename FromType, typename ToType> ToType castBits(FromType a);
diff --git a/crosstest/test_cast_main.cpp b/crosstest/test_cast_main.cpp
new file mode 100644
index 0000000..330f984
--- /dev/null
+++ b/crosstest/test_cast_main.cpp
@@ -0,0 +1,225 @@
+/* crosstest.py --test=test_cast.cpp --test=test_cast_to_u1.ll \
+   --driver=test_cast_main.cpp --prefix=Subzero_ --output=test_cast */
+
+#include <cstring>
+#include <iostream>
+#include <stdint.h>
+
+// Include test_cast.h twice - once normally, and once within the
+// Subzero_ namespace, corresponding to the llc and Subzero translated
+// object files, respectively.
+#include "test_cast.h"
+namespace Subzero_ {
+#include "test_cast.h"
+}
+
+#define XSTR(s) STR(s)
+#define STR(s) #s
+#define COMPARE(Func, FromCName, ToCName, Input)                               \
+  do {                                                                         \
+    ToCName ResultSz, ResultLlc;                                               \
+    ResultLlc = Func<FromCName, ToCName>(Input);                               \
+    ResultSz = Subzero_::Func<FromCName, ToCName>(Input);                      \
+    ++TotalTests;                                                              \
+    if (!memcmp(&ResultLlc, &ResultSz, sizeof(ToCName))) {                     \
+      ++Passes;                                                                \
+    } else {                                                                   \
+      ++Failures;                                                              \
+      std::cout << std::fixed << XSTR(Func)                                    \
+                << "<" XSTR(FromCName) ", " XSTR(ToCName) ">(" << Input        \
+                << "): sz=" << ResultSz << " llc=" << ResultLlc << "\n";       \
+    }                                                                          \
+  } while (0)
+
+template <typename FromType>
+void testValue(FromType Val, size_t &TotalTests, size_t &Passes,
+               size_t &Failures) {
+  COMPARE(cast, FromType, bool, Val);
+  COMPARE(cast, FromType, uint8_t, Val);
+  COMPARE(cast, FromType, int8_t, Val);
+  COMPARE(cast, FromType, uint16_t, Val);
+  COMPARE(cast, FromType, int16_t, Val);
+  COMPARE(cast, FromType, uint32_t, Val);
+  COMPARE(cast, FromType, int32_t, Val);
+  COMPARE(cast, FromType, uint64_t, Val);
+  COMPARE(cast, FromType, int64_t, Val);
+  COMPARE(cast, FromType, float, Val);
+  COMPARE(cast, FromType, double, Val);
+}
+
+int main(int argc, char **argv) {
+  size_t TotalTests = 0;
+  size_t Passes = 0;
+  size_t Failures = 0;
+
+  volatile bool ValsUi1[] = { false, true };
+  static const size_t NumValsUi1 = sizeof(ValsUi1) / sizeof(*ValsUi1);
+  volatile uint8_t ValsUi8[] = { 0, 1, 0x7e, 0x7f, 0x80, 0x81, 0xfe, 0xff };
+  static const size_t NumValsUi8 = sizeof(ValsUi8) / sizeof(*ValsUi8);
+
+  volatile int8_t ValsSi8[] = { 0, 1, 0x7e, 0x7f, 0x80, 0x81, 0xfe, 0xff };
+  static const size_t NumValsSi8 = sizeof(ValsSi8) / sizeof(*ValsSi8);
+
+  volatile uint16_t ValsUi16[] = { 0,      1,      0x7e,   0x7f,   0x80,
+                                   0x81,   0xfe,   0xff,   0x7ffe, 0x7fff,
+                                   0x8000, 0x8001, 0xfffe, 0xffff };
+  static const size_t NumValsUi16 = sizeof(ValsUi16) / sizeof(*ValsUi16);
+
+  volatile int16_t ValsSi16[] = { 0,      1,      0x7e,   0x7f,   0x80,
+                                  0x81,   0xfe,   0xff,   0x7ffe, 0x7fff,
+                                  0x8000, 0x8001, 0xfffe, 0xffff };
+  static const size_t NumValsSi16 = sizeof(ValsSi16) / sizeof(*ValsSi16);
+
+  volatile size_t ValsUi32[] = {
+    0,          1,          0x7e,       0x7f,       0x80,
+    0x81,       0xfe,       0xff,       0x7ffe,     0x7fff,
+    0x8000,     0x8001,     0xfffe,     0xffff,     0x7ffffffe,
+    0x7fffffff, 0x80000000, 0x80000001, 0xfffffffe, 0xffffffff
+  };
+  static const size_t NumValsUi32 = sizeof(ValsUi32) / sizeof(*ValsUi32);
+
+  volatile size_t ValsSi32[] = {
+    0,          1,          0x7e,       0x7f,       0x80,
+    0x81,       0xfe,       0xff,       0x7ffe,     0x7fff,
+    0x8000,     0x8001,     0xfffe,     0xffff,     0x7ffffffe,
+    0x7fffffff, 0x80000000, 0x80000001, 0xfffffffe, 0xffffffff
+  };
+  static const size_t NumValsSi32 = sizeof(ValsSi32) / sizeof(*ValsSi32);
+
+  volatile uint64_t ValsUi64[] = {
+    0,                     1,                     0x7e,
+    0x7f,                  0x80,                  0x81,
+    0xfe,                  0xff,                  0x7ffe,
+    0x7fff,                0x8000,                0x8001,
+    0xfffe,                0xffff,                0x7ffffffe,
+    0x7fffffff,            0x80000000,            0x80000001,
+    0xfffffffe,            0xffffffff,            0x100000000ull,
+    0x100000001ull,        0x7ffffffffffffffeull, 0x7fffffffffffffffull,
+    0x8000000000000000ull, 0x8000000000000001ull, 0xfffffffffffffffeull,
+    0xffffffffffffffffull
+  };
+  static const size_t NumValsUi64 = sizeof(ValsUi64) / sizeof(*ValsUi64);
+
+  volatile int64_t ValsSi64[] = {
+    0,                    1,                    0x7e,
+    0x7f,                 0x80,                 0x81,
+    0xfe,                 0xff,                 0x7ffe,
+    0x7fff,               0x8000,               0x8001,
+    0xfffe,               0xffff,               0x7ffffffe,
+    0x7fffffff,           0x80000000,           0x80000001,
+    0xfffffffe,           0xffffffff,           0x100000000ll,
+    0x100000001ll,        0x7ffffffffffffffell, 0x7fffffffffffffffll,
+    0x8000000000000000ll, 0x8000000000000001ll, 0xfffffffffffffffell,
+    0xffffffffffffffffll
+  };
+  static const size_t NumValsSi64 = sizeof(ValsSi64) / sizeof(*ValsSi64);
+
+  volatile float ValsF32[] = {
+    0,                    1,                    0x7e,
+    0x7f,                 0x80,                 0x81,
+    0xfe,                 0xff,                 0x7ffe,
+    0x7fff,               0x8000,               0x8001,
+    0xfffe,               0xffff,               0x7ffffffe,
+    0x7fffffff,           0x80000000,           0x80000001,
+    0xfffffffe,           0xffffffff,           0x100000000ll,
+    0x100000001ll,        0x7ffffffffffffffell, 0x7fffffffffffffffll,
+    0x8000000000000000ll, 0x8000000000000001ll, 0xfffffffffffffffell,
+    0xffffffffffffffffll
+  };
+  static const size_t NumValsF32 = sizeof(ValsF32) / sizeof(*ValsF32);
+
+  volatile double ValsF64[] = {
+    0,                    1,                    0x7e,
+    0x7f,                 0x80,                 0x81,
+    0xfe,                 0xff,                 0x7ffe,
+    0x7fff,               0x8000,               0x8001,
+    0xfffe,               0xffff,               0x7ffffffe,
+    0x7fffffff,           0x80000000,           0x80000001,
+    0xfffffffe,           0xffffffff,           0x100000000ll,
+    0x100000001ll,        0x7ffffffffffffffell, 0x7fffffffffffffffll,
+    0x8000000000000000ll, 0x8000000000000001ll, 0xfffffffffffffffell,
+    0xffffffffffffffffll
+  };
+  static const size_t NumValsF64 = sizeof(ValsF64) / sizeof(*ValsF64);
+
+  for (size_t i = 0; i < NumValsUi1; ++i) {
+    bool Val = ValsUi1[i];
+    testValue<bool>(Val, TotalTests, Passes, Failures);
+  }
+  for (size_t i = 0; i < NumValsUi8; ++i) {
+    uint8_t Val = ValsUi8[i];
+    testValue<uint8_t>(Val, TotalTests, Passes, Failures);
+  }
+  for (size_t i = 0; i < NumValsSi8; ++i) {
+    int8_t Val = ValsSi8[i];
+    testValue<int8_t>(Val, TotalTests, Passes, Failures);
+  }
+  for (size_t i = 0; i < NumValsUi16; ++i) {
+    uint16_t Val = ValsUi16[i];
+    testValue<uint16_t>(Val, TotalTests, Passes, Failures);
+  }
+  for (size_t i = 0; i < NumValsSi16; ++i) {
+    int16_t Val = ValsSi16[i];
+    testValue<int16_t>(Val, TotalTests, Passes, Failures);
+  }
+  for (size_t i = 0; i < NumValsUi32; ++i) {
+    uint32_t Val = ValsUi32[i];
+    testValue<uint32_t>(Val, TotalTests, Passes, Failures);
+    COMPARE(castBits, uint32_t, float, Val);
+  }
+  for (size_t i = 0; i < NumValsSi32; ++i) {
+    int32_t Val = ValsSi32[i];
+    testValue<int32_t>(Val, TotalTests, Passes, Failures);
+  }
+  for (size_t i = 0; i < NumValsUi64; ++i) {
+    uint64_t Val = ValsUi64[i];
+    testValue<uint64_t>(Val, TotalTests, Passes, Failures);
+    COMPARE(castBits, uint64_t, double, Val);
+  }
+  for (size_t i = 0; i < NumValsSi64; ++i) {
+    int64_t Val = ValsSi64[i];
+    testValue<int64_t>(Val, TotalTests, Passes, Failures);
+  }
+  for (size_t i = 0; i < NumValsF32; ++i) {
+    for (unsigned j = 0; j < 2; ++j) {
+      float Val = ValsF32[i];
+      if (j > 0)
+        Val = -Val;
+      testValue<float>(Val, TotalTests, Passes, Failures);
+      COMPARE(castBits, float, uint32_t, Val);
+    }
+  }
+  for (size_t i = 0; i < NumValsF64; ++i) {
+    for (unsigned j = 0; j < 2; ++j) {
+      double Val = ValsF64[i];
+      if (j > 0)
+        Val = -Val;
+      testValue<double>(Val, TotalTests, Passes, Failures);
+      COMPARE(castBits, double, uint64_t, Val);
+    }
+  }
+
+  std::cout << "TotalTests=" << TotalTests << " Passes=" << Passes
+            << " Failures=" << Failures << "\n";
+  return Failures;
+}
+
+////////////////////////////////////////////////////////////////
+
+// The following are helper definitions that should be part of the
+// Subzero runtime.
+
+extern "C" {
+uint32_t cvtdtoui32(double a) { return (uint32_t)a; }
+uint32_t cvtftoui32(float a) { return (uint32_t)a; }
+int64_t cvtdtosi64(double a) { return (int64_t)a; }
+int64_t cvtftosi64(float a) { return (int64_t)a; }
+uint64_t cvtdtoui64(double a) { return (uint64_t)a; }
+uint64_t cvtftoui64(float a) { return (uint64_t)a; }
+float cvtui64tof(uint64_t a) { return (float)a; }
+double cvtui64tod(uint64_t a) { return (double)a; }
+float cvtsi64tof(int64_t a) { return (float)a; }
+float cvtui32tof(uint32_t a) { return (float)a; }
+double cvtui32tod(uint32_t a) { return (double)a; }
+double cvtsi64tod(int64_t a) { return (double)a; }
+}
diff --git a/crosstest/test_cast_to_u1.ll b/crosstest/test_cast_to_u1.ll
new file mode 100644
index 0000000..f8a9ec6
--- /dev/null
+++ b/crosstest/test_cast_to_u1.ll
@@ -0,0 +1,92 @@
+target triple = "i686-pc-linux-gnu"
+
+define i32 @_Z4castIxbET0_T_(i64 %a) {
+entry:
+;  %tobool = icmp ne i64 %a, 0
+  %tobool = trunc i64 %a to i1
+  %tobool.ret_ext = zext i1 %tobool to i32
+  ret i32 %tobool.ret_ext
+}
+
+define i32 @_Z4castIybET0_T_(i64 %a) {
+entry:
+;  %tobool = icmp ne i64 %a, 0
+  %tobool = trunc i64 %a to i1
+  %tobool.ret_ext = zext i1 %tobool to i32
+  ret i32 %tobool.ret_ext
+}
+
+define i32 @_Z4castIibET0_T_(i32 %a) {
+entry:
+;  %tobool = icmp ne i32 %a, 0
+  %tobool = trunc i32 %a to i1
+  %tobool.ret_ext = zext i1 %tobool to i32
+  ret i32 %tobool.ret_ext
+}
+
+define i32 @_Z4castIjbET0_T_(i32 %a) {
+entry:
+;  %tobool = icmp ne i32 %a, 0
+  %tobool = trunc i32 %a to i1
+  %tobool.ret_ext = zext i1 %tobool to i32
+  ret i32 %tobool.ret_ext
+}
+
+define i32 @_Z4castIsbET0_T_(i32 %a) {
+entry:
+  %a.arg_trunc = trunc i32 %a to i16
+;  %tobool = icmp ne i16 %a.arg_trunc, 0
+  %tobool = trunc i16 %a.arg_trunc to i1
+  %tobool.ret_ext = zext i1 %tobool to i32
+  ret i32 %tobool.ret_ext
+}
+
+define i32 @_Z4castItbET0_T_(i32 %a) {
+entry:
+  %a.arg_trunc = trunc i32 %a to i16
+;  %tobool = icmp ne i16 %a.arg_trunc, 0
+  %tobool = trunc i16 %a.arg_trunc to i1
+  %tobool.ret_ext = zext i1 %tobool to i32
+  ret i32 %tobool.ret_ext
+}
+
+define i32 @_Z4castIabET0_T_(i32 %a) {
+entry:
+  %a.arg_trunc = trunc i32 %a to i8
+;  %tobool = icmp ne i8 %a.arg_trunc, 0
+  %tobool = trunc i8 %a.arg_trunc to i1
+  %tobool.ret_ext = zext i1 %tobool to i32
+  ret i32 %tobool.ret_ext
+}
+
+define i32 @_Z4castIhbET0_T_(i32 %a) {
+entry:
+  %a.arg_trunc = trunc i32 %a to i8
+;  %tobool = icmp ne i8 %a.arg_trunc, 0
+  %tobool = trunc i8 %a.arg_trunc to i1
+  %tobool.ret_ext = zext i1 %tobool to i32
+  ret i32 %tobool.ret_ext
+}
+
+define i32 @_Z4castIbbET0_T_(i32 %a) {
+entry:
+  %a.arg_trunc = trunc i32 %a to i1
+  %a.arg_trunc.ret_ext = zext i1 %a.arg_trunc to i32
+  ret i32 %a.arg_trunc.ret_ext
+}
+
+define i32 @_Z4castIdbET0_T_(double %a) {
+entry:
+;  %tobool = fcmp une double %a, 0.000000e+00
+  %tobool = fptoui double %a to i1
+  %tobool.ret_ext = zext i1 %tobool to i32
+  ret i32 %tobool.ret_ext
+}
+
+define i32 @_Z4castIfbET0_T_(float %a) {
+entry:
+;  %tobool = fcmp une float %a, 0.000000e+00
+  %tobool = fptoui float %a to i1
+  %tobool.ret_ext = zext i1 %tobool to i32
+  ret i32 %tobool.ret_ext
+}
diff --git a/crosstest/test_fcmp.def b/crosstest/test_fcmp.def
new file mode 100644
index 0000000..9f498b4
--- /dev/null
+++ b/crosstest/test_fcmp.def
@@ -0,0 +1,27 @@
+#ifndef TEST_FCMP_DEF
+#define TEST_FCMP_DEF
+
+#define XSTR(s) STR(s)
+#define STR(s) #s
+
+#define FCMP_TABLE \
+  /* cmp */ \
+  X(False) \
+  X(Oeq) \
+  X(Ogt) \
+  X(Oge) \
+  X(Olt) \
+  X(Ole) \
+  X(One) \
+  X(Ord) \
+  X(Ueq) \
+  X(Ugt) \
+  X(Uge) \
+  X(Ult) \
+  X(Ule) \
+  X(Une) \
+  X(Uno) \
+  X(True) \
+//#define X(cmp)
+
+#endif // TEST_FCMP_DEF
diff --git a/crosstest/test_fcmp.pnacl.ll b/crosstest/test_fcmp.pnacl.ll
new file mode 100644
index 0000000..7c4d42e
--- /dev/null
+++ b/crosstest/test_fcmp.pnacl.ll
@@ -0,0 +1,324 @@
+target triple = "i686-pc-linux-gnu"
+
+; This file is extracted from fp.pnacl.ll in the lit tests, with
+; the "internal" attribute removed from the functions.
+
+define i32 @fcmpFalseFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp false float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpFalseFloat:
+; CHECK: mov {{.*}}, 0
+
+define i32 @fcmpFalseDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp false double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpFalseDouble:
+; CHECK: mov {{.*}}, 0
+
+define i32 @fcmpOeqFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp oeq float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOeqFloat:
+; CHECK: ucomiss
+; CHECK: jne .
+; CHECK: jp .
+
+define i32 @fcmpOeqDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp oeq double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOeqDouble:
+; CHECK: ucomisd
+; CHECK: jne .
+; CHECK: jp .
+
+define i32 @fcmpOgtFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp ogt float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOgtFloat:
+; CHECK: ucomiss
+; CHECK: ja .
+
+define i32 @fcmpOgtDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp ogt double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOgtDouble:
+; CHECK: ucomisd
+; CHECK: ja .
+
+define i32 @fcmpOgeFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp oge float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOgeFloat:
+; CHECK: ucomiss
+; CHECK: jae .
+
+define i32 @fcmpOgeDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp oge double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOgeDouble:
+; CHECK: ucomisd
+; CHECK: jae .
+
+define i32 @fcmpOltFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp olt float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOltFloat:
+; CHECK: ucomiss
+; CHECK: ja .
+
+define i32 @fcmpOltDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp olt double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOltDouble:
+; CHECK: ucomisd
+; CHECK: ja .
+
+define i32 @fcmpOleFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp ole float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOleFloat:
+; CHECK: ucomiss
+; CHECK: jae .
+
+define i32 @fcmpOleDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp ole double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOleDouble:
+; CHECK: ucomisd
+; CHECK: jae .
+
+define i32 @fcmpOneFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp one float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOneFloat:
+; CHECK: ucomiss
+; CHECK: jne .
+
+define i32 @fcmpOneDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp one double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOneDouble:
+; CHECK: ucomisd
+; CHECK: jne .
+
+define i32 @fcmpOrdFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp ord float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOrdFloat:
+; CHECK: ucomiss
+; CHECK: jnp .
+
+define i32 @fcmpOrdDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp ord double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpOrdDouble:
+; CHECK: ucomisd
+; CHECK: jnp .
+
+define i32 @fcmpUeqFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp ueq float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUeqFloat:
+; CHECK: ucomiss
+; CHECK: je .
+
+define i32 @fcmpUeqDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp ueq double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUeqDouble:
+; CHECK: ucomisd
+; CHECK: je .
+
+define i32 @fcmpUgtFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp ugt float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUgtFloat:
+; CHECK: ucomiss
+; CHECK: jb .
+
+define i32 @fcmpUgtDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp ugt double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUgtDouble:
+; CHECK: ucomisd
+; CHECK: jb .
+
+define i32 @fcmpUgeFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp uge float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUgeFloat:
+; CHECK: ucomiss
+; CHECK: jbe .
+
+define i32 @fcmpUgeDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp uge double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUgeDouble:
+; CHECK: ucomisd
+; CHECK: jbe .
+
+define i32 @fcmpUltFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp ult float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUltFloat:
+; CHECK: ucomiss
+; CHECK: jb .
+
+define i32 @fcmpUltDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp ult double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUltDouble:
+; CHECK: ucomisd
+; CHECK: jb .
+
+define i32 @fcmpUleFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp ule float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUleFloat:
+; CHECK: ucomiss
+; CHECK: jbe .
+
+define i32 @fcmpUleDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp ule double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUleDouble:
+; CHECK: ucomisd
+; CHECK: jbe .
+
+define i32 @fcmpUneFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp une float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUneFloat:
+; CHECK: ucomiss
+; CHECK: je .
+; CHECK: jnp .
+
+define i32 @fcmpUneDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp une double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUneDouble:
+; CHECK: ucomisd
+; CHECK: je .
+; CHECK: jnp .
+
+define i32 @fcmpUnoFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp uno float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUnoFloat:
+; CHECK: ucomiss
+; CHECK: jp .
+
+define i32 @fcmpUnoDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp uno double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpUnoDouble:
+; CHECK: ucomisd
+; CHECK: jp .
+
+define i32 @fcmpTrueFloat(float %a, float %b) {
+entry:
+  %cmp = fcmp true float %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpTrueFloat:
+; CHECK: mov {{.*}}, 1
+
+define i32 @fcmpTrueDouble(double %a, double %b) {
+entry:
+  %cmp = fcmp true double %a, %b
+  %cmp.ret_ext = zext i1 %cmp to i32
+  ret i32 %cmp.ret_ext
+}
+; CHECK: fcmpTrueDouble:
+; CHECK: mov {{.*}}, 1
diff --git a/crosstest/test_fcmp_main.cpp b/crosstest/test_fcmp_main.cpp
new file mode 100644
index 0000000..8677c48
--- /dev/null
+++ b/crosstest/test_fcmp_main.cpp
@@ -0,0 +1,98 @@
+/* crosstest.py --test=test_fcmp.pnacl.ll --driver=test_fcmp_main.cpp \
+   --prefix=Subzero_ --output=test_fcmp */
+
+#include <cassert>
+#include <cfloat>
+#include <cmath>
+#include <iostream>
+
+#include "test_fcmp.def"
+
+#define X(cmp)                                                                 \
+  extern "C" bool fcmp##cmp##Float(float a, float b);                          \
+  extern "C" bool fcmp##cmp##Double(double a, double b);                       \
+  extern "C" bool Subzero_fcmp##cmp##Float(float a, float b);                  \
+  extern "C" bool Subzero_fcmp##cmp##Double(double a, double b);
+FCMP_TABLE;
+#undef X
+
+int main(int argc, char **argv) {
+  static const double NegInf = -1.0 / 0.0;
+  static const double Zero = 0.0;
+  static const double Ten = 10.0;
+  static const double PosInf = 1.0 / 0.0;
+  static const double Nan = 0.0 / 0.0;
+  assert(std::fpclassify(NegInf) == FP_INFINITE);
+  assert(std::fpclassify(PosInf) == FP_INFINITE);
+  assert(std::fpclassify(Nan) == FP_NAN);
+  assert(NegInf < Zero);
+  assert(NegInf < PosInf);
+  assert(Zero < PosInf);
+
+  volatile double Values[] = { NegInf,  Zero,    DBL_MIN, FLT_MIN, Ten,
+                               FLT_MAX, DBL_MAX, PosInf,  Nan, };
+  const static size_t NumValues = sizeof(Values) / sizeof(*Values);
+
+  typedef bool (*FuncTypeFloat)(float, float);
+  typedef bool (*FuncTypeDouble)(double, double);
+  static struct {
+    const char *Name;
+    FuncTypeFloat FuncFloatSz;
+    FuncTypeFloat FuncFloatLlc;
+    FuncTypeDouble FuncDoubleSz;
+    FuncTypeDouble FuncDoubleLlc;
+  } Funcs[] = {
+#define X(cmp)                                                                 \
+  {                                                                            \
+    "fcmp" STR(cmp), Subzero_fcmp##cmp##Float, fcmp##cmp##Float,               \
+        Subzero_fcmp##cmp##Double, fcmp##cmp##Double                           \
+  }                                                                            \
+  ,
+      FCMP_TABLE
+#undef X
+    };
+  const static size_t NumFuncs = sizeof(Funcs) / sizeof(*Funcs);
+
+  bool ResultSz, ResultLlc;
+
+  size_t TotalTests = 0;
+  size_t Passes = 0;
+  size_t Failures = 0;
+
+  for (size_t f = 0; f < NumFuncs; ++f) {
+    for (size_t i = 0; i < NumValues; ++i) {
+      for (size_t j = 0; j < NumValues; ++j) {
+        ++TotalTests;
+        float Value1Float = Values[i];
+        float Value2Float = Values[j];
+        ResultSz = Funcs[f].FuncFloatSz(Value1Float, Value2Float);
+        ResultLlc = Funcs[f].FuncFloatLlc(Value1Float, Value2Float);
+        if (ResultSz == ResultLlc) {
+          ++Passes;
+        } else {
+          ++Failures;
+          std::cout << Funcs[f].Name << "Float(" << Value1Float << ", "
+                    << Value2Float << "): sz=" << ResultSz
+                    << " llc=" << ResultLlc << std::endl;
+        }
+        ++TotalTests;
+        double Value1Double = Values[i];
+        double Value2Double = Values[j];
+        ResultSz = Funcs[f].FuncDoubleSz(Value1Double, Value2Double);
+        ResultLlc = Funcs[f].FuncDoubleLlc(Value1Double, Value2Double);
+        if (ResultSz == ResultLlc) {
+          ++Passes;
+        } else {
+          ++Failures;
+          std::cout << Funcs[f].Name << "Double(" << Value1Double << ", "
+                    << Value2Double << "): sz=" << ResultSz
+                    << " llc=" << ResultLlc << std::endl;
+        }
+      }
+    }
+  }
+
+  std::cout << "TotalTests=" << TotalTests << " Passes=" << Passes
+            << " Failures=" << Failures << "\n";
+  return Failures;
+}
diff --git a/crosstest/test_icmp.cpp b/crosstest/test_icmp.cpp
new file mode 100644
index 0000000..f1b144d
--- /dev/null
+++ b/crosstest/test_icmp.cpp
@@ -0,0 +1,21 @@
+// This aims to test the icmp bitcode instruction across all PNaCl
+// primitive integer types.
+
+#include <stdint.h>
+
+#include "test_icmp.h"
+
+#define X(cmp, op)                                                             \
+  bool icmp##cmp(uint8_t a, uint8_t b) { return a op b; }                      \
+  bool icmp##cmp(uint16_t a, uint16_t b) { return a op b; }                    \
+  bool icmp##cmp(uint32_t a, uint32_t b) { return a op b; }                    \
+  bool icmp##cmp(uint64_t a, uint64_t b) { return a op b; }
+ICMP_U_TABLE
+#undef X
+#define X(cmp, op)                                                             \
+  bool icmp##cmp(int8_t a, int8_t b) { return a op b; }                        \
+  bool icmp##cmp(int16_t a, int16_t b) { return a op b; }                      \
+  bool icmp##cmp(int32_t a, int32_t b) { return a op b; }                      \
+  bool icmp##cmp(int64_t a, int64_t b) { return a op b; }
+ICMP_S_TABLE
+#undef X
diff --git a/crosstest/test_icmp.def b/crosstest/test_icmp.def
new file mode 100644
index 0000000..c7cfc96
--- /dev/null
+++ b/crosstest/test_icmp.def
@@ -0,0 +1,25 @@
+#ifndef TEST_ICMP_DEF
+#define TEST_ICMP_DEF
+
+#define XSTR(s) STR(s)
+#define STR(s) #s
+
+#define ICMP_U_TABLE \
+  /* cmp, operator */ \
+  X(Eq,  ==) \
+  X(Ne,  !=) \
+  X(Ugt, >) \
+  X(Uge, >=) \
+  X(Ult, <) \
+  X(Ule, <=) \
+//#define X(cmp, op)
+
+#define ICMP_S_TABLE \
+  /* cmp, operator */ \
+  X(Sgt, >) \
+  X(Sge, >=) \
+  X(Slt, <) \
+  X(Sle, <=) \
+//#define X(cmp, op)
+
+#endif // TEST_ICMP_DEF
diff --git a/crosstest/test_icmp.h b/crosstest/test_icmp.h
new file mode 100644
index 0000000..d4ce9f1
--- /dev/null
+++ b/crosstest/test_icmp.h
@@ -0,0 +1,17 @@
+#include "test_icmp.def"
+
+#define X(cmp, op)                                                             \
+  bool icmp##cmp(uint8_t a, uint8_t b);                                        \
+  bool icmp##cmp(uint16_t a, uint16_t b);                                      \
+  bool icmp##cmp(uint32_t a, uint32_t b);                                      \
+  bool icmp##cmp(uint64_t a, uint64_t b);
+ICMP_U_TABLE
+#undef X
+
+#define X(cmp, op)                                                             \
+  bool icmp##cmp(int8_t a, int8_t b);                                          \
+  bool icmp##cmp(int16_t a, int16_t b);                                        \
+  bool icmp##cmp(int32_t a, int32_t b);                                        \
+  bool icmp##cmp(int64_t a, int64_t b);
+ICMP_S_TABLE
+#undef X
diff --git a/crosstest/test_icmp_main.cpp b/crosstest/test_icmp_main.cpp
new file mode 100644
index 0000000..3981fcf
--- /dev/null
+++ b/crosstest/test_icmp_main.cpp
@@ -0,0 +1,118 @@
+/* crosstest.py --test=test_icmp.cpp --driver=test_icmp_main.cpp \
+   --prefix=Subzero_ --output=test_icmp */
+
+#include <stdint.h>
+#include <iostream>
+
+// Include test_icmp.h twice - once normally, and once within the
+// Subzero_ namespace, corresponding to the llc and Subzero translated
+// object files, respectively.
+#include "test_icmp.h"
+namespace Subzero_ {
+#include "test_icmp.h"
+}
+
+volatile unsigned Values[] = { 0x0,        0x1,        0x7ffffffe, 0x7fffffff,
+                               0x80000000, 0x80000001, 0xfffffffe, 0xffffffff,
+                               0x7e,       0x7f,       0x80,       0x81,
+                               0xfe,       0xff,       0x100,      0x101,
+                               0x7ffe,     0x7fff,     0x8000,     0x8001,
+                               0xfffe,     0xffff,     0x10000,    0x10001, };
+const static size_t NumValues = sizeof(Values) / sizeof(*Values);
+
+template <typename TypeUnsigned, typename TypeSigned>
+void testsInt(size_t &TotalTests, size_t &Passes, size_t &Failures) {
+  typedef bool (*FuncTypeUnsigned)(TypeUnsigned, TypeUnsigned);
+  typedef bool (*FuncTypeSigned)(TypeSigned, TypeSigned);
+  static struct {
+    const char *Name;
+    FuncTypeUnsigned FuncLlc;
+    FuncTypeUnsigned FuncSz;
+  } Funcs[] = {
+#define X(cmp, op)                                                             \
+  {                                                                            \
+    STR(inst), (FuncTypeUnsigned)icmp##cmp,                                    \
+        (FuncTypeUnsigned)Subzero_::icmp##cmp                                  \
+  }                                                                            \
+  ,
+      ICMP_U_TABLE
+#undef X
+#define X(cmp, op)                                                             \
+  {                                                                            \
+    STR(inst), (FuncTypeUnsigned)(FuncTypeSigned)icmp##cmp,                    \
+        (FuncTypeUnsigned)(FuncTypeSigned)Subzero_::icmp##cmp                  \
+  }                                                                            \
+  ,
+          ICMP_S_TABLE
+#undef X
+    };
+  const static size_t NumFuncs = sizeof(Funcs) / sizeof(*Funcs);
+
+  if (sizeof(TypeUnsigned) <= sizeof(uint32_t)) {
+    // This is the "normal" version of the loop nest, for 32-bit or
+    // narrower types.
+    for (size_t f = 0; f < NumFuncs; ++f) {
+      for (size_t i = 0; i < NumValues; ++i) {
+        for (size_t j = 0; j < NumValues; ++j) {
+          TypeUnsigned Value1 = Values[i];
+          TypeUnsigned Value2 = Values[j];
+          ++TotalTests;
+          bool ResultSz = Funcs[f].FuncSz(Value1, Value2);
+          bool ResultLlc = Funcs[f].FuncLlc(Value1, Value2);
+          if (ResultSz == ResultLlc) {
+            ++Passes;
+          } else {
+            ++Failures;
+            std::cout << "icmp" << Funcs[f].Name << (8 * sizeof(TypeUnsigned))
+                      << "(" << Value1 << ", " << Value2 << "): sz=" << ResultSz
+                      << " llc=" << ResultLlc << std::endl;
+          }
+        }
+      }
+    }
+  } else {
+    // This is the 64-bit version.  Test values are synthesized from
+    // the 32-bit values in Values[].
+    for (size_t f = 0; f < NumFuncs; ++f) {
+      for (size_t iLo = 0; iLo < NumValues; ++iLo) {
+        for (size_t iHi = 0; iHi < NumValues; ++iHi) {
+          for (size_t jLo = 0; jLo < NumValues; ++jLo) {
+            for (size_t jHi = 0; jHi < NumValues; ++jHi) {
+              TypeUnsigned Value1 =
+                  (((TypeUnsigned)Values[iHi]) << 32) + Values[iLo];
+              TypeUnsigned Value2 =
+                  (((TypeUnsigned)Values[jHi]) << 32) + Values[jLo];
+              ++TotalTests;
+              bool ResultSz = Funcs[f].FuncSz(Value1, Value2);
+              bool ResultLlc = Funcs[f].FuncLlc(Value1, Value2);
+              if (ResultSz == ResultLlc) {
+                ++Passes;
+              } else {
+                ++Failures;
+                std::cout << "icmp" << Funcs[f].Name
+                          << (8 * sizeof(TypeUnsigned)) << "(" << Value1 << ", "
+                          << Value2 << "): sz=" << ResultSz
+                          << " llc=" << ResultLlc << std::endl;
+              }
+            }
+          }
+        }
+      }
+    }
+  }
+}
+
+int main(int argc, char **argv) {
+  size_t TotalTests = 0;
+  size_t Passes = 0;
+  size_t Failures = 0;
+
+  testsInt<uint8_t, int8_t>(TotalTests, Passes, Failures);
+  testsInt<uint16_t, int16_t>(TotalTests, Passes, Failures);
+  testsInt<uint32_t, int32_t>(TotalTests, Passes, Failures);
+  testsInt<uint64_t, int64_t>(TotalTests, Passes, Failures);
+
+  std::cout << "TotalTests=" << TotalTests << " Passes=" << Passes
+            << " Failures=" << Failures << "\n";
+  return Failures;
+}
diff --git a/pydir/build-pnacl-ir.py b/pydir/build-pnacl-ir.py
new file mode 100755
index 0000000..67dbefd
--- /dev/null
+++ b/pydir/build-pnacl-ir.py
@@ -0,0 +1,50 @@
+#!/usr/bin/env python2
+
+import argparse
+import os
+import sys
+import tempfile
+from utils import shellcmd
+
+if __name__ == '__main__':
+    argparser = argparse.ArgumentParser()
+    argparser.add_argument('cfile', nargs='+', type=str,
+        help='C file(s) to convert')
+    argparser.add_argument('--nacl_sdk_root', nargs='?', type=str,
+        help='Path to NACL_SDK_ROOT')
+    argparser.add_argument('--dir', nargs='?', type=str, default='.',
+                           help='Output directory')
+    argparser.add_argument('--disable-verify', action='store_true')
+    args = argparser.parse_args()
+
+    nacl_sdk_root = os.environ.get('NACL_SDK_ROOT', None)
+    if args.nacl_sdk_root:
+        nacl_sdk_root = os.path.expanduser(args.nacl_sdk_root)
+
+    if not nacl_sdk_root or not os.path.exists(nacl_sdk_root):
+        print '''\
+Please set the NACL_SDK_ROOT environment variable or pass the path through
+--nacl_sdk_root to point to a valid Native Client SDK installation.'''
+        sys.exit(1)
+
+    includes_path = os.path.join(nacl_sdk_root, 'include')
+    toolchain_path = os.path.join(nacl_sdk_root, 'toolchain', 'linux_pnacl')
+    clang_path = os.path.join(toolchain_path, 'bin64', 'pnacl-clang')
+    opt_path = os.path.join(toolchain_path, 'host_x86_64', 'bin', 'opt')
+
+    tempdir = tempfile.mkdtemp()
+
+    for cname in args.cfile:
+        basename = os.path.splitext(cname)[0]
+        llname = os.path.join(tempdir, basename + '.ll')
+        pnaclname = basename + '.pnacl.ll'
+        pnaclname = os.path.join(args.dir, pnaclname)
+
+        shellcmd(clang_path + ' -I{0} -c {1} -o {2}'.format(
+            includes_path, cname, llname))
+        shellcmd(opt_path +
+            ' -O2 -pnacl-abi-simplify-preopt -pnacl-abi-simplify-postopt' +
+            ('' if args.disable_verify else
+             ' -verify-pnaclabi-module -verify-pnaclabi-functions') +
+            ' -pnaclabi-allow-debug-metadata -disable-simplify-libcalls'
+            ' {0} -S -o {1}'.format(llname, pnaclname))
diff --git a/src/IceCfg.cpp b/src/IceCfg.cpp
index f2b1cc9..3d720fa 100644
--- a/src/IceCfg.cpp
+++ b/src/IceCfg.cpp
@@ -17,13 +17,16 @@
 #include "IceDefs.h"
 #include "IceInst.h"
 #include "IceOperand.h"
+#include "IceTargetLowering.h"
 
 namespace Ice {
 
 Cfg::Cfg(GlobalContext *Ctx)
     : Ctx(Ctx), FunctionName(""), ReturnType(IceType_void),
       IsInternalLinkage(false), HasError(false), ErrorMessage(""), Entry(NULL),
-      NextInstNumber(1), CurrentNode(NULL) {}
+      NextInstNumber(1),
+      Target(TargetLowering::createLowering(Ctx->getTargetArch(), this)),
+      CurrentNode(NULL) {}
 
 Cfg::~Cfg() {}
 
@@ -54,14 +57,107 @@
   Args.push_back(Arg);
 }
 
+// Returns whether the stack frame layout has been computed yet.  This
+// is used for dumping the stack frame location of Variables.
+bool Cfg::hasComputedFrame() const { return getTarget()->hasComputedFrame(); }
+
+void Cfg::translate() {
+  Ostream &Str = Ctx->getStrDump();
+  if (hasError())
+    return;
+
+  if (Ctx->isVerbose()) {
+    Str << "================ Initial CFG ================\n";
+    dump();
+  }
+
+  Timer T_translate;
+  // The set of translation passes and their order are determined by
+  // the target.
+  getTarget()->translate();
+  T_translate.printElapsedUs(getContext(), "translate()");
+
+  if (Ctx->isVerbose()) {
+    Str << "================ Final output ================\n";
+    dump();
+  }
+}
+
 void Cfg::computePredecessors() {
   for (NodeList::iterator I = Nodes.begin(), E = Nodes.end(); I != E; ++I) {
     (*I)->computePredecessors();
   }
 }
 
+// placePhiLoads() must be called before placePhiStores().
+void Cfg::placePhiLoads() {
+  for (NodeList::iterator I = Nodes.begin(), E = Nodes.end(); I != E; ++I) {
+    (*I)->placePhiLoads();
+  }
+}
+
+// placePhiStores() must be called after placePhiLoads().
+void Cfg::placePhiStores() {
+  for (NodeList::iterator I = Nodes.begin(), E = Nodes.end(); I != E; ++I) {
+    (*I)->placePhiStores();
+  }
+}
+
+void Cfg::deletePhis() {
+  for (NodeList::iterator I = Nodes.begin(), E = Nodes.end(); I != E; ++I) {
+    (*I)->deletePhis();
+  }
+}
+
+void Cfg::genCode() {
+  for (NodeList::iterator I = Nodes.begin(), E = Nodes.end(); I != E; ++I) {
+    (*I)->genCode();
+  }
+}
+
+// Compute the stack frame layout.
+void Cfg::genFrame() {
+  getTarget()->addProlog(Entry);
+  // TODO: Consider folding epilog generation into the final
+  // emission/assembly pass to avoid an extra iteration over the node
+  // list.  Or keep a separate list of exit nodes.
+  for (NodeList::iterator I = Nodes.begin(), E = Nodes.end(); I != E; ++I) {
+    CfgNode *Node = *I;
+    if (Node->getHasReturn())
+      getTarget()->addEpilog(Node);
+  }
+}
+
 // ======================== Dump routines ======================== //
 
+void Cfg::emit() {
+  Ostream &Str = Ctx->getStrEmit();
+  Timer T_emit;
+  if (!Ctx->testAndSetHasEmittedFirstMethod()) {
+    // Print a helpful command for assembling the output.
+    // TODO: have the Target emit the header
+    // TODO: need a per-file emit in addition to per-CFG
+    Str << "# $LLVM_BIN_PATH/llvm-mc"
+        << " -arch=x86"
+        << " -x86-asm-syntax=intel"
+        << " -filetype=obj"
+        << " -o=MyObj.o"
+        << "\n\n";
+  }
+  Str << "\t.text\n";
+  if (!getInternal()) {
+    IceString MangledName = getContext()->mangleName(getFunctionName());
+    Str << "\t.globl\t" << MangledName << "\n";
+    Str << "\t.type\t" << MangledName << ",@function\n";
+  }
+  for (NodeList::const_iterator I = Nodes.begin(), E = Nodes.end(); I != E;
+       ++I) {
+    (*I)->emit(this);
+  }
+  Str << "\n";
+  T_emit.printElapsedUs(Ctx, "emit()");
+}
+
 void Cfg::dump() {
   Ostream &Str = Ctx->getStrDump();
   setCurrentNode(getEntryNode());
diff --git a/src/IceCfg.h b/src/IceCfg.h
index 05e1e3b..5f3bfd3 100644
--- a/src/IceCfg.h
+++ b/src/IceCfg.h
@@ -58,7 +58,7 @@
   const NodeList &getNodes() const { return Nodes; }
 
   // Manage instruction numbering.
-  int newInstNumber() { return NextInstNumber++; }
+  int32_t newInstNumber() { return NextInstNumber++; }
 
   // Manage Variables.
   Variable *makeVariable(Type Ty, const CfgNode *Node,
@@ -70,16 +70,28 @@
   void addArg(Variable *Arg);
   const VarList &getArgs() const { return Args; }
 
+  // Miscellaneous accessors.
+  TargetLowering *getTarget() const { return Target.get(); }
+  bool hasComputedFrame() const;
+
+  // Passes over the CFG.
+  void translate();
   // After the CFG is fully constructed, iterate over the nodes and
   // compute the predecessor edges, in the form of
   // CfgNode::InEdges[].
   void computePredecessors();
+  void placePhiLoads();
+  void placePhiStores();
+  void deletePhis();
+  void genCode();
+  void genFrame();
 
   // Manage the CurrentNode field, which is used for validating the
   // Variable::DefNode field during dumping/emitting.
   void setCurrentNode(const CfgNode *Node) { CurrentNode = Node; }
   const CfgNode *getCurrentNode() const { return CurrentNode; }
 
+  void emit();
   void dump();
 
   // Allocate data of type T using the per-Cfg allocator.
@@ -124,9 +136,10 @@
   IceString ErrorMessage;
   CfgNode *Entry; // entry basic block
   NodeList Nodes; // linearized node list; Entry should be first
-  int NextInstNumber;
+  int32_t NextInstNumber;
   VarList Variables;
   VarList Args; // subset of Variables, in argument order
+  llvm::OwningPtr<TargetLowering> Target;
 
   // CurrentNode is maintained during dumping/emitting just for
   // validating Variable::DefNode.  Normally, a traversal over
diff --git a/src/IceCfgNode.cpp b/src/IceCfgNode.cpp
index fe8b70e0..c00b2c7 100644
--- a/src/IceCfgNode.cpp
+++ b/src/IceCfgNode.cpp
@@ -7,8 +7,8 @@
 //
 //===----------------------------------------------------------------------===//
 //
-// This file implements the CfgNode class, including the
-// complexities of instruction insertion and in-edge calculation.
+// This file implements the CfgNode class, including the complexities
+// of instruction insertion and in-edge calculation.
 //
 //===----------------------------------------------------------------------===//
 
@@ -16,11 +16,12 @@
 #include "IceCfgNode.h"
 #include "IceInst.h"
 #include "IceOperand.h"
+#include "IceTargetLowering.h"
 
 namespace Ice {
 
 CfgNode::CfgNode(Cfg *Func, SizeT LabelNumber, IceString Name)
-    : Func(Func), Number(LabelNumber), Name(Name) {}
+    : Func(Func), Number(LabelNumber), Name(Name), HasReturn(false) {}
 
 // Returns the name the node was created with.  If no name was given,
 // it synthesizes a (hopefully) unique name.
@@ -61,8 +62,135 @@
   }
 }
 
+// This does part 1 of Phi lowering, by creating a new dest variable
+// for each Phi instruction, replacing the Phi instruction's dest with
+// that variable, and adding an explicit assignment of the old dest to
+// the new dest.  For example,
+//   a=phi(...)
+// changes to
+//   "a_phi=phi(...); a=a_phi".
+//
+// This is in preparation for part 2 which deletes the Phi
+// instructions and appends assignment instructions to predecessor
+// blocks.  Note that this transformation preserves SSA form.
+void CfgNode::placePhiLoads() {
+  for (PhiList::iterator I = Phis.begin(), E = Phis.end(); I != E; ++I) {
+    Inst *Inst = (*I)->lower(Func, this);
+    Insts.insert(Insts.begin(), Inst);
+    Inst->updateVars(this);
+  }
+}
+
+// This does part 2 of Phi lowering.  For each Phi instruction at each
+// out-edge, create a corresponding assignment instruction, and add
+// all the assignments near the end of this block.  They need to be
+// added before any branch instruction, and also if the block ends
+// with a compare instruction followed by a branch instruction that we
+// may want to fuse, it's better to insert the new assignments before
+// the compare instruction.
+//
+// Note that this transformation takes the Phi dest variables out of
+// SSA form, as there may be assignments to the dest variable in
+// multiple blocks.
+//
+// TODO: Defer this pass until after register allocation, then split
+// critical edges, add the assignments, and lower them.  This should
+// reduce the amount of shuffling at the end of each block.
+void CfgNode::placePhiStores() {
+  // Find the insertion point.  TODO: After branch/compare fusing is
+  // implemented, try not to insert Phi stores between the compare and
+  // conditional branch instructions, otherwise the branch/compare
+  // pattern matching may fail.  However, the branch/compare sequence
+  // will have to be broken if the compare result is read (by the
+  // assignment) before it is written (by the compare).
+  InstList::iterator InsertionPoint = Insts.end();
+  // Every block must end in a terminator instruction.
+  assert(InsertionPoint != Insts.begin());
+  --InsertionPoint;
+  // Confirm via assert() that InsertionPoint is a terminator
+  // instruction.  Calling getTerminatorEdges() on a non-terminator
+  // instruction will cause an llvm_unreachable().
+  assert(((*InsertionPoint)->getTerminatorEdges(), true));
+
+  // Consider every out-edge.
+  for (NodeList::const_iterator I1 = OutEdges.begin(), E1 = OutEdges.end();
+       I1 != E1; ++I1) {
+    CfgNode *Target = *I1;
+    // Consider every Phi instruction at the out-edge.
+    for (PhiList::const_iterator I2 = Target->Phis.begin(),
+                                 E2 = Target->Phis.end();
+         I2 != E2; ++I2) {
+      Operand *Operand = (*I2)->getOperandForTarget(this);
+      assert(Operand);
+      Variable *Dest = (*I2)->getDest();
+      assert(Dest);
+      InstAssign *NewInst = InstAssign::create(Func, Dest, Operand);
+      // If Src is a variable, set the Src and Dest variables to
+      // prefer each other for register allocation.
+      if (Variable *Src = llvm::dyn_cast<Variable>(Operand)) {
+        bool AllowOverlap = false;
+        Dest->setPreferredRegister(Src, AllowOverlap);
+        Src->setPreferredRegister(Dest, AllowOverlap);
+      }
+      Insts.insert(InsertionPoint, NewInst);
+      NewInst->updateVars(this);
+    }
+  }
+}
+
+// Deletes the phi instructions after the loads and stores are placed.
+void CfgNode::deletePhis() {
+  for (PhiList::iterator I = Phis.begin(), E = Phis.end(); I != E; ++I) {
+    (*I)->setDeleted();
+  }
+}
+
+// Drives the target lowering.  Passes the current instruction and the
+// next non-deleted instruction for target lowering.
+void CfgNode::genCode() {
+  TargetLowering *Target = Func->getTarget();
+  LoweringContext &Context = Target->getContext();
+  // Lower only the regular instructions.  Defer the Phi instructions.
+  Context.init(this);
+  while (!Context.atEnd()) {
+    InstList::iterator Orig = Context.getCur();
+    if (llvm::isa<InstRet>(*Orig))
+      setHasReturn();
+    Target->lower();
+    // Ensure target lowering actually moved the cursor.
+    assert(Context.getCur() != Orig);
+  }
+}
+
 // ======================== Dump routines ======================== //
 
+void CfgNode::emit(Cfg *Func) const {
+  Func->setCurrentNode(this);
+  Ostream &Str = Func->getContext()->getStrEmit();
+  if (Func->getEntryNode() == this) {
+    Str << Func->getContext()->mangleName(Func->getFunctionName()) << ":\n";
+  }
+  Str << getAsmName() << ":\n";
+  for (PhiList::const_iterator I = Phis.begin(), E = Phis.end(); I != E; ++I) {
+    InstPhi *Inst = *I;
+    if (Inst->isDeleted())
+      continue;
+    // Emitting a Phi instruction should cause an error.
+    Inst->emit(Func);
+  }
+  for (InstList::const_iterator I = Insts.begin(), E = Insts.end(); I != E;
+       ++I) {
+    Inst *Inst = *I;
+    if (Inst->isDeleted())
+      continue;
+    // Here we detect redundant assignments like "mov eax, eax" and
+    // suppress them.
+    if (Inst->isRedundantAssign())
+      continue;
+    (*I)->emit(Func);
+  }
+}
+
 void CfgNode::dump(Cfg *Func) const {
   Func->setCurrentNode(this);
   Ostream &Str = Func->getContext()->getStrDump();
diff --git a/src/IceCfgNode.h b/src/IceCfgNode.h
index bf96aef..645dc57 100644
--- a/src/IceCfgNode.h
+++ b/src/IceCfgNode.h
@@ -29,6 +29,14 @@
   // Access the label number and name for this node.
   SizeT getIndex() const { return Number; }
   IceString getName() const;
+  IceString getAsmName() const {
+    return ".L" + Func->getFunctionName() + "$" + getName();
+  }
+
+  // The HasReturn flag indicates that this node contains a return
+  // instruction and therefore needs an epilog.
+  void setHasReturn() { HasReturn = true; }
+  bool getHasReturn() const { return HasReturn; }
 
   // Access predecessor and successor edge lists.
   const NodeList &getInEdges() const { return InEdges; }
@@ -42,6 +50,11 @@
   // node's successors.
   void computePredecessors();
 
+  void placePhiLoads();
+  void placePhiStores();
+  void deletePhis();
+  void genCode();
+  void emit(Cfg *Func) const;
   void dump(Cfg *Func) const;
 
 private:
@@ -51,6 +64,7 @@
   Cfg *const Func;
   const SizeT Number; // label index
   IceString Name;     // for dumping only
+  bool HasReturn;     // does this block need an epilog?
   NodeList InEdges;   // in no particular order
   NodeList OutEdges;  // in no particular order
   PhiList Phis;       // unordered set of phi instructions
diff --git a/src/IceDefs.h b/src/IceDefs.h
index 25c384a..6c5cb1a 100644
--- a/src/IceDefs.h
+++ b/src/IceDefs.h
@@ -35,16 +35,23 @@
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Support/Timer.h"
 
+// Roll our own static_assert<> in the absence of C++11.  TODO: change
+// to static_assert<> with C++11.
+template <bool> struct staticAssert;
+template <> struct staticAssert<true> {}; // only true is defined
+#define STATIC_ASSERT(x) staticAssert<(x)>()
+
 namespace Ice {
 
+class Cfg;
 class CfgNode;
 class Constant;
 class GlobalContext;
-class Cfg;
 class Inst;
 class InstPhi;
 class InstTarget;
 class Operand;
+class TargetLowering;
 class Variable;
 
 // TODO: Switch over to LLVM's ADT container classes.
diff --git a/src/IceGlobalContext.cpp b/src/IceGlobalContext.cpp
index 11de013..ab63b4c 100644
--- a/src/IceGlobalContext.cpp
+++ b/src/IceGlobalContext.cpp
@@ -17,6 +17,7 @@
 #include "IceCfg.h"
 #include "IceGlobalContext.h"
 #include "IceOperand.h"
+#include "IceTargetLowering.h"
 
 namespace Ice {
 
@@ -75,18 +76,17 @@
 
 GlobalContext::GlobalContext(llvm::raw_ostream *OsDump,
                              llvm::raw_ostream *OsEmit, VerboseMask Mask,
+                             TargetArch Arch, OptLevel Opt,
                              IceString TestPrefix)
     : StrDump(OsDump), StrEmit(OsEmit), VMask(Mask),
-      ConstPool(new ConstantPool()), TestPrefix(TestPrefix) {}
+      ConstPool(new ConstantPool()), Arch(Arch), Opt(Opt),
+      TestPrefix(TestPrefix), HasEmittedFirstMethod(false) {}
 
 // In this context, name mangling means to rewrite a symbol using a
 // given prefix.  For a C++ symbol, nest the original symbol inside
 // the "prefix" namespace.  For other symbols, just prepend the
 // prefix.
 IceString GlobalContext::mangleName(const IceString &Name) const {
-  // TODO: Add explicit tests (beyond the implicit tests in the linker
-  // that come from the cross tests).
-  //
   // An already-nested name like foo::bar() gets pushed down one
   // level, making it equivalent to Prefix::foo::bar().
   //   _ZN3foo3barExyz ==> _ZN6Prefix3foo3barExyz
@@ -100,9 +100,9 @@
 
   unsigned PrefixLength = getTestPrefix().length();
   char NameBase[1 + Name.length()];
-  const size_t BufLen = 30 + Name.length() + getTestPrefix().length();
+  const size_t BufLen = 30 + Name.length() + PrefixLength;
   char NewName[BufLen];
-  uint32_t BaseLength = 0;
+  uint32_t BaseLength = 0; // using uint32_t due to sscanf format string
 
   int ItemsParsed = sscanf(Name.c_str(), "_ZN%s", NameBase);
   if (ItemsParsed == 1) {
@@ -118,15 +118,28 @@
   }
 
   ItemsParsed = sscanf(Name.c_str(), "_Z%u%s", &BaseLength, NameBase);
-  if (ItemsParsed == 2) {
-    // Transform _Z3barxyz ==> ZN6Prefix3barExyz
-    //                          ^^^^^^^^    ^
+  if (ItemsParsed == 2 && BaseLength <= strlen(NameBase)) {
+    // Transform _Z3barxyz ==> _ZN6Prefix3barExyz
+    //                           ^^^^^^^^    ^
     // (splice in "N6Prefix", and insert "E" after "3bar")
+    // But an "I" after the identifier indicates a template argument
+    // list terminated with "E"; insert the new "E" before/after the
+    // old "E".  E.g.:
+    // Transform _Z3barIabcExyz ==> _ZN6Prefix3barIabcEExyz
+    //                                ^^^^^^^^         ^
+    // (splice in "N6Prefix", and insert "E" after "3barIabcE")
     char OrigName[Name.length()];
     char OrigSuffix[Name.length()];
-    strncpy(OrigName, NameBase, BaseLength);
-    OrigName[BaseLength] = '\0';
-    strcpy(OrigSuffix, NameBase + BaseLength);
+    uint32_t ActualBaseLength = BaseLength;
+    if (NameBase[ActualBaseLength] == 'I') {
+      ++ActualBaseLength;
+      while (NameBase[ActualBaseLength] != 'E' &&
+             NameBase[ActualBaseLength] != '\0')
+        ++ActualBaseLength;
+    }
+    strncpy(OrigName, NameBase, ActualBaseLength);
+    OrigName[ActualBaseLength] = '\0';
+    strcpy(OrigSuffix, NameBase + ActualBaseLength);
     snprintf(NewName, BufLen, "_ZN%u%s%u%sE%s", PrefixLength,
              getTestPrefix().c_str(), BaseLength, OrigName, OrigSuffix);
     return NewName;
diff --git a/src/IceGlobalContext.h b/src/IceGlobalContext.h
index 9224d89..1ad5f07 100644
--- a/src/IceGlobalContext.h
+++ b/src/IceGlobalContext.h
@@ -30,7 +30,8 @@
 class GlobalContext {
 public:
   GlobalContext(llvm::raw_ostream *OsDump, llvm::raw_ostream *OsEmit,
-                VerboseMask Mask, IceString TestPrefix);
+                VerboseMask Mask, TargetArch Arch, OptLevel Opt,
+                IceString TestPrefix);
   ~GlobalContext();
 
   // Returns true if any of the specified options in the verbose mask
@@ -48,6 +49,9 @@
   Ostream &getStrDump() { return StrDump; }
   Ostream &getStrEmit() { return StrEmit; }
 
+  TargetArch getTargetArch() const { return Arch; }
+  OptLevel getOptLevel() const { return Opt; }
+
   // When emitting assembly, we allow a string to be prepended to
   // names of translated functions.  This makes it easier to create an
   // execution test against a reference translator like llc, with both
@@ -55,6 +59,15 @@
   IceString getTestPrefix() const { return TestPrefix; }
   IceString mangleName(const IceString &Name) const;
 
+  // The purpose of HasEmitted is to add a header comment at the
+  // beginning of assembly code emission, doing it once per file
+  // rather than once per function.
+  bool testAndSetHasEmittedFirstMethod() {
+    bool HasEmitted = HasEmittedFirstMethod;
+    HasEmittedFirstMethod = true;
+    return HasEmitted;
+  }
+
   // Manage Constants.
   // getConstant*() functions are not const because they might add
   // something to the constant pool.
@@ -75,7 +88,10 @@
   llvm::BumpPtrAllocator Allocator;
   VerboseMask VMask;
   llvm::OwningPtr<class ConstantPool> ConstPool;
+  const TargetArch Arch;
+  const OptLevel Opt;
   const IceString TestPrefix;
+  bool HasEmittedFirstMethod;
   GlobalContext(const GlobalContext &) LLVM_DELETED_FUNCTION;
   GlobalContext &operator=(const GlobalContext &) LLVM_DELETED_FUNCTION;
 };
diff --git a/src/IceInst.cpp b/src/IceInst.cpp
index 391f197..3f0c97d 100644
--- a/src/IceInst.cpp
+++ b/src/IceInst.cpp
@@ -22,7 +22,7 @@
 namespace {
 
 // Using non-anonymous struct so that array_lengthof works.
-const struct _InstArithmeticAttributes {
+const struct InstArithmeticAttributes_ {
   const char *DisplayString;
   bool IsCommutative;
 } InstArithmeticAttributes[] = {
@@ -36,7 +36,7 @@
     llvm::array_lengthof(InstArithmeticAttributes);
 
 // Using non-anonymous struct so that array_lengthof works.
-const struct _InstCastAttributes {
+const struct InstCastAttributes_ {
   const char *DisplayString;
 } InstCastAttributes[] = {
 #define X(tag, str)                                                            \
@@ -48,7 +48,7 @@
 const size_t InstCastAttributesSize = llvm::array_lengthof(InstCastAttributes);
 
 // Using non-anonymous struct so that array_lengthof works.
-const struct _InstFcmpAttributes {
+const struct InstFcmpAttributes_ {
   const char *DisplayString;
 } InstFcmpAttributes[] = {
 #define X(tag, str)                                                            \
@@ -60,7 +60,7 @@
 const size_t InstFcmpAttributesSize = llvm::array_lengthof(InstFcmpAttributes);
 
 // Using non-anonymous struct so that array_lengthof works.
-const struct _InstIcmpAttributes {
+const struct InstIcmpAttributes_ {
   const char *DisplayString;
 } InstIcmpAttributes[] = {
 #define X(tag, str)                                                            \
@@ -180,6 +180,35 @@
   addSource(Source);
 }
 
+// Find the source operand corresponding to the incoming edge for the
+// given node.  TODO: This uses a linear-time search, which could be
+// improved if it becomes a problem.
+Operand *InstPhi::getOperandForTarget(CfgNode *Target) const {
+  for (SizeT I = 0; I < getSrcSize(); ++I) {
+    if (Labels[I] == Target)
+      return getSrc(I);
+  }
+  llvm_unreachable("Phi target not found");
+  return NULL;
+}
+
+// Change "a=phi(...)" to "a_phi=phi(...)" and return a new
+// instruction "a=a_phi".
+Inst *InstPhi::lower(Cfg *Func, CfgNode *Node) {
+  Variable *Dest = getDest();
+  assert(Dest);
+  IceString PhiName = Dest->getName() + "_phi";
+  Variable *NewSrc = Func->makeVariable(Dest->getType(), Node, PhiName);
+  this->Dest = NewSrc;
+  InstAssign *NewInst = InstAssign::create(Func, Dest, NewSrc);
+  // Set Dest and NewSrc to have affinity with each other, as a hint
+  // for register allocation.
+  Dest->setPreferredRegister(NewSrc, false);
+  NewSrc->setPreferredRegister(Dest, false);
+  Dest->replaceDefinition(NewInst, Node);
+  return NewInst;
+}
+
 InstRet::InstRet(Cfg *Func, Operand *RetValue)
     : Inst(Func, Ret, RetValue ? 1 : 0, NULL) {
   if (RetValue)
@@ -233,11 +262,35 @@
 InstUnreachable::InstUnreachable(Cfg *Func)
     : Inst(Func, Inst::Unreachable, 0, NULL) {}
 
+InstFakeDef::InstFakeDef(Cfg *Func, Variable *Dest, Variable *Src)
+    : Inst(Func, Inst::FakeDef, Src ? 1 : 0, Dest) {
+  assert(Dest);
+  if (Src)
+    addSource(Src);
+}
+
+InstFakeUse::InstFakeUse(Cfg *Func, Variable *Src)
+    : Inst(Func, Inst::FakeUse, 1, NULL) {
+  assert(Src);
+  addSource(Src);
+}
+
+InstFakeKill::InstFakeKill(Cfg *Func, const VarList &KilledRegs,
+                           const Inst *Linked)
+    : Inst(Func, Inst::FakeKill, KilledRegs.size(), NULL), Linked(Linked) {
+  for (VarList::const_iterator I = KilledRegs.begin(), E = KilledRegs.end();
+       I != E; ++I) {
+    Variable *Var = *I;
+    addSource(Var);
+  }
+}
+
 // ======================== Dump routines ======================== //
 
 void Inst::dumpDecorated(const Cfg *Func) const {
   Ostream &Str = Func->getContext()->getStrDump();
-  if (!Func->getContext()->isVerbose(IceV_Deleted) && isDeleted())
+  if (!Func->getContext()->isVerbose(IceV_Deleted) &&
+      (isDeleted() || isRedundantAssign()))
     return;
   if (Func->getContext()->isVerbose(IceV_InstNumbers)) {
     char buf[30];
@@ -255,6 +308,10 @@
   Str << "\n";
 }
 
+void Inst::emit(const Cfg * /*Func*/) const {
+  llvm_unreachable("emit() called on a non-lowered instruction");
+}
+
 void Inst::dump(const Cfg *Func) const {
   Ostream &Str = Func->getContext()->getStrDump();
   dumpDest(Func);
@@ -271,6 +328,15 @@
   }
 }
 
+void Inst::emitSources(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  for (SizeT I = 0; I < getSrcSize(); ++I) {
+    if (I > 0)
+      Str << ", ";
+    getSrc(I)->emit(Func);
+  }
+}
+
 void Inst::dumpDest(const Cfg *Func) const {
   if (getDest())
     getDest()->dump(Func);
@@ -406,7 +472,7 @@
 
 void InstRet::dump(const Cfg *Func) const {
   Ostream &Str = Func->getContext()->getStrDump();
-  Type Ty = hasRetValue() ? getSrc(0)->getType() : IceType_void;
+  Type Ty = hasRetValue() ? getRetValue()->getType() : IceType_void;
   Str << "ret " << Ty;
   if (hasRetValue()) {
     Str << " ";
@@ -433,4 +499,58 @@
   Str << "unreachable";
 }
 
+void InstFakeDef::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  Str << "\t# ";
+  getDest()->emit(Func);
+  Str << " = def.pseudo ";
+  emitSources(Func);
+  Str << "\n";
+}
+
+void InstFakeDef::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  dumpDest(Func);
+  Str << " = def.pseudo ";
+  dumpSources(Func);
+}
+
+void InstFakeUse::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  Str << "\t# ";
+  Str << "use.pseudo ";
+  emitSources(Func);
+  Str << "\n";
+}
+
+void InstFakeUse::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "use.pseudo ";
+  dumpSources(Func);
+}
+
+void InstFakeKill::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  Str << "\t# ";
+  if (Linked->isDeleted())
+    Str << "// ";
+  Str << "kill.pseudo ";
+  emitSources(Func);
+  Str << "\n";
+}
+
+void InstFakeKill::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  if (Linked->isDeleted())
+    Str << "// ";
+  Str << "kill.pseudo ";
+  dumpSources(Func);
+}
+
+void InstTarget::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "[TARGET] ";
+  Inst::dump(Func);
+}
+
 } // end of namespace Ice
diff --git a/src/IceInst.def b/src/IceInst.def
index 60c613d..a9cadb2 100644
--- a/src/IceInst.def
+++ b/src/IceInst.def
@@ -85,5 +85,4 @@
   X(Sle,         "sle")
 //#define X(tag, str)
 
-
 #endif // SUBZERO_SRC_ICEINST_DEF
diff --git a/src/IceInst.h b/src/IceInst.h
index 57f8b9e..3067c26 100644
--- a/src/IceInst.h
+++ b/src/IceInst.h
@@ -47,7 +47,12 @@
     Ret,
     Select,
     Store,
-    Switch
+    Switch,
+    FakeDef,  // not part of LLVM/PNaCl bitcode
+    FakeUse,  // not part of LLVM/PNaCl bitcode
+    FakeKill, // not part of LLVM/PNaCl bitcode
+    Target    // target-specific low-level ICE
+              // Anything >= Target is an InstTarget subclass.
   };
   InstKind getKind() const { return Kind; }
 
@@ -83,10 +88,13 @@
   // basic blocks, i.e. used in a different block from their definition.
   void updateVars(CfgNode *Node);
 
+  virtual void emit(const Cfg *Func) const;
   virtual void dump(const Cfg *Func) const;
   void dumpDecorated(const Cfg *Func) const;
+  void emitSources(const Cfg *Func) const;
   void dumpSources(const Cfg *Func) const;
   void dumpDest(const Cfg *Func) const;
+  virtual bool isRedundantAssign() const { return false; }
 
   virtual ~Inst() {}
 
@@ -154,6 +162,7 @@
     ICEINSTARITHMETIC_TABLE
 #undef X
   };
+
   static InstArithmetic *create(Cfg *Func, OpKind Op, Variable *Dest,
                                 Operand *Source1, Operand *Source2) {
     return new (Func->allocateInst<InstArithmetic>())
@@ -279,6 +288,7 @@
     ICEINSTCAST_TABLE
 #undef X
   };
+
   static InstCast *create(Cfg *Func, OpKind CastKind, Variable *Dest,
                           Operand *Source) {
     return new (Func->allocateInst<InstCast>())
@@ -305,6 +315,7 @@
     ICEINSTFCMP_TABLE
 #undef X
   };
+
   static InstFcmp *create(Cfg *Func, FCond Condition, Variable *Dest,
                           Operand *Source1, Operand *Source2) {
     return new (Func->allocateInst<InstFcmp>())
@@ -332,6 +343,7 @@
     ICEINSTICMP_TABLE
 #undef X
   };
+
   static InstIcmp *create(Cfg *Func, ICond Condition, Variable *Dest,
                           Operand *Source1, Operand *Source2) {
     return new (Func->allocateInst<InstIcmp>())
@@ -376,6 +388,8 @@
     return new (Func->allocateInst<InstPhi>()) InstPhi(Func, MaxSrcs, Dest);
   }
   void addArgument(Operand *Source, CfgNode *Label);
+  Operand *getOperandForTarget(CfgNode *Target) const;
+  Inst *lower(Cfg *Func, CfgNode *Node);
   virtual void dump(const Cfg *Func) const;
   static bool classof(const Inst *Inst) { return Inst->getKind() == Phi; }
 
@@ -522,6 +536,104 @@
   virtual ~InstUnreachable() {}
 };
 
+// FakeDef instruction.  This creates a fake definition of a variable,
+// which is how we represent the case when an instruction produces
+// multiple results.  This doesn't happen with high-level ICE
+// instructions, but might with lowered instructions.  For example,
+// this would be a way to represent condition flags being modified by
+// an instruction.
+//
+// It's generally useful to set the optional source operand to be the
+// dest variable of the instruction that actually produces the FakeDef
+// dest.  Otherwise, the original instruction could be dead-code
+// eliminated if its dest operand is unused, and therefore the FakeDef
+// dest wouldn't be properly initialized.
+class InstFakeDef : public Inst {
+public:
+  static InstFakeDef *create(Cfg *Func, Variable *Dest, Variable *Src = NULL) {
+    return new (Func->allocateInst<InstFakeDef>()) InstFakeDef(Func, Dest, Src);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return Inst->getKind() == FakeDef; }
+
+private:
+  InstFakeDef(Cfg *Func, Variable *Dest, Variable *Src);
+  InstFakeDef(const InstFakeDef &) LLVM_DELETED_FUNCTION;
+  InstFakeDef &operator=(const InstFakeDef &) LLVM_DELETED_FUNCTION;
+  virtual ~InstFakeDef() {}
+};
+
+// FakeUse instruction.  This creates a fake use of a variable, to
+// keep the instruction that produces that variable from being
+// dead-code eliminated.  This is useful in a variety of lowering
+// situations.  The FakeUse instruction has no dest, so it can itself
+// never be dead-code eliminated.
+class InstFakeUse : public Inst {
+public:
+  static InstFakeUse *create(Cfg *Func, Variable *Src) {
+    return new (Func->allocateInst<InstFakeUse>()) InstFakeUse(Func, Src);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return Inst->getKind() == FakeUse; }
+
+private:
+  InstFakeUse(Cfg *Func, Variable *Src);
+  InstFakeUse(const InstFakeUse &) LLVM_DELETED_FUNCTION;
+  InstFakeUse &operator=(const InstFakeUse &) LLVM_DELETED_FUNCTION;
+  virtual ~InstFakeUse() {}
+};
+
+// FakeKill instruction.  This "kills" a set of variables by adding a
+// trivial live range at this instruction to each variable.  The
+// primary use is to indicate that scratch registers are killed after
+// a call, so that the register allocator won't assign a scratch
+// register to a variable whose live range spans a call.
+//
+// The FakeKill instruction also holds a pointer to the instruction
+// that kills the set of variables, so that if that linked instruction
+// gets dead-code eliminated, the FakeKill instruction will as well.
+class InstFakeKill : public Inst {
+public:
+  static InstFakeKill *create(Cfg *Func, const VarList &KilledRegs,
+                              const Inst *Linked) {
+    return new (Func->allocateInst<InstFakeKill>())
+        InstFakeKill(Func, KilledRegs, Linked);
+  }
+  const Inst *getLinked() const { return Linked; }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return Inst->getKind() == FakeKill; }
+
+private:
+  InstFakeKill(Cfg *Func, const VarList &KilledRegs, const Inst *Linked);
+  InstFakeKill(const InstFakeKill &) LLVM_DELETED_FUNCTION;
+  InstFakeKill &operator=(const InstFakeKill &) LLVM_DELETED_FUNCTION;
+  virtual ~InstFakeKill() {}
+
+  // This instruction is ignored if Linked->isDeleted() is true.
+  const Inst *Linked;
+};
+
+// The Target instruction is the base class for all target-specific
+// instructions.
+class InstTarget : public Inst {
+public:
+  virtual void emit(const Cfg *Func) const = 0;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return Inst->getKind() >= Target; }
+
+protected:
+  InstTarget(Cfg *Func, InstKind Kind, SizeT MaxSrcs, Variable *Dest)
+      : Inst(Func, Kind, MaxSrcs, Dest) {
+    assert(Kind >= Target);
+  }
+  InstTarget(const InstTarget &) LLVM_DELETED_FUNCTION;
+  InstTarget &operator=(const InstTarget &) LLVM_DELETED_FUNCTION;
+  virtual ~InstTarget() {}
+};
+
 } // end of namespace Ice
 
 #endif // SUBZERO_SRC_ICEINST_H
diff --git a/src/IceInstX8632.cpp b/src/IceInstX8632.cpp
new file mode 100644
index 0000000..5d3f01b
--- /dev/null
+++ b/src/IceInstX8632.cpp
@@ -0,0 +1,868 @@
+//===- subzero/src/IceInstX8632.cpp - X86-32 instruction implementation ---===//
+//
+//                        The Subzero Code Generator
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the InstX8632 and OperandX8632 classes,
+// primarily the constructors and the dump()/emit() methods.
+//
+//===----------------------------------------------------------------------===//
+
+#include "IceCfg.h"
+#include "IceCfgNode.h"
+#include "IceInst.h"
+#include "IceInstX8632.h"
+#include "IceTargetLoweringX8632.h"
+#include "IceOperand.h"
+
+namespace Ice {
+
+namespace {
+
+const struct InstX8632BrAttributes_ {
+  const char *DisplayString;
+  const char *EmitString;
+} InstX8632BrAttributes[] = {
+#define X(tag, dump, emit)                                                     \
+  { dump, emit }                                                               \
+  ,
+    ICEINSTX8632BR_TABLE
+#undef X
+  };
+const size_t InstX8632BrAttributesSize =
+    llvm::array_lengthof(InstX8632BrAttributes);
+
+const struct TypeX8632Attributes_ {
+  const char *CvtString;   // i (integer), s (single FP), d (double FP)
+  const char *SdSsString;  // ss, sd, or <blank>
+  const char *WidthString; // {byte,word,dword,qword} ptr
+} TypeX8632Attributes[] = {
+#define X(tag, cvt, sdss, width)                                               \
+  { cvt, "" sdss, width }                                                      \
+  ,
+    ICETYPEX8632_TABLE
+#undef X
+  };
+const size_t TypeX8632AttributesSize =
+    llvm::array_lengthof(TypeX8632Attributes);
+
+} // end of anonymous namespace
+
+const char *InstX8632::getWidthString(Type Ty) {
+  return TypeX8632Attributes[Ty].WidthString;
+}
+
+OperandX8632Mem::OperandX8632Mem(Cfg *Func, Type Ty, Variable *Base,
+                                 Constant *Offset, Variable *Index,
+                                 uint32_t Shift)
+    : OperandX8632(kMem, Ty), Base(Base), Offset(Offset), Index(Index),
+      Shift(Shift) {
+  assert(Shift <= 3);
+  Vars = NULL;
+  NumVars = 0;
+  if (Base)
+    ++NumVars;
+  if (Index)
+    ++NumVars;
+  if (NumVars) {
+    Vars = Func->allocateArrayOf<Variable *>(NumVars);
+    SizeT I = 0;
+    if (Base)
+      Vars[I++] = Base;
+    if (Index)
+      Vars[I++] = Index;
+    assert(I == NumVars);
+  }
+}
+
+InstX8632Mul::InstX8632Mul(Cfg *Func, Variable *Dest, Variable *Source1,
+                           Operand *Source2)
+    : InstX8632(Func, InstX8632::Mul, 2, Dest) {
+  addSource(Source1);
+  addSource(Source2);
+}
+
+InstX8632Shld::InstX8632Shld(Cfg *Func, Variable *Dest, Variable *Source1,
+                             Variable *Source2)
+    : InstX8632(Func, InstX8632::Shld, 3, Dest) {
+  addSource(Dest);
+  addSource(Source1);
+  addSource(Source2);
+}
+
+InstX8632Shrd::InstX8632Shrd(Cfg *Func, Variable *Dest, Variable *Source1,
+                             Variable *Source2)
+    : InstX8632(Func, InstX8632::Shrd, 3, Dest) {
+  addSource(Dest);
+  addSource(Source1);
+  addSource(Source2);
+}
+
+InstX8632Label::InstX8632Label(Cfg *Func, TargetX8632 *Target)
+    : InstX8632(Func, InstX8632::Label, 0, NULL),
+      Number(Target->makeNextLabelNumber()) {}
+
+IceString InstX8632Label::getName(const Cfg *Func) const {
+  char buf[30];
+  snprintf(buf, llvm::array_lengthof(buf), "%u", Number);
+  return ".L" + Func->getFunctionName() + "$__" + buf;
+}
+
+InstX8632Br::InstX8632Br(Cfg *Func, CfgNode *TargetTrue, CfgNode *TargetFalse,
+                         InstX8632Label *Label, InstX8632Br::BrCond Condition)
+    : InstX8632(Func, InstX8632::Br, 0, NULL), Condition(Condition),
+      TargetTrue(TargetTrue), TargetFalse(TargetFalse), Label(Label) {}
+
+InstX8632Call::InstX8632Call(Cfg *Func, Variable *Dest, Operand *CallTarget)
+    : InstX8632(Func, InstX8632::Call, 1, Dest) {
+  HasSideEffects = true;
+  addSource(CallTarget);
+}
+
+InstX8632Cdq::InstX8632Cdq(Cfg *Func, Variable *Dest, Operand *Source)
+    : InstX8632(Func, InstX8632::Cdq, 1, Dest) {
+  assert(Dest->getRegNum() == TargetX8632::Reg_edx);
+  assert(llvm::isa<Variable>(Source));
+  assert(llvm::dyn_cast<Variable>(Source)->getRegNum() == TargetX8632::Reg_eax);
+  addSource(Source);
+}
+
+InstX8632Cvt::InstX8632Cvt(Cfg *Func, Variable *Dest, Operand *Source)
+    : InstX8632(Func, InstX8632::Cvt, 1, Dest) {
+  addSource(Source);
+}
+
+InstX8632Icmp::InstX8632Icmp(Cfg *Func, Operand *Src0, Operand *Src1)
+    : InstX8632(Func, InstX8632::Icmp, 2, NULL) {
+  addSource(Src0);
+  addSource(Src1);
+}
+
+InstX8632Ucomiss::InstX8632Ucomiss(Cfg *Func, Operand *Src0, Operand *Src1)
+    : InstX8632(Func, InstX8632::Ucomiss, 2, NULL) {
+  addSource(Src0);
+  addSource(Src1);
+}
+
+InstX8632Test::InstX8632Test(Cfg *Func, Operand *Src1, Operand *Src2)
+    : InstX8632(Func, InstX8632::Test, 2, NULL) {
+  addSource(Src1);
+  addSource(Src2);
+}
+
+InstX8632Store::InstX8632Store(Cfg *Func, Operand *Value, OperandX8632 *Mem)
+    : InstX8632(Func, InstX8632::Store, 2, NULL) {
+  addSource(Value);
+  addSource(Mem);
+}
+
+InstX8632Mov::InstX8632Mov(Cfg *Func, Variable *Dest, Operand *Source)
+    : InstX8632(Func, InstX8632::Mov, 1, Dest) {
+  addSource(Source);
+}
+
+InstX8632Movsx::InstX8632Movsx(Cfg *Func, Variable *Dest, Operand *Source)
+    : InstX8632(Func, InstX8632::Movsx, 1, Dest) {
+  addSource(Source);
+}
+
+InstX8632Movzx::InstX8632Movzx(Cfg *Func, Variable *Dest, Operand *Source)
+    : InstX8632(Func, InstX8632::Movzx, 1, Dest) {
+  addSource(Source);
+}
+
+InstX8632Fld::InstX8632Fld(Cfg *Func, Operand *Src)
+    : InstX8632(Func, InstX8632::Fld, 1, NULL) {
+  addSource(Src);
+}
+
+InstX8632Fstp::InstX8632Fstp(Cfg *Func, Variable *Dest)
+    : InstX8632(Func, InstX8632::Fstp, 0, Dest) {}
+
+InstX8632Pop::InstX8632Pop(Cfg *Func, Variable *Dest)
+    : InstX8632(Func, InstX8632::Pop, 0, Dest) {}
+
+InstX8632Push::InstX8632Push(Cfg *Func, Operand *Source,
+                             bool SuppressStackAdjustment)
+    : InstX8632(Func, InstX8632::Push, 1, NULL),
+      SuppressStackAdjustment(SuppressStackAdjustment) {
+  addSource(Source);
+}
+
+bool InstX8632Mov::isRedundantAssign() const {
+  Variable *Src = llvm::dyn_cast<Variable>(getSrc(0));
+  if (Src == NULL)
+    return false;
+  if (getDest()->hasReg() && getDest()->getRegNum() == Src->getRegNum()) {
+    // TODO: On x86-64, instructions like "mov eax, eax" are used to
+    // clear the upper 32 bits of rax.  We need to recognize and
+    // preserve these.
+    return true;
+  }
+  if (!getDest()->hasReg() && !Src->hasReg() &&
+      Dest->getStackOffset() == Src->getStackOffset())
+    return true;
+  return false;
+}
+
+InstX8632Ret::InstX8632Ret(Cfg *Func, Variable *Source)
+    : InstX8632(Func, InstX8632::Ret, Source ? 1 : 0, NULL) {
+  if (Source)
+    addSource(Source);
+}
+
+// ======================== Dump routines ======================== //
+
+void InstX8632::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "[X8632] ";
+  Inst::dump(Func);
+}
+
+void InstX8632Label::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  Str << getName(Func) << ":\n";
+}
+
+void InstX8632Label::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << getName(Func) << ":";
+}
+
+void InstX8632Br::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  Str << "\t";
+
+  if (Condition == Br_None) {
+    Str << "jmp";
+  } else {
+    Str << InstX8632BrAttributes[Condition].EmitString;
+  }
+
+  if (Label) {
+    Str << "\t" << Label->getName(Func) << "\n";
+  } else {
+    if (Condition == Br_None) {
+      Str << "\t" << getTargetFalse()->getAsmName() << "\n";
+    } else {
+      Str << "\t" << getTargetTrue()->getAsmName() << "\n";
+      if (getTargetFalse()) {
+        Str << "\tjmp\t" << getTargetFalse()->getAsmName() << "\n";
+      }
+    }
+  }
+}
+
+void InstX8632Br::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "br ";
+
+  if (Condition == Br_None) {
+    Str << "label %"
+        << (Label ? Label->getName(Func) : getTargetFalse()->getName());
+    return;
+  }
+
+  Str << InstX8632BrAttributes[Condition].DisplayString;
+  if (Label) {
+    Str << ", label %" << Label->getName(Func);
+  } else {
+    Str << ", label %" << getTargetTrue()->getName();
+    if (getTargetFalse()) {
+      Str << ", label %" << getTargetFalse()->getName();
+    }
+  }
+}
+
+void InstX8632Call::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 1);
+  Str << "\tcall\t";
+  getCallTarget()->emit(Func);
+  Str << "\n";
+  Func->getTarget()->resetStackAdjustment();
+}
+
+void InstX8632Call::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  if (getDest()) {
+    dumpDest(Func);
+    Str << " = ";
+  }
+  Str << "call ";
+  getCallTarget()->dump(Func);
+}
+
+// The ShiftHack parameter is used to emit "cl" instead of "ecx" for
+// shift instructions, in order to be syntactically valid.  The
+// Opcode parameter needs to be char* and not IceString because of
+// template issues.
+void emitTwoAddress(const char *Opcode, const Inst *Inst, const Cfg *Func,
+                    bool ShiftHack) {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(Inst->getSrcSize() == 2);
+  assert(Inst->getDest() == Inst->getSrc(0));
+  Str << "\t" << Opcode << "\t";
+  Inst->getDest()->emit(Func);
+  Str << ", ";
+  bool EmittedSrc1 = false;
+  if (ShiftHack) {
+    Variable *ShiftReg = llvm::dyn_cast<Variable>(Inst->getSrc(1));
+    if (ShiftReg && ShiftReg->getRegNum() == TargetX8632::Reg_ecx) {
+      Str << "cl";
+      EmittedSrc1 = true;
+    }
+  }
+  if (!EmittedSrc1)
+    Inst->getSrc(1)->emit(Func);
+  Str << "\n";
+}
+
+template <> const char *InstX8632Add::Opcode = "add";
+template <> const char *InstX8632Adc::Opcode = "adc";
+template <> const char *InstX8632Addss::Opcode = "addss";
+template <> const char *InstX8632Sub::Opcode = "sub";
+template <> const char *InstX8632Subss::Opcode = "subss";
+template <> const char *InstX8632Sbb::Opcode = "sbb";
+template <> const char *InstX8632And::Opcode = "and";
+template <> const char *InstX8632Or::Opcode = "or";
+template <> const char *InstX8632Xor::Opcode = "xor";
+template <> const char *InstX8632Imul::Opcode = "imul";
+template <> const char *InstX8632Mulss::Opcode = "mulss";
+template <> const char *InstX8632Div::Opcode = "div";
+template <> const char *InstX8632Idiv::Opcode = "idiv";
+template <> const char *InstX8632Divss::Opcode = "divss";
+template <> const char *InstX8632Shl::Opcode = "shl";
+template <> const char *InstX8632Shr::Opcode = "shr";
+template <> const char *InstX8632Sar::Opcode = "sar";
+
+template <> void InstX8632Addss::emit(const Cfg *Func) const {
+  char buf[30];
+  snprintf(buf, llvm::array_lengthof(buf), "add%s",
+           TypeX8632Attributes[getDest()->getType()].SdSsString);
+  emitTwoAddress(buf, this, Func);
+}
+
+template <> void InstX8632Subss::emit(const Cfg *Func) const {
+  char buf[30];
+  snprintf(buf, llvm::array_lengthof(buf), "sub%s",
+           TypeX8632Attributes[getDest()->getType()].SdSsString);
+  emitTwoAddress(buf, this, Func);
+}
+
+template <> void InstX8632Mulss::emit(const Cfg *Func) const {
+  char buf[30];
+  snprintf(buf, llvm::array_lengthof(buf), "mul%s",
+           TypeX8632Attributes[getDest()->getType()].SdSsString);
+  emitTwoAddress(buf, this, Func);
+}
+
+template <> void InstX8632Divss::emit(const Cfg *Func) const {
+  char buf[30];
+  snprintf(buf, llvm::array_lengthof(buf), "div%s",
+           TypeX8632Attributes[getDest()->getType()].SdSsString);
+  emitTwoAddress(buf, this, Func);
+}
+
+template <> void InstX8632Imul::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 2);
+  if (getDest()->getType() == IceType_i8) {
+    // The 8-bit version of imul only allows the form "imul r/m8".
+    Variable *Src0 = llvm::dyn_cast<Variable>(getSrc(0));
+    assert(Src0 && Src0->getRegNum() == TargetX8632::Reg_eax);
+    Str << "\timul\t";
+    getSrc(1)->emit(Func);
+    Str << "\n";
+  } else if (llvm::isa<Constant>(getSrc(1))) {
+    Str << "\timul\t";
+    getDest()->emit(Func);
+    Str << ", ";
+    getSrc(0)->emit(Func);
+    Str << ", ";
+    getSrc(1)->emit(Func);
+    Str << "\n";
+  } else {
+    emitTwoAddress("imul", this, Func);
+  }
+}
+
+void InstX8632Mul::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 2);
+  assert(llvm::isa<Variable>(getSrc(0)));
+  assert(llvm::dyn_cast<Variable>(getSrc(0))->getRegNum() ==
+         TargetX8632::Reg_eax);
+  assert(getDest()->getRegNum() == TargetX8632::Reg_eax); // TODO: allow edx?
+  Str << "\tmul\t";
+  getSrc(1)->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Mul::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  dumpDest(Func);
+  Str << " = mul." << getDest()->getType() << " ";
+  dumpSources(Func);
+}
+
+void InstX8632Shld::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 3);
+  assert(getDest() == getSrc(0));
+  Str << "\tshld\t";
+  getDest()->emit(Func);
+  Str << ", ";
+  getSrc(1)->emit(Func);
+  Str << ", ";
+  if (Variable *ShiftReg = llvm::dyn_cast<Variable>(getSrc(2))) {
+    assert(ShiftReg->getRegNum() == TargetX8632::Reg_ecx);
+    Str << "cl";
+  } else {
+    getSrc(2)->emit(Func);
+  }
+  Str << "\n";
+}
+
+void InstX8632Shld::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  dumpDest(Func);
+  Str << " = shld." << getDest()->getType() << " ";
+  dumpSources(Func);
+}
+
+void InstX8632Shrd::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 3);
+  assert(getDest() == getSrc(0));
+  Str << "\tshrd\t";
+  getDest()->emit(Func);
+  Str << ", ";
+  getSrc(1)->emit(Func);
+  Str << ", ";
+  if (Variable *ShiftReg = llvm::dyn_cast<Variable>(getSrc(2))) {
+    assert(ShiftReg->getRegNum() == TargetX8632::Reg_ecx);
+    Str << "cl";
+  } else {
+    getSrc(2)->emit(Func);
+  }
+  Str << "\n";
+}
+
+void InstX8632Shrd::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  dumpDest(Func);
+  Str << " = shrd." << getDest()->getType() << " ";
+  dumpSources(Func);
+}
+
+void InstX8632Cdq::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 1);
+  Str << "\tcdq\n";
+}
+
+void InstX8632Cdq::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  dumpDest(Func);
+  Str << " = cdq." << getSrc(0)->getType() << " ";
+  dumpSources(Func);
+}
+
+void InstX8632Cvt::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 1);
+  Str << "\tcvts" << TypeX8632Attributes[getSrc(0)->getType()].CvtString << "2s"
+      << TypeX8632Attributes[getDest()->getType()].CvtString << "\t";
+  getDest()->emit(Func);
+  Str << ", ";
+  getSrc(0)->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Cvt::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  dumpDest(Func);
+  Str << " = cvts" << TypeX8632Attributes[getSrc(0)->getType()].CvtString
+      << "2s" << TypeX8632Attributes[getDest()->getType()].CvtString << " ";
+  dumpSources(Func);
+}
+
+void InstX8632Icmp::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 2);
+  Str << "\tcmp\t";
+  getSrc(0)->emit(Func);
+  Str << ", ";
+  getSrc(1)->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Icmp::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "cmp." << getSrc(0)->getType() << " ";
+  dumpSources(Func);
+}
+
+void InstX8632Ucomiss::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 2);
+  Str << "\tucomi" << TypeX8632Attributes[getSrc(0)->getType()].SdSsString
+      << "\t";
+  getSrc(0)->emit(Func);
+  Str << ", ";
+  getSrc(1)->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Ucomiss::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "ucomiss." << getSrc(0)->getType() << " ";
+  dumpSources(Func);
+}
+
+void InstX8632Test::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 2);
+  Str << "\ttest\t";
+  getSrc(0)->emit(Func);
+  Str << ", ";
+  getSrc(1)->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Test::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "test." << getSrc(0)->getType() << " ";
+  dumpSources(Func);
+}
+
+void InstX8632Store::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 2);
+  Str << "\tmov\t";
+  getSrc(1)->emit(Func);
+  Str << ", ";
+  getSrc(0)->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Store::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "mov." << getSrc(0)->getType() << " ";
+  getSrc(1)->dump(Func);
+  Str << ", ";
+  getSrc(0)->dump(Func);
+}
+
+void InstX8632Mov::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 1);
+  Str << "\tmov" << TypeX8632Attributes[getDest()->getType()].SdSsString
+      << "\t";
+  // For an integer truncation operation, src is wider than dest.
+  // Ideally, we use a mov instruction whose data width matches the
+  // narrower dest.  This is a problem if e.g. src is a register like
+  // esi or si where there is no 8-bit version of the register.  To be
+  // safe, we instead widen the dest to match src.  This works even
+  // for stack-allocated dest variables because typeWidthOnStack()
+  // pads to a 4-byte boundary even if only a lower portion is used.
+  assert(Func->getTarget()->typeWidthInBytesOnStack(getDest()->getType()) ==
+         Func->getTarget()->typeWidthInBytesOnStack(getSrc(0)->getType()));
+  getDest()->asType(getSrc(0)->getType()).emit(Func);
+  Str << ", ";
+  getSrc(0)->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Mov::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "mov." << getDest()->getType() << " ";
+  dumpDest(Func);
+  Str << ", ";
+  dumpSources(Func);
+}
+
+void InstX8632Movsx::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 1);
+  Str << "\tmovsx\t";
+  getDest()->emit(Func);
+  Str << ", ";
+  getSrc(0)->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Movsx::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "movsx." << getDest()->getType() << "." << getSrc(0)->getType();
+  Str << " ";
+  dumpDest(Func);
+  Str << ", ";
+  dumpSources(Func);
+}
+
+void InstX8632Movzx::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 1);
+  Str << "\tmovzx\t";
+  getDest()->emit(Func);
+  Str << ", ";
+  getSrc(0)->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Movzx::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "movzx." << getDest()->getType() << "." << getSrc(0)->getType();
+  Str << " ";
+  dumpDest(Func);
+  Str << ", ";
+  dumpSources(Func);
+}
+
+void InstX8632Fld::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 1);
+  Type Ty = getSrc(0)->getType();
+  Variable *Var = llvm::dyn_cast<Variable>(getSrc(0));
+  if (Var && Var->hasReg()) {
+    // This is a physical xmm register, so we need to spill it to a
+    // temporary stack slot.
+    SizeT Width = typeWidthInBytes(Ty);
+    Str << "\tsub\tesp, " << Width << "\n";
+    Str << "\tmov" << TypeX8632Attributes[Ty].SdSsString << "\t"
+        << TypeX8632Attributes[Ty].WidthString << " [esp], ";
+    Var->emit(Func);
+    Str << "\n";
+    Str << "\tfld\t" << TypeX8632Attributes[Ty].WidthString << " [esp]\n";
+    Str << "\tadd\tesp, " << Width << "\n";
+    return;
+  }
+  Str << "\tfld\t";
+  getSrc(0)->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Fld::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "fld." << getSrc(0)->getType() << " ";
+  dumpSources(Func);
+}
+
+void InstX8632Fstp::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 0);
+  if (getDest() == NULL) {
+    Str << "\tfstp\tst(0)\n";
+    return;
+  }
+  if (!getDest()->hasReg()) {
+    Str << "\tfstp\t";
+    getDest()->emit(Func);
+    Str << "\n";
+    return;
+  }
+  // Dest is a physical (xmm) register, so st(0) needs to go through
+  // memory.  Hack this by creating a temporary stack slot, spilling
+  // st(0) there, loading it into the xmm register, and deallocating
+  // the stack slot.
+  Type Ty = getDest()->getType();
+  size_t Width = typeWidthInBytes(Ty);
+  Str << "\tsub\tesp, " << Width << "\n";
+  Str << "\tfstp\t" << TypeX8632Attributes[Ty].WidthString << " [esp]\n";
+  Str << "\tmov" << TypeX8632Attributes[Ty].SdSsString << "\t";
+  getDest()->emit(Func);
+  Str << ", " << TypeX8632Attributes[Ty].WidthString << " [esp]\n";
+  Str << "\tadd\tesp, " << Width << "\n";
+}
+
+void InstX8632Fstp::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  dumpDest(Func);
+  Str << " = fstp." << getDest()->getType() << ", st(0)";
+  Str << "\n";
+}
+
+void InstX8632Pop::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 0);
+  Str << "\tpop\t";
+  getDest()->emit(Func);
+  Str << "\n";
+}
+
+void InstX8632Pop::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  dumpDest(Func);
+  Str << " = pop." << getDest()->getType() << " ";
+}
+
+void InstX8632Push::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(getSrcSize() == 1);
+  Type Ty = getSrc(0)->getType();
+  Variable *Var = llvm::dyn_cast<Variable>(getSrc(0));
+  if ((Ty == IceType_f32 || Ty == IceType_f64) && Var && Var->hasReg()) {
+    // The xmm registers can't be directly pushed, so we fake it by
+    // decrementing esp and then storing to [esp].
+    Str << "\tsub\tesp, " << typeWidthInBytes(Ty) << "\n";
+    if (!SuppressStackAdjustment)
+      Func->getTarget()->updateStackAdjustment(typeWidthInBytes(Ty));
+    Str << "\tmov" << TypeX8632Attributes[Ty].SdSsString << "\t"
+        << TypeX8632Attributes[Ty].WidthString << " [esp], ";
+    getSrc(0)->emit(Func);
+    Str << "\n";
+  } else if (Ty == IceType_f64 && (!Var || !Var->hasReg())) {
+    // A double on the stack has to be pushed as two halves.  Push the
+    // upper half followed by the lower half for little-endian.  TODO:
+    // implement.
+    llvm_unreachable("Missing support for pushing doubles from memory");
+  } else {
+    Str << "\tpush\t";
+    getSrc(0)->emit(Func);
+    Str << "\n";
+    if (!SuppressStackAdjustment)
+      Func->getTarget()->updateStackAdjustment(4);
+  }
+}
+
+void InstX8632Push::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "push." << getSrc(0)->getType() << " ";
+  dumpSources(Func);
+}
+
+void InstX8632Ret::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  Str << "\tret\n";
+}
+
+void InstX8632Ret::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Type Ty = (getSrcSize() == 0 ? IceType_void : getSrc(0)->getType());
+  Str << "ret." << Ty << " ";
+  dumpSources(Func);
+}
+
+void OperandX8632::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  Str << "<OperandX8632>";
+}
+
+void OperandX8632Mem::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  Str << TypeX8632Attributes[getType()].WidthString << " ";
+  // TODO: The following is an almost verbatim paste of dump().
+  bool Dumped = false;
+  Str << "[";
+  if (Base) {
+    Base->emit(Func);
+    Dumped = true;
+  }
+  if (Index) {
+    assert(Base);
+    Str << "+";
+    if (Shift > 0)
+      Str << (1u << Shift) << "*";
+    Index->emit(Func);
+    Dumped = true;
+  }
+  // Pretty-print the Offset.
+  bool OffsetIsZero = false;
+  bool OffsetIsNegative = false;
+  if (Offset == NULL) {
+    OffsetIsZero = true;
+  } else if (ConstantInteger *CI = llvm::dyn_cast<ConstantInteger>(Offset)) {
+    OffsetIsZero = (CI->getValue() == 0);
+    OffsetIsNegative = (static_cast<int64_t>(CI->getValue()) < 0);
+  }
+  if (!OffsetIsZero) { // Suppress if Offset is known to be 0
+    if (Dumped) {
+      if (!OffsetIsNegative) // Suppress if Offset is known to be negative
+        Str << "+";
+    }
+    Offset->emit(Func);
+  }
+  Str << "]";
+}
+
+void OperandX8632Mem::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  bool Dumped = false;
+  Str << "[";
+  if (Base) {
+    Base->dump(Func);
+    Dumped = true;
+  }
+  if (Index) {
+    assert(Base);
+    Str << "+";
+    if (Shift > 0)
+      Str << (1u << Shift) << "*";
+    Index->dump(Func);
+    Dumped = true;
+  }
+  // Pretty-print the Offset.
+  bool OffsetIsZero = false;
+  bool OffsetIsNegative = false;
+  if (Offset == NULL) {
+    OffsetIsZero = true;
+  } else if (ConstantInteger *CI = llvm::dyn_cast<ConstantInteger>(Offset)) {
+    OffsetIsZero = (CI->getValue() == 0);
+    OffsetIsNegative = (static_cast<int64_t>(CI->getValue()) < 0);
+  }
+  if (!OffsetIsZero) { // Suppress if Offset is known to be 0
+    if (Dumped) {
+      if (!OffsetIsNegative) // Suppress if Offset is known to be negative
+        Str << "+";
+    }
+    Offset->dump(Func);
+  }
+  Str << "]";
+}
+
+void VariableSplit::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  assert(Var->getLocalUseNode() == NULL ||
+         Var->getLocalUseNode() == Func->getCurrentNode());
+  assert(!Var->hasReg());
+  // The following is copied/adapted from TargetX8632::emitVariable().
+  const TargetLowering *Target = Func->getTarget();
+  const Type Ty = IceType_i32;
+  Str << TypeX8632Attributes[Ty].WidthString << " ["
+      << Target->getRegName(Target->getFrameOrStackReg(), Ty);
+  int32_t Offset = Var->getStackOffset() + Target->getStackAdjustment();
+  if (Part == High)
+    Offset += 4;
+  if (Offset) {
+    if (Offset > 0)
+      Str << "+";
+    Str << Offset;
+  }
+  Str << "]";
+}
+
+void VariableSplit::dump(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrDump();
+  switch (Part) {
+  case Low:
+    Str << "low";
+    break;
+  case High:
+    Str << "high";
+    break;
+  default:
+    Str << "???";
+    break;
+  }
+  Str << "(";
+  Var->dump(Func);
+  Str << ")";
+}
+
+} // end of namespace Ice
diff --git a/src/IceInstX8632.def b/src/IceInstX8632.def
new file mode 100644
index 0000000..665dd8d
--- /dev/null
+++ b/src/IceInstX8632.def
@@ -0,0 +1,71 @@
+//===- subzero/src/IceInstX8632.def - X-macros for x86-32 insts -*- C++ -*-===//
+//
+//                        The Subzero Code Generator
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines properties of lowered x86-32 instructions in the
+// form of x-macros.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SUBZERO_SRC_ICEINSTX8632_DEF
+#define SUBZERO_SRC_ICEINSTX8632_DEF
+
+#define REGX8632_TABLE                                                  \
+  /* val, init, name, name16, name8, scratch, preserved, stackptr,      \
+     frameptr, isI8, isInt, isFP */                                     \
+  X(Reg_eax, = 0,             "eax",  "ax", "al", 1, 0, 0, 0, 1, 1, 0)  \
+  X(Reg_ecx, = Reg_eax + 1,   "ecx",  "cx", "cl", 1, 0, 0, 0, 1, 1, 0)  \
+  X(Reg_edx, = Reg_eax + 2,   "edx",  "dx", "dl", 1, 0, 0, 0, 1, 1, 0)  \
+  X(Reg_ebx, = Reg_eax + 3,   "ebx",  "bx", "bl", 0, 1, 0, 0, 1, 1, 0)  \
+  X(Reg_esp, = Reg_eax + 4,   "esp",  "sp",     , 0, 0, 1, 0, 0, 1, 0)  \
+  X(Reg_ebp, = Reg_eax + 5,   "ebp",  "bp",     , 0, 1, 0, 1, 0, 1, 0)  \
+  X(Reg_esi, = Reg_eax + 6,   "esi",  "si",     , 0, 1, 0, 0, 0, 1, 0)  \
+  X(Reg_edi, = Reg_eax + 7,   "edi",  "di",     , 0, 1, 0, 0, 0, 1, 0)  \
+  X(Reg_ah,   /*none*/,       "???",      , "ah", 0, 0, 0, 0, 1, 0, 0)  \
+  X(Reg_xmm0, /*none*/,       "xmm0",     ,     , 1, 0, 0, 0, 0, 0, 1)  \
+  X(Reg_xmm1, = Reg_xmm0 + 1, "xmm1",     ,     , 1, 0, 0, 0, 0, 0, 1)  \
+  X(Reg_xmm2, = Reg_xmm0 + 2, "xmm2",     ,     , 1, 0, 0, 0, 0, 0, 1)  \
+  X(Reg_xmm3, = Reg_xmm0 + 3, "xmm3",     ,     , 1, 0, 0, 0, 0, 0, 1)  \
+  X(Reg_xmm4, = Reg_xmm0 + 4, "xmm4",     ,     , 1, 0, 0, 0, 0, 0, 1)  \
+  X(Reg_xmm5, = Reg_xmm0 + 5, "xmm5",     ,     , 1, 0, 0, 0, 0, 0, 1)  \
+  X(Reg_xmm6, = Reg_xmm0 + 6, "xmm6",     ,     , 1, 0, 0, 0, 0, 0, 1)  \
+  X(Reg_xmm7, = Reg_xmm0 + 7, "xmm7",     ,     , 1, 0, 0, 0, 0, 0, 1)  \
+//#define X(val, init, name, name16, name8, scratch, preserved, stackptr,
+//          frameptr, isI8, isInt, isFP)
+
+
+#define ICEINSTX8632BR_TABLE   \
+  /* enum value, dump, emit */ \
+  X(Br_a,        "a",  "ja")   \
+  X(Br_ae,       "ae", "jae")  \
+  X(Br_b,        "b",  "jb")   \
+  X(Br_be,       "be", "jbe")  \
+  X(Br_e,        "e",  "je")   \
+  X(Br_g,        "g",  "jg")   \
+  X(Br_ge,       "ge", "jge")  \
+  X(Br_l,        "l",  "jl")   \
+  X(Br_le,       "le", "jle")  \
+  X(Br_ne,       "ne", "jne")  \
+  X(Br_np,       "np", "jnp")  \
+  X(Br_p,        "p",  "jp")   \
+//#define X(tag, dump, emit)
+
+#define ICETYPEX8632_TABLE                \
+  /* tag,         cvt, sdss, width */     \
+  X(IceType_void, "?",     , "???")       \
+  X(IceType_i1,   "i",     , "byte ptr")  \
+  X(IceType_i8,   "i",     , "byte ptr")  \
+  X(IceType_i16,  "i",     , "word ptr")  \
+  X(IceType_i32,  "i",     , "dword ptr") \
+  X(IceType_i64,  "i",     , "qword ptr") \
+  X(IceType_f32,  "s", "ss", "dword ptr") \
+  X(IceType_f64,  "d", "sd", "qword ptr") \
+  X(IceType_NUM,  "?",     , "???")       \
+//#define X(tag, cvt, sdss, width)
+
+#endif // SUBZERO_SRC_ICEINSTX8632_DEF
diff --git a/src/IceInstX8632.h b/src/IceInstX8632.h
new file mode 100644
index 0000000..8a6f14a
--- /dev/null
+++ b/src/IceInstX8632.h
@@ -0,0 +1,720 @@
+//===- subzero/src/IceInstX8632.h - Low-level x86 instructions --*- C++ -*-===//
+//
+//                        The Subzero Code Generator
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file declares the InstX8632 and OperandX8632 classes and
+// their subclasses.  This represents the machine instructions and
+// operands used for x86-32 code selection.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SUBZERO_SRC_ICEINSTX8632_H
+#define SUBZERO_SRC_ICEINSTX8632_H
+
+#include "IceDefs.h"
+#include "IceInst.h"
+#include "IceInstX8632.def"
+#include "IceOperand.h"
+
+namespace Ice {
+
+class TargetX8632;
+
+// OperandX8632 extends the Operand hierarchy.  Its subclasses are
+// OperandX8632Mem and VariableSplit.
+class OperandX8632 : public Operand {
+public:
+  enum OperandKindX8632 {
+    k__Start = Operand::kTarget,
+    kMem,
+    kSplit
+  };
+  virtual void emit(const Cfg *Func) const = 0;
+  void dump(const Cfg *Func) const;
+
+protected:
+  OperandX8632(OperandKindX8632 Kind, Type Ty)
+      : Operand(static_cast<OperandKind>(Kind), Ty) {}
+  virtual ~OperandX8632() {}
+
+private:
+  OperandX8632(const OperandX8632 &) LLVM_DELETED_FUNCTION;
+  OperandX8632 &operator=(const OperandX8632 &) LLVM_DELETED_FUNCTION;
+};
+
+// OperandX8632Mem represents the m32 addressing mode, with optional
+// base and index registers, a constant offset, and a fixed shift
+// value for the index register.
+class OperandX8632Mem : public OperandX8632 {
+public:
+  static OperandX8632Mem *create(Cfg *Func, Type Ty, Variable *Base,
+                                 Constant *Offset, Variable *Index = NULL,
+                                 uint32_t Shift = 0) {
+    return new (Func->allocate<OperandX8632Mem>())
+        OperandX8632Mem(Func, Ty, Base, Offset, Index, Shift);
+  }
+  Variable *getBase() const { return Base; }
+  Constant *getOffset() const { return Offset; }
+  Variable *getIndex() const { return Index; }
+  uint32_t getShift() const { return Shift; }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+
+  static bool classof(const Operand *Operand) {
+    return Operand->getKind() == static_cast<OperandKind>(kMem);
+  }
+
+private:
+  OperandX8632Mem(Cfg *Func, Type Ty, Variable *Base, Constant *Offset,
+                  Variable *Index, uint32_t Shift);
+  OperandX8632Mem(const OperandX8632Mem &) LLVM_DELETED_FUNCTION;
+  OperandX8632Mem &operator=(const OperandX8632Mem &) LLVM_DELETED_FUNCTION;
+  virtual ~OperandX8632Mem() {}
+  Variable *Base;
+  Constant *Offset;
+  Variable *Index;
+  uint32_t Shift;
+};
+
+// VariableSplit is a way to treat an f64 memory location as a pair
+// of i32 locations (Low and High).  This is needed for some cases
+// of the Bitcast instruction.  Since it's not possible for integer
+// registers to access the XMM registers and vice versa, the
+// lowering forces the f64 to be spilled to the stack and then
+// accesses through the VariableSplit.
+class VariableSplit : public OperandX8632 {
+public:
+  enum Portion {
+    Low,
+    High
+  };
+  static VariableSplit *create(Cfg *Func, Variable *Var, Portion Part) {
+    return new (Func->allocate<VariableSplit>()) VariableSplit(Func, Var, Part);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+
+  static bool classof(const Operand *Operand) {
+    return Operand->getKind() == static_cast<OperandKind>(kSplit);
+  }
+
+private:
+  VariableSplit(Cfg *Func, Variable *Var, Portion Part)
+      : OperandX8632(kSplit, IceType_i32), Func(Func), Var(Var), Part(Part) {
+    assert(Var->getType() == IceType_f64);
+    Vars = Func->allocateArrayOf<Variable *>(1);
+    Vars[0] = Var;
+    NumVars = 1;
+  }
+  VariableSplit(const VariableSplit &) LLVM_DELETED_FUNCTION;
+  VariableSplit &operator=(const VariableSplit &) LLVM_DELETED_FUNCTION;
+  virtual ~VariableSplit() { Func->deallocateArrayOf<Variable *>(Vars); }
+  Cfg *Func; // Held only for the destructor.
+  Variable *Var;
+  Portion Part;
+};
+
+class InstX8632 : public InstTarget {
+public:
+  enum InstKindX8632 {
+    k__Start = Inst::Target,
+    Adc,
+    Add,
+    Addss,
+    And,
+    Br,
+    Call,
+    Cdq,
+    Cvt,
+    Div,
+    Divss,
+    Fld,
+    Fstp,
+    Icmp,
+    Idiv,
+    Imul,
+    Label,
+    Load,
+    Mov,
+    Movsx,
+    Movzx,
+    Mul,
+    Mulss,
+    Or,
+    Pop,
+    Push,
+    Ret,
+    Sar,
+    Sbb,
+    Shl,
+    Shld,
+    Shr,
+    Shrd,
+    Store,
+    Sub,
+    Subss,
+    Test,
+    Ucomiss,
+    Xor
+  };
+  static const char *getWidthString(Type Ty);
+  virtual void emit(const Cfg *Func) const = 0;
+  virtual void dump(const Cfg *Func) const;
+
+protected:
+  InstX8632(Cfg *Func, InstKindX8632 Kind, SizeT Maxsrcs, Variable *Dest)
+      : InstTarget(Func, static_cast<InstKind>(Kind), Maxsrcs, Dest) {}
+  virtual ~InstX8632() {}
+  static bool isClassof(const Inst *Inst, InstKindX8632 MyKind) {
+    return Inst->getKind() == static_cast<InstKind>(MyKind);
+  }
+
+private:
+  InstX8632(const InstX8632 &) LLVM_DELETED_FUNCTION;
+  InstX8632 &operator=(const InstX8632 &) LLVM_DELETED_FUNCTION;
+};
+
+// InstX8632Label represents an intra-block label that is the
+// target of an intra-block branch.  These are used for lowering i1
+// calculations, Select instructions, and 64-bit compares on a 32-bit
+// architecture, without basic block splitting.  Basic block splitting
+// is not so desirable for several reasons, one of which is the impact
+// on decisions based on whether a variable's live range spans
+// multiple basic blocks.
+//
+// Intra-block control flow must be used with caution.  Consider the
+// sequence for "c = (a >= b ? x : y)".
+//     cmp a, b
+//     br lt, L1
+//     mov c, x
+//     jmp L2
+//   L1:
+//     mov c, y
+//   L2:
+//
+// Labels L1 and L2 are intra-block labels.  Without knowledge of the
+// intra-block control flow, liveness analysis will determine the "mov
+// c, x" instruction to be dead.  One way to prevent this is to insert
+// a "FakeUse(c)" instruction anywhere between the two "mov c, ..."
+// instructions, e.g.:
+//
+//     cmp a, b
+//     br lt, L1
+//     mov c, x
+//     jmp L2
+//     FakeUse(c)
+//   L1:
+//     mov c, y
+//   L2:
+//
+// The down-side is that "mov c, x" can never be dead-code eliminated
+// even if there are no uses of c.  As unlikely as this situation is,
+// it may be prevented by running dead code elimination before
+// lowering.
+class InstX8632Label : public InstX8632 {
+public:
+  static InstX8632Label *create(Cfg *Func, TargetX8632 *Target) {
+    return new (Func->allocate<InstX8632Label>()) InstX8632Label(Func, Target);
+  }
+  IceString getName(const Cfg *Func) const;
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+
+private:
+  InstX8632Label(Cfg *Func, TargetX8632 *Target);
+  InstX8632Label(const InstX8632Label &) LLVM_DELETED_FUNCTION;
+  InstX8632Label &operator=(const InstX8632Label &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Label() {}
+  SizeT Number; // used only for unique label string generation
+};
+
+// Conditional and unconditional branch instruction.
+class InstX8632Br : public InstX8632 {
+public:
+  enum BrCond {
+#define X(tag, dump, emit) tag,
+    ICEINSTX8632BR_TABLE
+#undef X
+        Br_None
+  };
+
+  // Create a conditional branch to a node.
+  static InstX8632Br *create(Cfg *Func, CfgNode *TargetTrue,
+                             CfgNode *TargetFalse, BrCond Condition) {
+    return new (Func->allocate<InstX8632Br>())
+        InstX8632Br(Func, TargetTrue, TargetFalse, NULL, Condition);
+  }
+  // Create an unconditional branch to a node.
+  static InstX8632Br *create(Cfg *Func, CfgNode *Target) {
+    return new (Func->allocate<InstX8632Br>())
+        InstX8632Br(Func, NULL, Target, NULL, Br_None);
+  }
+  // Create a non-terminator conditional branch to a node, with a
+  // fallthrough to the next instruction in the current node.  This is
+  // used for switch lowering.
+  static InstX8632Br *create(Cfg *Func, CfgNode *Target, BrCond Condition) {
+    return new (Func->allocate<InstX8632Br>())
+        InstX8632Br(Func, Target, NULL, NULL, Condition);
+  }
+  // Create a conditional intra-block branch (or unconditional, if
+  // Condition==None) to a label in the current block.
+  static InstX8632Br *create(Cfg *Func, InstX8632Label *Label,
+                             BrCond Condition) {
+    return new (Func->allocate<InstX8632Br>())
+        InstX8632Br(Func, NULL, NULL, Label, Condition);
+  }
+  CfgNode *getTargetTrue() const { return TargetTrue; }
+  CfgNode *getTargetFalse() const { return TargetFalse; }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Br); }
+
+private:
+  InstX8632Br(Cfg *Func, CfgNode *TargetTrue, CfgNode *TargetFalse,
+              InstX8632Label *Label, BrCond Condition);
+  InstX8632Br(const InstX8632Br &) LLVM_DELETED_FUNCTION;
+  InstX8632Br &operator=(const InstX8632Br &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Br() {}
+  BrCond Condition;
+  CfgNode *TargetTrue;
+  CfgNode *TargetFalse;
+  InstX8632Label *Label; // Intra-block branch target
+};
+
+// Call instruction.  Arguments should have already been pushed.
+class InstX8632Call : public InstX8632 {
+public:
+  static InstX8632Call *create(Cfg *Func, Variable *Dest, Operand *CallTarget) {
+    return new (Func->allocate<InstX8632Call>())
+        InstX8632Call(Func, Dest, CallTarget);
+  }
+  Operand *getCallTarget() const { return getSrc(0); }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Call); }
+
+private:
+  InstX8632Call(Cfg *Func, Variable *Dest, Operand *CallTarget);
+  InstX8632Call(const InstX8632Call &) LLVM_DELETED_FUNCTION;
+  InstX8632Call &operator=(const InstX8632Call &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Call() {}
+};
+
+// See the definition of emitTwoAddress() for a description of
+// ShiftHack.
+void emitTwoAddress(const char *Opcode, const Inst *Inst, const Cfg *Func,
+                    bool ShiftHack = false);
+
+template <InstX8632::InstKindX8632 K, bool ShiftHack = false>
+class InstX8632Binop : public InstX8632 {
+public:
+  // Create an ordinary binary-op instruction like add or sub.
+  static InstX8632Binop *create(Cfg *Func, Variable *Dest, Operand *Source) {
+    return new (Func->allocate<InstX8632Binop>())
+        InstX8632Binop(Func, Dest, Source);
+  }
+  virtual void emit(const Cfg *Func) const {
+    emitTwoAddress(Opcode, this, Func, ShiftHack);
+  }
+  virtual void dump(const Cfg *Func) const {
+    Ostream &Str = Func->getContext()->getStrDump();
+    dumpDest(Func);
+    Str << " = " << Opcode << "." << getDest()->getType() << " ";
+    dumpSources(Func);
+  }
+  static bool classof(const Inst *Inst) { return isClassof(Inst, K); }
+
+private:
+  InstX8632Binop(Cfg *Func, Variable *Dest, Operand *Source)
+      : InstX8632(Func, K, 2, Dest) {
+    addSource(Dest);
+    addSource(Source);
+  }
+  InstX8632Binop(const InstX8632Binop &) LLVM_DELETED_FUNCTION;
+  InstX8632Binop &operator=(const InstX8632Binop &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Binop() {}
+  static const char *Opcode;
+};
+
+template <InstX8632::InstKindX8632 K> class InstX8632Ternop : public InstX8632 {
+public:
+  // Create a ternary-op instruction like div or idiv.
+  static InstX8632Ternop *create(Cfg *Func, Variable *Dest, Operand *Source1,
+                                 Operand *Source2) {
+    return new (Func->allocate<InstX8632Ternop>())
+        InstX8632Ternop(Func, Dest, Source1, Source2);
+  }
+  virtual void emit(const Cfg *Func) const {
+    Ostream &Str = Func->getContext()->getStrEmit();
+    assert(getSrcSize() == 3);
+    Str << "\t" << Opcode << "\t";
+    getSrc(1)->emit(Func);
+    Str << "\n";
+  }
+  virtual void dump(const Cfg *Func) const {
+    Ostream &Str = Func->getContext()->getStrDump();
+    dumpDest(Func);
+    Str << " = " << Opcode << "." << getDest()->getType() << " ";
+    dumpSources(Func);
+  }
+  static bool classof(const Inst *Inst) { return isClassof(Inst, K); }
+
+private:
+  InstX8632Ternop(Cfg *Func, Variable *Dest, Operand *Source1, Operand *Source2)
+      : InstX8632(Func, K, 3, Dest) {
+    addSource(Dest);
+    addSource(Source1);
+    addSource(Source2);
+  }
+  InstX8632Ternop(const InstX8632Ternop &) LLVM_DELETED_FUNCTION;
+  InstX8632Ternop &operator=(const InstX8632Ternop &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Ternop() {}
+  static const char *Opcode;
+};
+
+typedef InstX8632Binop<InstX8632::Add> InstX8632Add;
+typedef InstX8632Binop<InstX8632::Adc> InstX8632Adc;
+typedef InstX8632Binop<InstX8632::Addss> InstX8632Addss;
+typedef InstX8632Binop<InstX8632::Sub> InstX8632Sub;
+typedef InstX8632Binop<InstX8632::Subss> InstX8632Subss;
+typedef InstX8632Binop<InstX8632::Sbb> InstX8632Sbb;
+typedef InstX8632Binop<InstX8632::And> InstX8632And;
+typedef InstX8632Binop<InstX8632::Or> InstX8632Or;
+typedef InstX8632Binop<InstX8632::Xor> InstX8632Xor;
+typedef InstX8632Binop<InstX8632::Imul> InstX8632Imul;
+typedef InstX8632Binop<InstX8632::Mulss> InstX8632Mulss;
+typedef InstX8632Binop<InstX8632::Divss> InstX8632Divss;
+typedef InstX8632Binop<InstX8632::Shl, true> InstX8632Shl;
+typedef InstX8632Binop<InstX8632::Shr, true> InstX8632Shr;
+typedef InstX8632Binop<InstX8632::Sar, true> InstX8632Sar;
+typedef InstX8632Ternop<InstX8632::Idiv> InstX8632Idiv;
+typedef InstX8632Ternop<InstX8632::Div> InstX8632Div;
+
+// Mul instruction - unsigned multiply.
+class InstX8632Mul : public InstX8632 {
+public:
+  static InstX8632Mul *create(Cfg *Func, Variable *Dest, Variable *Source1,
+                              Operand *Source2) {
+    return new (Func->allocate<InstX8632Mul>())
+        InstX8632Mul(Func, Dest, Source1, Source2);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Mul); }
+
+private:
+  InstX8632Mul(Cfg *Func, Variable *Dest, Variable *Source1, Operand *Source2);
+  InstX8632Mul(const InstX8632Mul &) LLVM_DELETED_FUNCTION;
+  InstX8632Mul &operator=(const InstX8632Mul &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Mul() {}
+};
+
+// Shld instruction - shift across a pair of operands.  TODO: Verify
+// that the validator accepts the shld instruction.
+class InstX8632Shld : public InstX8632 {
+public:
+  static InstX8632Shld *create(Cfg *Func, Variable *Dest, Variable *Source1,
+                               Variable *Source2) {
+    return new (Func->allocate<InstX8632Shld>())
+        InstX8632Shld(Func, Dest, Source1, Source2);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Shld); }
+
+private:
+  InstX8632Shld(Cfg *Func, Variable *Dest, Variable *Source1,
+                Variable *Source2);
+  InstX8632Shld(const InstX8632Shld &) LLVM_DELETED_FUNCTION;
+  InstX8632Shld &operator=(const InstX8632Shld &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Shld() {}
+};
+
+// Shrd instruction - shift across a pair of operands.  TODO: Verify
+// that the validator accepts the shrd instruction.
+class InstX8632Shrd : public InstX8632 {
+public:
+  static InstX8632Shrd *create(Cfg *Func, Variable *Dest, Variable *Source1,
+                               Variable *Source2) {
+    return new (Func->allocate<InstX8632Shrd>())
+        InstX8632Shrd(Func, Dest, Source1, Source2);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Shrd); }
+
+private:
+  InstX8632Shrd(Cfg *Func, Variable *Dest, Variable *Source1,
+                Variable *Source2);
+  InstX8632Shrd(const InstX8632Shrd &) LLVM_DELETED_FUNCTION;
+  InstX8632Shrd &operator=(const InstX8632Shrd &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Shrd() {}
+};
+
+// Cdq instruction - sign-extend eax into edx
+class InstX8632Cdq : public InstX8632 {
+public:
+  static InstX8632Cdq *create(Cfg *Func, Variable *Dest, Operand *Source) {
+    return new (Func->allocate<InstX8632Cdq>())
+        InstX8632Cdq(Func, Dest, Source);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Cdq); }
+
+private:
+  InstX8632Cdq(Cfg *Func, Variable *Dest, Operand *Source);
+  InstX8632Cdq(const InstX8632Cdq &) LLVM_DELETED_FUNCTION;
+  InstX8632Cdq &operator=(const InstX8632Cdq &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Cdq() {}
+};
+
+// Cvt instruction - wrapper for cvtsX2sY where X and Y are in {s,d,i}
+// as appropriate.  s=float, d=double, i=int.  X and Y are determined
+// from dest/src types.  Sign and zero extension on the integer
+// operand needs to be done separately.
+class InstX8632Cvt : public InstX8632 {
+public:
+  static InstX8632Cvt *create(Cfg *Func, Variable *Dest, Operand *Source) {
+    return new (Func->allocate<InstX8632Cvt>())
+        InstX8632Cvt(Func, Dest, Source);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Cvt); }
+
+private:
+  InstX8632Cvt(Cfg *Func, Variable *Dest, Operand *Source);
+  InstX8632Cvt(const InstX8632Cvt &) LLVM_DELETED_FUNCTION;
+  InstX8632Cvt &operator=(const InstX8632Cvt &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Cvt() {}
+};
+
+// cmp - Integer compare instruction.
+class InstX8632Icmp : public InstX8632 {
+public:
+  static InstX8632Icmp *create(Cfg *Func, Operand *Src1, Operand *Src2) {
+    return new (Func->allocate<InstX8632Icmp>())
+        InstX8632Icmp(Func, Src1, Src2);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Icmp); }
+
+private:
+  InstX8632Icmp(Cfg *Func, Operand *Src1, Operand *Src2);
+  InstX8632Icmp(const InstX8632Icmp &) LLVM_DELETED_FUNCTION;
+  InstX8632Icmp &operator=(const InstX8632Icmp &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Icmp() {}
+};
+
+// ucomiss/ucomisd - floating-point compare instruction.
+class InstX8632Ucomiss : public InstX8632 {
+public:
+  static InstX8632Ucomiss *create(Cfg *Func, Operand *Src1, Operand *Src2) {
+    return new (Func->allocate<InstX8632Ucomiss>())
+        InstX8632Ucomiss(Func, Src1, Src2);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Ucomiss); }
+
+private:
+  InstX8632Ucomiss(Cfg *Func, Operand *Src1, Operand *Src2);
+  InstX8632Ucomiss(const InstX8632Ucomiss &) LLVM_DELETED_FUNCTION;
+  InstX8632Ucomiss &operator=(const InstX8632Ucomiss &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Ucomiss() {}
+};
+
+// Test instruction.
+class InstX8632Test : public InstX8632 {
+public:
+  static InstX8632Test *create(Cfg *Func, Operand *Source1, Operand *Source2) {
+    return new (Func->allocate<InstX8632Test>())
+        InstX8632Test(Func, Source1, Source2);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Test); }
+
+private:
+  InstX8632Test(Cfg *Func, Operand *Source1, Operand *Source2);
+  InstX8632Test(const InstX8632Test &) LLVM_DELETED_FUNCTION;
+  InstX8632Test &operator=(const InstX8632Test &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Test() {}
+};
+
+// This is essentially a "mov" instruction with an OperandX8632Mem
+// operand instead of Variable as the destination.  It's important
+// for liveness that there is no Dest operand.
+class InstX8632Store : public InstX8632 {
+public:
+  static InstX8632Store *create(Cfg *Func, Operand *Value, OperandX8632 *Mem) {
+    return new (Func->allocate<InstX8632Store>())
+        InstX8632Store(Func, Value, Mem);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Store); }
+
+private:
+  InstX8632Store(Cfg *Func, Operand *Value, OperandX8632 *Mem);
+  InstX8632Store(const InstX8632Store &) LLVM_DELETED_FUNCTION;
+  InstX8632Store &operator=(const InstX8632Store &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Store() {}
+};
+
+// Move/assignment instruction - wrapper for mov/movss/movsd.
+class InstX8632Mov : public InstX8632 {
+public:
+  static InstX8632Mov *create(Cfg *Func, Variable *Dest, Operand *Source) {
+    return new (Func->allocate<InstX8632Mov>())
+        InstX8632Mov(Func, Dest, Source);
+  }
+  virtual bool isRedundantAssign() const;
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Mov); }
+
+private:
+  InstX8632Mov(Cfg *Func, Variable *Dest, Operand *Source);
+  InstX8632Mov(const InstX8632Mov &) LLVM_DELETED_FUNCTION;
+  InstX8632Mov &operator=(const InstX8632Mov &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Mov() {}
+};
+
+// Movsx - copy from a narrower integer type to a wider integer
+// type, with sign extension.
+class InstX8632Movsx : public InstX8632 {
+public:
+  static InstX8632Movsx *create(Cfg *Func, Variable *Dest, Operand *Source) {
+    return new (Func->allocate<InstX8632Movsx>())
+        InstX8632Movsx(Func, Dest, Source);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Movsx); }
+
+private:
+  InstX8632Movsx(Cfg *Func, Variable *Dest, Operand *Source);
+  InstX8632Movsx(const InstX8632Movsx &) LLVM_DELETED_FUNCTION;
+  InstX8632Movsx &operator=(const InstX8632Movsx &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Movsx() {}
+};
+
+// Movsx - copy from a narrower integer type to a wider integer
+// type, with zero extension.
+class InstX8632Movzx : public InstX8632 {
+public:
+  static InstX8632Movzx *create(Cfg *Func, Variable *Dest, Operand *Source) {
+    return new (Func->allocate<InstX8632Movzx>())
+        InstX8632Movzx(Func, Dest, Source);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Movzx); }
+
+private:
+  InstX8632Movzx(Cfg *Func, Variable *Dest, Operand *Source);
+  InstX8632Movzx(const InstX8632Movzx &) LLVM_DELETED_FUNCTION;
+  InstX8632Movzx &operator=(const InstX8632Movzx &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Movzx() {}
+};
+
+// Fld - load a value onto the x87 FP stack.
+class InstX8632Fld : public InstX8632 {
+public:
+  static InstX8632Fld *create(Cfg *Func, Operand *Src) {
+    return new (Func->allocate<InstX8632Fld>()) InstX8632Fld(Func, Src);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Fld); }
+
+private:
+  InstX8632Fld(Cfg *Func, Operand *Src);
+  InstX8632Fld(const InstX8632Fld &) LLVM_DELETED_FUNCTION;
+  InstX8632Fld &operator=(const InstX8632Fld &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Fld() {}
+};
+
+// Fstp - store x87 st(0) into memory and pop st(0).
+class InstX8632Fstp : public InstX8632 {
+public:
+  static InstX8632Fstp *create(Cfg *Func, Variable *Dest) {
+    return new (Func->allocate<InstX8632Fstp>()) InstX8632Fstp(Func, Dest);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Fstp); }
+
+private:
+  InstX8632Fstp(Cfg *Func, Variable *Dest);
+  InstX8632Fstp(const InstX8632Fstp &) LLVM_DELETED_FUNCTION;
+  InstX8632Fstp &operator=(const InstX8632Fstp &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Fstp() {}
+};
+
+class InstX8632Pop : public InstX8632 {
+public:
+  static InstX8632Pop *create(Cfg *Func, Variable *Dest) {
+    return new (Func->allocate<InstX8632Pop>()) InstX8632Pop(Func, Dest);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Pop); }
+
+private:
+  InstX8632Pop(Cfg *Func, Variable *Dest);
+  InstX8632Pop(const InstX8632Pop &) LLVM_DELETED_FUNCTION;
+  InstX8632Pop &operator=(const InstX8632Pop &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Pop() {}
+};
+
+class InstX8632Push : public InstX8632 {
+public:
+  static InstX8632Push *create(Cfg *Func, Operand *Source,
+                               bool SuppressStackAdjustment) {
+    return new (Func->allocate<InstX8632Push>())
+        InstX8632Push(Func, Source, SuppressStackAdjustment);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Push); }
+
+private:
+  InstX8632Push(Cfg *Func, Operand *Source, bool SuppressStackAdjustment);
+  InstX8632Push(const InstX8632Push &) LLVM_DELETED_FUNCTION;
+  InstX8632Push &operator=(const InstX8632Push &) LLVM_DELETED_FUNCTION;
+  bool SuppressStackAdjustment;
+  virtual ~InstX8632Push() {}
+};
+
+// Ret instruction.  Currently only supports the "ret" version that
+// does not pop arguments.  This instruction takes a Source operand
+// (for non-void returning functions) for liveness analysis, though
+// a FakeUse before the ret would do just as well.
+class InstX8632Ret : public InstX8632 {
+public:
+  static InstX8632Ret *create(Cfg *Func, Variable *Source = NULL) {
+    return new (Func->allocate<InstX8632Ret>()) InstX8632Ret(Func, Source);
+  }
+  virtual void emit(const Cfg *Func) const;
+  virtual void dump(const Cfg *Func) const;
+  static bool classof(const Inst *Inst) { return isClassof(Inst, Ret); }
+
+private:
+  InstX8632Ret(Cfg *Func, Variable *Source);
+  InstX8632Ret(const InstX8632Ret &) LLVM_DELETED_FUNCTION;
+  InstX8632Ret &operator=(const InstX8632Ret &) LLVM_DELETED_FUNCTION;
+  virtual ~InstX8632Ret() {}
+};
+
+} // end of namespace Ice
+
+#endif // SUBZERO_SRC_ICEINSTX8632_H
diff --git a/src/IceOperand.cpp b/src/IceOperand.cpp
index 1009a33..56520d3 100644
--- a/src/IceOperand.cpp
+++ b/src/IceOperand.cpp
@@ -16,6 +16,7 @@
 #include "IceCfg.h"
 #include "IceInst.h"
 #include "IceOperand.h"
+#include "IceTargetLowering.h" // dumping stack/frame pointer register
 
 namespace Ice {
 
@@ -27,6 +28,14 @@
   return A.Name < B.Name;
 }
 
+bool operator<(const RegWeight &A, const RegWeight &B) {
+  return A.getWeight() < B.getWeight();
+}
+bool operator<=(const RegWeight &A, const RegWeight &B) { return !(B < A); }
+bool operator==(const RegWeight &A, const RegWeight &B) {
+  return !(B < A) && !(A < B);
+}
+
 void Variable::setUse(const Inst *Inst, const CfgNode *Node) {
   if (DefNode == NULL)
     return;
@@ -66,19 +75,57 @@
   return buf;
 }
 
+Variable Variable::asType(Type Ty) {
+  Variable V(Ty, DefNode, Number, Name);
+  V.RegNum = RegNum;
+  V.StackOffset = StackOffset;
+  return V;
+}
+
 // ======================== dump routines ======================== //
 
+void Variable::emit(const Cfg *Func) const {
+  Func->getTarget()->emitVariable(this, Func);
+}
+
 void Variable::dump(const Cfg *Func) const {
   Ostream &Str = Func->getContext()->getStrDump();
   const CfgNode *CurrentNode = Func->getCurrentNode();
   (void)CurrentNode; // used only in assert()
   assert(CurrentNode == NULL || DefNode == NULL || DefNode == CurrentNode);
-  Str << "%" << getName();
+  if (Func->getContext()->isVerbose(IceV_RegOrigins) ||
+      (!hasReg() && !Func->getTarget()->hasComputedFrame()))
+    Str << "%" << getName();
+  if (hasReg()) {
+    if (Func->getContext()->isVerbose(IceV_RegOrigins))
+      Str << ":";
+    Str << Func->getTarget()->getRegName(RegNum, getType());
+  } else if (Func->getTarget()->hasComputedFrame()) {
+    if (Func->getContext()->isVerbose(IceV_RegOrigins))
+      Str << ":";
+    Str << "[" << Func->getTarget()->getRegName(
+                      Func->getTarget()->getFrameOrStackReg(), IceType_i32);
+    int32_t Offset = getStackOffset();
+    if (Offset) {
+      if (Offset > 0)
+        Str << "+";
+      Str << Offset;
+    }
+    Str << "]";
+  }
 }
 
-void Operand::dump(const Cfg *Func) const {
-  Ostream &Str = Func->getContext()->getStrDump();
-  Str << "Operand<?>";
+void ConstantRelocatable::emit(const Cfg *Func) const {
+  Ostream &Str = Func->getContext()->getStrEmit();
+  if (SuppressMangling)
+    Str << Name;
+  else
+    Str << Func->getContext()->mangleName(Name);
+  if (Offset) {
+    if (Offset > 0)
+      Str << "+";
+    Str << Offset;
+  }
 }
 
 void ConstantRelocatable::dump(const Cfg *Func) const {
@@ -88,4 +135,12 @@
     Str << "+" << Offset;
 }
 
+Ostream &operator<<(Ostream &Str, const RegWeight &W) {
+  if (W.getWeight() == RegWeight::Inf)
+    Str << "Inf";
+  else
+    Str << W.getWeight();
+  return Str;
+}
+
 } // end of namespace Ice
diff --git a/src/IceOperand.h b/src/IceOperand.h
index fcad7b9..c75be78 100644
--- a/src/IceOperand.h
+++ b/src/IceOperand.h
@@ -49,6 +49,7 @@
     assert(I < getNumVars());
     return Vars[I];
   }
+  virtual void emit(const Cfg *Func) const = 0;
   virtual void dump(const Cfg *Func) const = 0;
 
   // Query whether this object was allocated in isolation, or added to
@@ -79,6 +80,7 @@
 // constants are allocated from a global arena and are pooled.
 class Constant : public Operand {
 public:
+  virtual void emit(const Cfg *Func) const = 0;
   virtual void dump(const Cfg *Func) const = 0;
 
   static bool classof(const Operand *Operand) {
@@ -107,6 +109,10 @@
         ConstantPrimitive(Ty, Value);
   }
   T getValue() const { return Value; }
+  virtual void emit(const Cfg *Func) const {
+    Ostream &Str = Func->getContext()->getStrEmit();
+    Str << getValue();
+  }
   virtual void dump(const Cfg *Func) const {
     Ostream &Str = Func->getContext()->getStrDump();
     Str << getValue();
@@ -163,6 +169,7 @@
   IceString getName() const { return Name; }
   void setSuppressMangling(bool Value) { SuppressMangling = Value; }
   bool getSuppressMangling() const { return SuppressMangling; }
+  virtual void emit(const Cfg *Func) const;
   virtual void dump(const Cfg *Func) const;
 
   static bool classof(const Operand *Operand) {
@@ -184,6 +191,34 @@
   bool SuppressMangling;
 };
 
+// RegWeight is a wrapper for a uint32_t weight value, with a
+// special value that represents infinite weight, and an addWeight()
+// method that ensures that W+infinity=infinity.
+class RegWeight {
+public:
+  RegWeight() : Weight(0) {}
+  RegWeight(uint32_t Weight) : Weight(Weight) {}
+  const static uint32_t Inf = ~0; // Force regalloc to give a register
+  const static uint32_t Zero = 0; // Force regalloc NOT to give a register
+  void addWeight(uint32_t Delta) {
+    if (Delta == Inf)
+      Weight = Inf;
+    else if (Weight != Inf)
+      Weight += Delta;
+  }
+  void addWeight(const RegWeight &Other) { addWeight(Other.Weight); }
+  void setWeight(uint32_t Val) { Weight = Val; }
+  uint32_t getWeight() const { return Weight; }
+  bool isInf() const { return Weight == Inf; }
+
+private:
+  uint32_t Weight;
+};
+Ostream &operator<<(Ostream &Str, const RegWeight &W);
+bool operator<(const RegWeight &A, const RegWeight &B);
+bool operator<=(const RegWeight &A, const RegWeight &B);
+bool operator==(const RegWeight &A, const RegWeight &B);
+
 // Variable represents an operand that is register-allocated or
 // stack-allocated.  If it is register-allocated, it will ultimately
 // have a non-negative RegNum field.
@@ -208,23 +243,66 @@
   bool getIsArg() const { return IsArgument; }
   void setIsArg(Cfg *Func);
 
+  int32_t getStackOffset() const { return StackOffset; }
+  void setStackOffset(int32_t Offset) { StackOffset = Offset; }
+
+  static const int32_t NoRegister = -1;
+  bool hasReg() const { return getRegNum() != NoRegister; }
+  int32_t getRegNum() const { return RegNum; }
+  void setRegNum(int32_t NewRegNum) {
+    // Regnum shouldn't be set more than once.
+    assert(!hasReg() || RegNum == NewRegNum);
+    RegNum = NewRegNum;
+  }
+
+  RegWeight getWeight() const { return Weight; }
+  void setWeight(uint32_t NewWeight) { Weight = NewWeight; }
+  void setWeightInfinite() { Weight = RegWeight::Inf; }
+
+  Variable *getPreferredRegister() const { return RegisterPreference; }
+  bool getRegisterOverlap() const { return AllowRegisterOverlap; }
+  void setPreferredRegister(Variable *Prefer, bool Overlap) {
+    RegisterPreference = Prefer;
+    AllowRegisterOverlap = Overlap;
+  }
+
+  Variable *getLo() const { return LoVar; }
+  Variable *getHi() const { return HiVar; }
+  void setLoHi(Variable *Lo, Variable *Hi) {
+    assert(LoVar == NULL);
+    assert(HiVar == NULL);
+    LoVar = Lo;
+    HiVar = Hi;
+  }
+  // Creates a temporary copy of the variable with a different type.
+  // Used primarily for syntactic correctness of textual assembly
+  // emission.  Note that only basic information is copied, in
+  // particular not DefInst, IsArgument, Weight, RegisterPreference,
+  // AllowRegisterOverlap, LoVar, HiVar, VarsReal.
+  Variable asType(Type Ty);
+
+  virtual void emit(const Cfg *Func) const;
   virtual void dump(const Cfg *Func) const;
 
   static bool classof(const Operand *Operand) {
     return Operand->getKind() == kVariable;
   }
 
+  // The destructor is public because of the asType() method.
+  virtual ~Variable() {}
+
 private:
   Variable(Type Ty, const CfgNode *Node, SizeT Index, const IceString &Name)
       : Operand(kVariable, Ty), Number(Index), Name(Name), DefInst(NULL),
-        DefNode(Node), IsArgument(false) {
+        DefNode(Node), IsArgument(false), StackOffset(0), RegNum(NoRegister),
+        Weight(1), RegisterPreference(NULL), AllowRegisterOverlap(false),
+        LoVar(NULL), HiVar(NULL) {
     Vars = VarsReal;
     Vars[0] = this;
     NumVars = 1;
   }
   Variable(const Variable &) LLVM_DELETED_FUNCTION;
   Variable &operator=(const Variable &) LLVM_DELETED_FUNCTION;
-  virtual ~Variable() {}
   // Number is unique across all variables, and is used as a
   // (bit)vector index for liveness analysis.
   const SizeT Number;
@@ -241,6 +319,32 @@
   // of incrementally computing and maintaining the information.
   const CfgNode *DefNode;
   bool IsArgument;
+  // StackOffset is the canonical location on stack (only if
+  // RegNum<0 || IsArgument).
+  int32_t StackOffset;
+  // RegNum is the allocated register, or NoRegister if it isn't
+  // register-allocated.
+  int32_t RegNum;
+  RegWeight Weight; // Register allocation priority
+  // RegisterPreference says that if possible, the register allocator
+  // should prefer the register that was assigned to this linked
+  // variable.  It also allows a spill slot to share its stack
+  // location with another variable, if that variable does not get
+  // register-allocated and therefore has a stack location.
+  Variable *RegisterPreference;
+  // AllowRegisterOverlap says that it is OK to honor
+  // RegisterPreference and "share" a register even if the two live
+  // ranges overlap.
+  bool AllowRegisterOverlap;
+  // LoVar and HiVar are needed for lowering from 64 to 32 bits.  When
+  // lowering from I64 to I32 on a 32-bit architecture, we split the
+  // variable into two machine-size pieces.  LoVar is the low-order
+  // machine-size portion, and HiVar is the remaining high-order
+  // portion.  TODO: It's wasteful to penalize all variables on all
+  // targets this way; use a sparser representation.  It's also
+  // wasteful for a 64-bit target.
+  Variable *LoVar;
+  Variable *HiVar;
   // VarsReal (and Operand::Vars) are set up such that Vars[0] ==
   // this.
   Variable *VarsReal[1];
diff --git a/src/IceTargetLowering.cpp b/src/IceTargetLowering.cpp
new file mode 100644
index 0000000..0d07475
--- /dev/null
+++ b/src/IceTargetLowering.cpp
@@ -0,0 +1,147 @@
+//===- subzero/src/IceTargetLowering.cpp - Basic lowering implementation --===//
+//
+//                        The Subzero Code Generator
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the skeleton of the TargetLowering class,
+// specifically invoking the appropriate lowering method for a given
+// instruction kind and driving global register allocation.  It also
+// implements the non-deleted instruction iteration in
+// LoweringContext.
+//
+//===----------------------------------------------------------------------===//
+
+#include "IceCfg.h" // setError()
+#include "IceCfgNode.h"
+#include "IceOperand.h"
+#include "IceTargetLowering.h"
+#include "IceTargetLoweringX8632.h"
+
+namespace Ice {
+
+void LoweringContext::init(CfgNode *N) {
+  Node = N;
+  Cur = getNode()->getInsts().begin();
+  End = getNode()->getInsts().end();
+  skipDeleted(Cur);
+  Next = Cur;
+  advance(Next);
+}
+
+void LoweringContext::insert(Inst *Inst) {
+  getNode()->getInsts().insert(Next, Inst);
+  Inst->updateVars(getNode());
+}
+
+void LoweringContext::skipDeleted(InstList::iterator &I) {
+  while (I != End && (*I)->isDeleted())
+    ++I;
+}
+
+void LoweringContext::advance(InstList::iterator &I) {
+  if (I != End) {
+    ++I;
+    skipDeleted(I);
+  }
+}
+
+TargetLowering *TargetLowering::createLowering(TargetArch Target, Cfg *Func) {
+  // These statements can be #ifdef'd to specialize the code generator
+  // to a subset of the available targets.  TODO: use CRTP.
+  if (Target == Target_X8632)
+    return TargetX8632::create(Func);
+#if 0
+  if (Target == Target_X8664)
+    return IceTargetX8664::create(Func);
+  if (Target == Target_ARM32)
+    return IceTargetARM32::create(Func);
+  if (Target == Target_ARM64)
+    return IceTargetARM64::create(Func);
+#endif
+  Func->setError("Unsupported target");
+  return NULL;
+}
+
+// Lowers a single instruction according to the information in
+// Context, by checking the Context.Cur instruction kind and calling
+// the appropriate lowering method.  The lowering method should insert
+// target instructions at the Cur.Next insertion point, and should not
+// delete the Context.Cur instruction or advance Context.Cur.
+//
+// The lowering method may look ahead in the instruction stream as
+// desired, and lower additional instructions in conjunction with the
+// current one, for example fusing a compare and branch.  If it does,
+// it should advance Context.Cur to point to the next non-deleted
+// instruction to process, and it should delete any additional
+// instructions it consumes.
+void TargetLowering::lower() {
+  assert(!Context.atEnd());
+  Inst *Inst = *Context.getCur();
+  switch (Inst->getKind()) {
+  case Inst::Alloca:
+    lowerAlloca(llvm::dyn_cast<InstAlloca>(Inst));
+    break;
+  case Inst::Arithmetic:
+    lowerArithmetic(llvm::dyn_cast<InstArithmetic>(Inst));
+    break;
+  case Inst::Assign:
+    lowerAssign(llvm::dyn_cast<InstAssign>(Inst));
+    break;
+  case Inst::Br:
+    lowerBr(llvm::dyn_cast<InstBr>(Inst));
+    break;
+  case Inst::Call:
+    lowerCall(llvm::dyn_cast<InstCall>(Inst));
+    break;
+  case Inst::Cast:
+    lowerCast(llvm::dyn_cast<InstCast>(Inst));
+    break;
+  case Inst::Fcmp:
+    lowerFcmp(llvm::dyn_cast<InstFcmp>(Inst));
+    break;
+  case Inst::Icmp:
+    lowerIcmp(llvm::dyn_cast<InstIcmp>(Inst));
+    break;
+  case Inst::Load:
+    lowerLoad(llvm::dyn_cast<InstLoad>(Inst));
+    break;
+  case Inst::Phi:
+    lowerPhi(llvm::dyn_cast<InstPhi>(Inst));
+    break;
+  case Inst::Ret:
+    lowerRet(llvm::dyn_cast<InstRet>(Inst));
+    break;
+  case Inst::Select:
+    lowerSelect(llvm::dyn_cast<InstSelect>(Inst));
+    break;
+  case Inst::Store:
+    lowerStore(llvm::dyn_cast<InstStore>(Inst));
+    break;
+  case Inst::Switch:
+    lowerSwitch(llvm::dyn_cast<InstSwitch>(Inst));
+    break;
+  case Inst::Unreachable:
+    lowerUnreachable(llvm::dyn_cast<InstUnreachable>(Inst));
+    break;
+  case Inst::FakeDef:
+  case Inst::FakeUse:
+  case Inst::FakeKill:
+  case Inst::Target:
+    // These are all Target instruction types and shouldn't be
+    // encountered at this stage.
+    Func->setError("Can't lower unsupported instruction type");
+    break;
+  }
+  Inst->setDeleted();
+
+  postLower();
+
+  Context.advanceCur();
+  Context.advanceNext();
+}
+
+} // end of namespace Ice
diff --git a/src/IceTargetLowering.h b/src/IceTargetLowering.h
new file mode 100644
index 0000000..a24c51d
--- /dev/null
+++ b/src/IceTargetLowering.h
@@ -0,0 +1,197 @@
+//===- subzero/src/IceTargetLowering.h - Lowering interface -----*- C++ -*-===//
+//
+//                        The Subzero Code Generator
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file declares the TargetLowering and LoweringContext
+// classes.  TargetLowering is an abstract class used to drive the
+// translation/lowering process.  LoweringContext maintains a
+// context for lowering each instruction, offering conveniences such
+// as iterating over non-deleted instructions.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SUBZERO_SRC_ICETARGETLOWERING_H
+#define SUBZERO_SRC_ICETARGETLOWERING_H
+
+#include "IceDefs.h"
+#include "IceTypes.h"
+
+#include "IceInst.h" // for the names of the Inst subtypes
+
+namespace Ice {
+
+// LoweringContext makes it easy to iterate through non-deleted
+// instructions in a node, and insert new (lowered) instructions at
+// the current point.  Along with the instruction list container and
+// associated iterators, it holds the current node, which is needed
+// when inserting new instructions in order to track whether variables
+// are used as single-block or multi-block.
+class LoweringContext {
+public:
+  LoweringContext() : Node(NULL) {}
+  ~LoweringContext() {}
+  void init(CfgNode *Node);
+  Inst *getNextInst() const {
+    if (Next == End)
+      return NULL;
+    return *Next;
+  }
+  CfgNode *getNode() const { return Node; }
+  bool atEnd() const { return Cur == End; }
+  InstList::iterator getCur() const { return Cur; }
+  InstList::iterator getEnd() const { return End; }
+  void insert(Inst *Inst);
+  void advanceCur() { Cur = Next; }
+  void advanceNext() { advance(Next); }
+  void setInsertPoint(const InstList::iterator &Position) { Next = Position; }
+
+private:
+  // Node is the argument to Inst::updateVars().
+  CfgNode *Node;
+  // Cur points to the current instruction being considered.  It is
+  // guaranteed to point to a non-deleted instruction, or to be End.
+  InstList::iterator Cur;
+  // Next doubles as a pointer to the next valid instruction (if any),
+  // and the new-instruction insertion point.  It is also updated for
+  // the caller in case the lowering consumes more than one high-level
+  // instruction.  It is guaranteed to point to a non-deleted
+  // instruction after Cur, or to be End.  TODO: Consider separating
+  // the notion of "next valid instruction" and "new instruction
+  // insertion point", to avoid confusion when previously-deleted
+  // instructions come between the two points.
+  InstList::iterator Next;
+  // End is a copy of Insts.end(), used if Next needs to be advanced.
+  InstList::iterator End;
+
+  void skipDeleted(InstList::iterator &I);
+  void advance(InstList::iterator &I);
+  LoweringContext(const LoweringContext &) LLVM_DELETED_FUNCTION;
+  LoweringContext &operator=(const LoweringContext &) LLVM_DELETED_FUNCTION;
+};
+
+class TargetLowering {
+public:
+  static TargetLowering *createLowering(TargetArch Target, Cfg *Func);
+  void translate() {
+    switch (Ctx->getOptLevel()) {
+    case Opt_m1:
+      translateOm1();
+      break;
+    case Opt_0:
+      translateO0();
+      break;
+    case Opt_1:
+      translateO1();
+      break;
+    case Opt_2:
+      translateO2();
+      break;
+    default:
+      Func->setError("Target doesn't specify lowering steps.");
+      break;
+    }
+  }
+  virtual void translateOm1() {
+    Func->setError("Target doesn't specify Om1 lowering steps.");
+  }
+  virtual void translateO0() {
+    Func->setError("Target doesn't specify O0 lowering steps.");
+  }
+  virtual void translateO1() {
+    Func->setError("Target doesn't specify O1 lowering steps.");
+  }
+  virtual void translateO2() {
+    Func->setError("Target doesn't specify O2 lowering steps.");
+  }
+
+  // Lowers a single instruction.
+  void lower();
+
+  // Returns a variable pre-colored to the specified physical
+  // register.  This is generally used to get very direct access to
+  // the register such as in the prolog or epilog or for marking
+  // scratch registers as killed by a call.
+  virtual Variable *getPhysicalRegister(SizeT RegNum) = 0;
+  // Returns a printable name for the register.
+  virtual IceString getRegName(SizeT RegNum, Type Ty) const = 0;
+
+  virtual bool hasFramePointer() const { return false; }
+  virtual SizeT getFrameOrStackReg() const = 0;
+  virtual size_t typeWidthInBytesOnStack(Type Ty) = 0;
+  bool hasComputedFrame() const { return HasComputedFrame; }
+  int32_t getStackAdjustment() const { return StackAdjustment; }
+  void updateStackAdjustment(int32_t Offset) { StackAdjustment += Offset; }
+  void resetStackAdjustment() { StackAdjustment = 0; }
+  LoweringContext &getContext() { return Context; }
+
+  enum RegSet {
+    RegSet_None = 0,
+    RegSet_CallerSave = 1 << 0,
+    RegSet_CalleeSave = 1 << 1,
+    RegSet_StackPointer = 1 << 2,
+    RegSet_FramePointer = 1 << 3,
+    RegSet_All = ~RegSet_None
+  };
+  typedef uint32_t RegSetMask;
+
+  virtual llvm::SmallBitVector getRegisterSet(RegSetMask Include,
+                                              RegSetMask Exclude) const = 0;
+  virtual const llvm::SmallBitVector &getRegisterSetForType(Type Ty) const = 0;
+  void regAlloc();
+
+  virtual void emitVariable(const Variable *Var, const Cfg *Func) const = 0;
+
+  virtual void addProlog(CfgNode *Node) = 0;
+  virtual void addEpilog(CfgNode *Node) = 0;
+
+  virtual ~TargetLowering() {}
+
+protected:
+  TargetLowering(Cfg *Func)
+      : Func(Func), Ctx(Func->getContext()), HasComputedFrame(false),
+        StackAdjustment(0) {}
+  virtual void lowerAlloca(const InstAlloca *Inst) = 0;
+  virtual void lowerArithmetic(const InstArithmetic *Inst) = 0;
+  virtual void lowerAssign(const InstAssign *Inst) = 0;
+  virtual void lowerBr(const InstBr *Inst) = 0;
+  virtual void lowerCall(const InstCall *Inst) = 0;
+  virtual void lowerCast(const InstCast *Inst) = 0;
+  virtual void lowerFcmp(const InstFcmp *Inst) = 0;
+  virtual void lowerIcmp(const InstIcmp *Inst) = 0;
+  virtual void lowerLoad(const InstLoad *Inst) = 0;
+  virtual void lowerPhi(const InstPhi *Inst) = 0;
+  virtual void lowerRet(const InstRet *Inst) = 0;
+  virtual void lowerSelect(const InstSelect *Inst) = 0;
+  virtual void lowerStore(const InstStore *Inst) = 0;
+  virtual void lowerSwitch(const InstSwitch *Inst) = 0;
+  virtual void lowerUnreachable(const InstUnreachable *Inst) = 0;
+
+  // This gives the target an opportunity to post-process the lowered
+  // expansion before returning.  The primary intention is to do some
+  // Register Manager activity as necessary, specifically to eagerly
+  // allocate registers based on affinity and other factors.  The
+  // simplest lowering does nothing here and leaves it all to a
+  // subsequent global register allocation pass.
+  virtual void postLower() {}
+
+  Cfg *Func;
+  GlobalContext *Ctx;
+  bool HasComputedFrame;
+  // StackAdjustment keeps track of the current stack offset from its
+  // natural location, as arguments are pushed for a function call.
+  int32_t StackAdjustment;
+  LoweringContext Context;
+
+private:
+  TargetLowering(const TargetLowering &) LLVM_DELETED_FUNCTION;
+  TargetLowering &operator=(const TargetLowering &) LLVM_DELETED_FUNCTION;
+};
+
+} // end of namespace Ice
+
+#endif // SUBZERO_SRC_ICETARGETLOWERING_H
diff --git a/src/IceTargetLoweringX8632.cpp b/src/IceTargetLoweringX8632.cpp
new file mode 100644
index 0000000..32246c4
--- /dev/null
+++ b/src/IceTargetLoweringX8632.cpp
@@ -0,0 +1,1881 @@
+//===- subzero/src/IceTargetLoweringX8632.cpp - x86-32 lowering -----------===//
+//
+//                        The Subzero Code Generator
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the TargetLoweringX8632 class, which
+// consists almost entirely of the lowering sequence for each
+// high-level instruction.  It also implements
+// TargetX8632Fast::postLower() which does the simplest possible
+// register allocation for the "fast" target.
+//
+//===----------------------------------------------------------------------===//
+
+#include "IceDefs.h"
+#include "IceCfg.h"
+#include "IceCfgNode.h"
+#include "IceInstX8632.h"
+#include "IceOperand.h"
+#include "IceTargetLoweringX8632.def"
+#include "IceTargetLoweringX8632.h"
+
+namespace Ice {
+
+namespace {
+
+// The following table summarizes the logic for lowering the fcmp instruction.
+// There is one table entry for each of the 16 conditions.  A comment in
+// lowerFcmp() describes the lowering template.  In the most general case, there
+// is a compare followed by two conditional branches, because some fcmp
+// conditions don't map to a single x86 conditional branch.  However, in many
+// cases it is possible to swap the operands in the comparison and have a single
+// conditional branch.  Since it's quite tedious to validate the table by hand,
+// good execution tests are helpful.
+
+const struct TableFcmp_ {
+  uint32_t Default;
+  bool SwapOperands;
+  InstX8632Br::BrCond C1, C2;
+} TableFcmp[] = {
+#define X(val, dflt, swap, C1, C2)                                             \
+  { dflt, swap, InstX8632Br::C1, InstX8632Br::C2 }                             \
+  ,
+    FCMPX8632_TABLE
+#undef X
+  };
+const size_t TableFcmpSize = llvm::array_lengthof(TableFcmp);
+
+// The following table summarizes the logic for lowering the icmp instruction
+// for i32 and narrower types.  Each icmp condition has a clear mapping to an
+// x86 conditional branch instruction.
+
+const struct TableIcmp32_ {
+  InstX8632Br::BrCond Mapping;
+} TableIcmp32[] = {
+#define X(val, C_32, C1_64, C2_64, C3_64)                                      \
+  { InstX8632Br::C_32 }                                                        \
+  ,
+    ICMPX8632_TABLE
+#undef X
+  };
+const size_t TableIcmp32Size = llvm::array_lengthof(TableIcmp32);
+
+// The following table summarizes the logic for lowering the icmp instruction
+// for the i64 type.  For Eq and Ne, two separate 32-bit comparisons and
+// conditional branches are needed.  For the other conditions, three separate
+// conditional branches are needed.
+const struct TableIcmp64_ {
+  InstX8632Br::BrCond C1, C2, C3;
+} TableIcmp64[] = {
+#define X(val, C_32, C1_64, C2_64, C3_64)                                      \
+  { InstX8632Br::C1_64, InstX8632Br::C2_64, InstX8632Br::C3_64 }               \
+  ,
+    ICMPX8632_TABLE
+#undef X
+  };
+const size_t TableIcmp64Size = llvm::array_lengthof(TableIcmp64);
+
+InstX8632Br::BrCond getIcmp32Mapping(InstIcmp::ICond Cond) {
+  size_t Index = static_cast<size_t>(Cond);
+  assert(Index < TableIcmp32Size);
+  return TableIcmp32[Index].Mapping;
+}
+
+// In some cases, there are x-macros tables for both high-level and
+// low-level instructions/operands that use the same enum key value.
+// The tables are kept separate to maintain a proper separation
+// between abstraction layers.  There is a risk that the tables
+// could get out of sync if enum values are reordered or if entries
+// are added or deleted.  This dummy function uses static_assert to
+// ensure everything is kept in sync.
+void xMacroIntegrityCheck() {
+  // Validate the enum values in FCMPX8632_TABLE.
+  {
+    // Define a temporary set of enum values based on low-level
+    // table entries.
+    enum _tmp_enum {
+#define X(val, dflt, swap, C1, C2) _tmp_##val,
+      FCMPX8632_TABLE
+#undef X
+    };
+// Define a set of constants based on high-level table entries.
+#define X(tag, str) static const int _table1_##tag = InstFcmp::tag;
+    ICEINSTFCMP_TABLE;
+#undef X
+// Define a set of constants based on low-level table entries,
+// and ensure the table entry keys are consistent.
+#define X(val, dflt, swap, C1, C2)                                             \
+  static const int _table2_##val = _tmp_##val;                                 \
+  STATIC_ASSERT(_table1_##val == _table2_##val);
+    FCMPX8632_TABLE;
+#undef X
+// Repeat the static asserts with respect to the high-level
+// table entries in case the high-level table has extra entries.
+#define X(tag, str) STATIC_ASSERT(_table1_##tag == _table2_##tag);
+    ICEINSTFCMP_TABLE;
+#undef X
+  }
+
+  // Validate the enum values in ICMPX8632_TABLE.
+  {
+    // Define a temporary set of enum values based on low-level
+    // table entries.
+    enum _tmp_enum {
+#define X(val, C_32, C1_64, C2_64, C3_64) _tmp_##val,
+      ICMPX8632_TABLE
+#undef X
+    };
+// Define a set of constants based on high-level table entries.
+#define X(tag, str) static const int _table1_##tag = InstIcmp::tag;
+    ICEINSTICMP_TABLE;
+#undef X
+// Define a set of constants based on low-level table entries,
+// and ensure the table entry keys are consistent.
+#define X(val, C_32, C1_64, C2_64, C3_64)                                      \
+  static const int _table2_##val = _tmp_##val;                                 \
+  STATIC_ASSERT(_table1_##val == _table2_##val);
+    ICMPX8632_TABLE;
+#undef X
+// Repeat the static asserts with respect to the high-level
+// table entries in case the high-level table has extra entries.
+#define X(tag, str) STATIC_ASSERT(_table1_##tag == _table2_##tag);
+    ICEINSTICMP_TABLE;
+#undef X
+  }
+
+  // Validate the enum values in ICETYPEX8632_TABLE.
+  {
+    // Define a temporary set of enum values based on low-level
+    // table entries.
+    enum _tmp_enum {
+#define X(tag, cvt, sdss, width) _tmp_##tag,
+      ICETYPEX8632_TABLE
+#undef X
+    };
+// Define a set of constants based on high-level table entries.
+#define X(tag, size, align, str) static const int _table1_##tag = tag;
+    ICETYPE_TABLE;
+#undef X
+// Define a set of constants based on low-level table entries,
+// and ensure the table entry keys are consistent.
+#define X(tag, cvt, sdss, width)                                               \
+  static const int _table2_##tag = _tmp_##tag;                                 \
+  STATIC_ASSERT(_table1_##tag == _table2_##tag);
+    ICETYPEX8632_TABLE;
+#undef X
+// Repeat the static asserts with respect to the high-level
+// table entries in case the high-level table has extra entries.
+#define X(tag, size, align, str) STATIC_ASSERT(_table1_##tag == _table2_##tag);
+    ICETYPE_TABLE;
+#undef X
+  }
+}
+
+} // end of anonymous namespace
+
+TargetX8632::TargetX8632(Cfg *Func)
+    : TargetLowering(Func), IsEbpBasedFrame(false), FrameSizeLocals(0),
+      LocalsSizeBytes(0), NextLabelNumber(0), ComputedLiveRanges(false),
+      PhysicalRegisters(VarList(Reg_NUM)) {
+  // TODO: Don't initialize IntegerRegisters and friends every time.
+  // Instead, initialize in some sort of static initializer for the
+  // class.
+  llvm::SmallBitVector IntegerRegisters(Reg_NUM);
+  llvm::SmallBitVector IntegerRegistersI8(Reg_NUM);
+  llvm::SmallBitVector FloatRegisters(Reg_NUM);
+  llvm::SmallBitVector InvalidRegisters(Reg_NUM);
+  ScratchRegs.resize(Reg_NUM);
+#define X(val, init, name, name16, name8, scratch, preserved, stackptr,        \
+          frameptr, isI8, isInt, isFP)                                         \
+  IntegerRegisters[val] = isInt;                                               \
+  IntegerRegistersI8[val] = isI8;                                              \
+  FloatRegisters[val] = isFP;                                                  \
+  ScratchRegs[val] = scratch;
+  REGX8632_TABLE;
+#undef X
+  TypeToRegisterSet[IceType_void] = InvalidRegisters;
+  TypeToRegisterSet[IceType_i1] = IntegerRegistersI8;
+  TypeToRegisterSet[IceType_i8] = IntegerRegistersI8;
+  TypeToRegisterSet[IceType_i16] = IntegerRegisters;
+  TypeToRegisterSet[IceType_i32] = IntegerRegisters;
+  TypeToRegisterSet[IceType_i64] = IntegerRegisters;
+  TypeToRegisterSet[IceType_f32] = FloatRegisters;
+  TypeToRegisterSet[IceType_f64] = FloatRegisters;
+}
+
+void TargetX8632::translateOm1() {
+  GlobalContext *Context = Func->getContext();
+  Ostream &Str = Context->getStrDump();
+  Timer T_placePhiLoads;
+  Func->placePhiLoads();
+  if (Func->hasError())
+    return;
+  T_placePhiLoads.printElapsedUs(Context, "placePhiLoads()");
+  Timer T_placePhiStores;
+  Func->placePhiStores();
+  if (Func->hasError())
+    return;
+  T_placePhiStores.printElapsedUs(Context, "placePhiStores()");
+  Timer T_deletePhis;
+  Func->deletePhis();
+  if (Func->hasError())
+    return;
+  T_deletePhis.printElapsedUs(Context, "deletePhis()");
+  if (Context->isVerbose()) {
+    Str << "================ After Phi lowering ================\n";
+    Func->dump();
+  }
+
+  Timer T_genCode;
+  Func->genCode();
+  if (Func->hasError())
+    return;
+  T_genCode.printElapsedUs(Context, "genCode()");
+  if (Context->isVerbose()) {
+    Str << "================ After initial x8632 codegen ================\n";
+    Func->dump();
+  }
+
+  Timer T_genFrame;
+  Func->genFrame();
+  if (Func->hasError())
+    return;
+  T_genFrame.printElapsedUs(Context, "genFrame()");
+  if (Context->isVerbose()) {
+    Str << "================ After stack frame mapping ================\n";
+    Func->dump();
+  }
+}
+
+IceString TargetX8632::RegNames[] = {
+#define X(val, init, name, name16, name8, scratch, preserved, stackptr,        \
+          frameptr, isI8, isInt, isFP)                                         \
+  name,
+  REGX8632_TABLE
+#undef X
+};
+
+Variable *TargetX8632::getPhysicalRegister(SizeT RegNum) {
+  assert(RegNum < PhysicalRegisters.size());
+  Variable *Reg = PhysicalRegisters[RegNum];
+  if (Reg == NULL) {
+    CfgNode *Node = NULL; // NULL means multi-block lifetime
+    Reg = Func->makeVariable(IceType_i32, Node);
+    Reg->setRegNum(RegNum);
+    PhysicalRegisters[RegNum] = Reg;
+  }
+  return Reg;
+}
+
+IceString TargetX8632::getRegName(SizeT RegNum, Type Ty) const {
+  assert(RegNum < Reg_NUM);
+  static IceString RegNames8[] = {
+#define X(val, init, name, name16, name8, scratch, preserved, stackptr,        \
+          frameptr, isI8, isInt, isFP)                                         \
+  "" name8,
+    REGX8632_TABLE
+#undef X
+  };
+  static IceString RegNames16[] = {
+#define X(val, init, name, name16, name8, scratch, preserved, stackptr,        \
+          frameptr, isI8, isInt, isFP)                                         \
+  "" name16,
+    REGX8632_TABLE
+#undef X
+  };
+  switch (Ty) {
+  case IceType_i1:
+  case IceType_i8:
+    return RegNames8[RegNum];
+  case IceType_i16:
+    return RegNames16[RegNum];
+  default:
+    return RegNames[RegNum];
+  }
+}
+
+void TargetX8632::emitVariable(const Variable *Var, const Cfg *Func) const {
+  Ostream &Str = Ctx->getStrEmit();
+  assert(Var->getLocalUseNode() == NULL ||
+         Var->getLocalUseNode() == Func->getCurrentNode());
+  if (Var->hasReg()) {
+    Str << getRegName(Var->getRegNum(), Var->getType());
+    return;
+  }
+  Str << InstX8632::getWidthString(Var->getType());
+  Str << " [" << getRegName(getFrameOrStackReg(), IceType_i32);
+  int32_t Offset = Var->getStackOffset() + getStackAdjustment();
+  if (Offset) {
+    if (Offset > 0)
+      Str << "+";
+    Str << Offset;
+  }
+  Str << "]";
+}
+
+// Helper function for addProlog().  Sets the frame offset for Arg,
+// updates InArgsSizeBytes according to Arg's width, and generates an
+// instruction to copy Arg into its assigned register if applicable.
+// For an I64 arg that has been split into Lo and Hi components, it
+// calls itself recursively on the components, taking care to handle
+// Lo first because of the little-endian architecture.
+void TargetX8632::setArgOffsetAndCopy(Variable *Arg, Variable *FramePtr,
+                                      int32_t BasicFrameOffset,
+                                      int32_t &InArgsSizeBytes) {
+  Variable *Lo = Arg->getLo();
+  Variable *Hi = Arg->getHi();
+  Type Ty = Arg->getType();
+  if (Lo && Hi && Ty == IceType_i64) {
+    assert(Lo->getType() != IceType_i64); // don't want infinite recursion
+    assert(Hi->getType() != IceType_i64); // don't want infinite recursion
+    setArgOffsetAndCopy(Lo, FramePtr, BasicFrameOffset, InArgsSizeBytes);
+    setArgOffsetAndCopy(Hi, FramePtr, BasicFrameOffset, InArgsSizeBytes);
+    return;
+  }
+  Arg->setStackOffset(BasicFrameOffset + InArgsSizeBytes);
+  if (Arg->hasReg()) {
+    assert(Ty != IceType_i64);
+    OperandX8632Mem *Mem = OperandX8632Mem::create(
+        Func, Ty, FramePtr,
+        Ctx->getConstantInt(IceType_i32, Arg->getStackOffset()));
+    _mov(Arg, Mem);
+  }
+  InArgsSizeBytes += typeWidthInBytesOnStack(Ty);
+}
+
+void TargetX8632::addProlog(CfgNode *Node) {
+  // If SimpleCoalescing is false, each variable without a register
+  // gets its own unique stack slot, which leads to large stack
+  // frames.  If SimpleCoalescing is true, then each "global" variable
+  // without a register gets its own slot, but "local" variable slots
+  // are reused across basic blocks.  E.g., if A and B are local to
+  // block 1 and C is local to block 2, then C may share a slot with A
+  // or B.
+  const bool SimpleCoalescing = true;
+  int32_t InArgsSizeBytes = 0;
+  int32_t RetIpSizeBytes = 4;
+  int32_t PreservedRegsSizeBytes = 0;
+  LocalsSizeBytes = 0;
+  Context.init(Node);
+  Context.setInsertPoint(Context.getCur());
+
+  // Determine stack frame offsets for each Variable without a
+  // register assignment.  This can be done as one variable per stack
+  // slot.  Or, do coalescing by running the register allocator again
+  // with an infinite set of registers (as a side effect, this gives
+  // variables a second chance at physical register assignment).
+  //
+  // A middle ground approach is to leverage sparsity and allocate one
+  // block of space on the frame for globals (variables with
+  // multi-block lifetime), and one block to share for locals
+  // (single-block lifetime).
+
+  llvm::SmallBitVector CalleeSaves =
+      getRegisterSet(RegSet_CalleeSave, RegSet_None);
+
+  int32_t GlobalsSize = 0;
+  std::vector<int> LocalsSize(Func->getNumNodes());
+
+  // Prepass.  Compute RegsUsed, PreservedRegsSizeBytes, and
+  // LocalsSizeBytes.
+  RegsUsed = llvm::SmallBitVector(CalleeSaves.size());
+  const VarList &Variables = Func->getVariables();
+  const VarList &Args = Func->getArgs();
+  for (VarList::const_iterator I = Variables.begin(), E = Variables.end();
+       I != E; ++I) {
+    Variable *Var = *I;
+    if (Var->hasReg()) {
+      RegsUsed[Var->getRegNum()] = true;
+      continue;
+    }
+    // An argument passed on the stack already has a stack slot.
+    if (Var->getIsArg())
+      continue;
+    // A spill slot linked to a variable with a stack slot should reuse
+    // that stack slot.
+    if (Var->getWeight() == RegWeight::Zero && Var->getRegisterOverlap()) {
+      if (Variable *Linked = Var->getPreferredRegister()) {
+        if (!Linked->hasReg())
+          continue;
+      }
+    }
+    int32_t Increment = typeWidthInBytesOnStack(Var->getType());
+    if (SimpleCoalescing) {
+      if (Var->isMultiblockLife()) {
+        GlobalsSize += Increment;
+      } else {
+        SizeT NodeIndex = Var->getLocalUseNode()->getIndex();
+        LocalsSize[NodeIndex] += Increment;
+        if (LocalsSize[NodeIndex] > LocalsSizeBytes)
+          LocalsSizeBytes = LocalsSize[NodeIndex];
+      }
+    } else {
+      LocalsSizeBytes += Increment;
+    }
+  }
+  LocalsSizeBytes += GlobalsSize;
+
+  // Add push instructions for preserved registers.
+  for (SizeT i = 0; i < CalleeSaves.size(); ++i) {
+    if (CalleeSaves[i] && RegsUsed[i]) {
+      PreservedRegsSizeBytes += 4;
+      const bool SuppressStackAdjustment = true;
+      _push(getPhysicalRegister(i), SuppressStackAdjustment);
+    }
+  }
+
+  // Generate "push ebp; mov ebp, esp"
+  if (IsEbpBasedFrame) {
+    assert((RegsUsed & getRegisterSet(RegSet_FramePointer, RegSet_None))
+               .count() == 0);
+    PreservedRegsSizeBytes += 4;
+    Variable *ebp = getPhysicalRegister(Reg_ebp);
+    Variable *esp = getPhysicalRegister(Reg_esp);
+    const bool SuppressStackAdjustment = true;
+    _push(ebp, SuppressStackAdjustment);
+    _mov(ebp, esp);
+  }
+
+  // Generate "sub esp, LocalsSizeBytes"
+  if (LocalsSizeBytes)
+    _sub(getPhysicalRegister(Reg_esp),
+         Ctx->getConstantInt(IceType_i32, LocalsSizeBytes));
+
+  resetStackAdjustment();
+
+  // Fill in stack offsets for args, and copy args into registers for
+  // those that were register-allocated.  Args are pushed right to
+  // left, so Arg[0] is closest to the stack/frame pointer.
+  //
+  // TODO: Make this right for different width args, calling
+  // conventions, etc.  For one thing, args passed in registers will
+  // need to be copied/shuffled to their home registers (the
+  // RegManager code may have some permutation logic to leverage),
+  // and if they have no home register, home space will need to be
+  // allocated on the stack to copy into.
+  Variable *FramePtr = getPhysicalRegister(getFrameOrStackReg());
+  int32_t BasicFrameOffset = PreservedRegsSizeBytes + RetIpSizeBytes;
+  if (!IsEbpBasedFrame)
+    BasicFrameOffset += LocalsSizeBytes;
+  for (SizeT i = 0; i < Args.size(); ++i) {
+    Variable *Arg = Args[i];
+    setArgOffsetAndCopy(Arg, FramePtr, BasicFrameOffset, InArgsSizeBytes);
+  }
+
+  // Fill in stack offsets for locals.
+  int32_t TotalGlobalsSize = GlobalsSize;
+  GlobalsSize = 0;
+  LocalsSize.assign(LocalsSize.size(), 0);
+  int32_t NextStackOffset = 0;
+  for (VarList::const_iterator I = Variables.begin(), E = Variables.end();
+       I != E; ++I) {
+    Variable *Var = *I;
+    if (Var->hasReg()) {
+      RegsUsed[Var->getRegNum()] = true;
+      continue;
+    }
+    if (Var->getIsArg())
+      continue;
+    if (Var->getWeight() == RegWeight::Zero && Var->getRegisterOverlap()) {
+      if (Variable *Linked = Var->getPreferredRegister()) {
+        if (!Linked->hasReg()) {
+          // TODO: Make sure Linked has already been assigned a stack
+          // slot.
+          Var->setStackOffset(Linked->getStackOffset());
+          continue;
+        }
+      }
+    }
+    int32_t Increment = typeWidthInBytesOnStack(Var->getType());
+    if (SimpleCoalescing) {
+      if (Var->isMultiblockLife()) {
+        GlobalsSize += Increment;
+        NextStackOffset = GlobalsSize;
+      } else {
+        SizeT NodeIndex = Var->getLocalUseNode()->getIndex();
+        LocalsSize[NodeIndex] += Increment;
+        NextStackOffset = TotalGlobalsSize + LocalsSize[NodeIndex];
+      }
+    } else {
+      NextStackOffset += Increment;
+    }
+    if (IsEbpBasedFrame)
+      Var->setStackOffset(-NextStackOffset);
+    else
+      Var->setStackOffset(LocalsSizeBytes - NextStackOffset);
+  }
+  this->FrameSizeLocals = NextStackOffset;
+  this->HasComputedFrame = true;
+
+  if (Func->getContext()->isVerbose(IceV_Frame)) {
+    Func->getContext()->getStrDump() << "LocalsSizeBytes=" << LocalsSizeBytes
+                                     << "\n"
+                                     << "InArgsSizeBytes=" << InArgsSizeBytes
+                                     << "\n"
+                                     << "PreservedRegsSizeBytes="
+                                     << PreservedRegsSizeBytes << "\n";
+  }
+}
+
+void TargetX8632::addEpilog(CfgNode *Node) {
+  InstList &Insts = Node->getInsts();
+  InstList::reverse_iterator RI, E;
+  for (RI = Insts.rbegin(), E = Insts.rend(); RI != E; ++RI) {
+    if (llvm::isa<InstX8632Ret>(*RI))
+      break;
+  }
+  if (RI == E)
+    return;
+
+  // Convert the reverse_iterator position into its corresponding
+  // (forward) iterator position.
+  InstList::iterator InsertPoint = RI.base();
+  --InsertPoint;
+  Context.init(Node);
+  Context.setInsertPoint(InsertPoint);
+
+  Variable *esp = getPhysicalRegister(Reg_esp);
+  if (IsEbpBasedFrame) {
+    Variable *ebp = getPhysicalRegister(Reg_ebp);
+    _mov(esp, ebp);
+    _pop(ebp);
+  } else {
+    // add esp, LocalsSizeBytes
+    if (LocalsSizeBytes)
+      _add(esp, Ctx->getConstantInt(IceType_i32, LocalsSizeBytes));
+  }
+
+  // Add pop instructions for preserved registers.
+  llvm::SmallBitVector CalleeSaves =
+      getRegisterSet(RegSet_CalleeSave, RegSet_None);
+  for (SizeT i = 0; i < CalleeSaves.size(); ++i) {
+    SizeT j = CalleeSaves.size() - i - 1;
+    if (j == Reg_ebp && IsEbpBasedFrame)
+      continue;
+    if (CalleeSaves[j] && RegsUsed[j]) {
+      _pop(getPhysicalRegister(j));
+    }
+  }
+}
+
+void TargetX8632::split64(Variable *Var) {
+  switch (Var->getType()) {
+  default:
+    return;
+  case IceType_i64:
+  // TODO: Only consider F64 if we need to push each half when
+  // passing as an argument to a function call.  Note that each half
+  // is still typed as I32.
+  case IceType_f64:
+    break;
+  }
+  Variable *Lo = Var->getLo();
+  Variable *Hi = Var->getHi();
+  if (Lo) {
+    assert(Hi);
+    return;
+  }
+  assert(Hi == NULL);
+  Lo = Func->makeVariable(IceType_i32, Context.getNode(),
+                          Var->getName() + "__lo");
+  Hi = Func->makeVariable(IceType_i32, Context.getNode(),
+                          Var->getName() + "__hi");
+  Var->setLoHi(Lo, Hi);
+  if (Var->getIsArg()) {
+    Lo->setIsArg(Func);
+    Hi->setIsArg(Func);
+  }
+}
+
+Operand *TargetX8632::loOperand(Operand *Operand) {
+  assert(Operand->getType() == IceType_i64);
+  if (Operand->getType() != IceType_i64)
+    return Operand;
+  if (Variable *Var = llvm::dyn_cast<Variable>(Operand)) {
+    split64(Var);
+    return Var->getLo();
+  }
+  if (ConstantInteger *Const = llvm::dyn_cast<ConstantInteger>(Operand)) {
+    uint64_t Mask = (1ull << 32) - 1;
+    return Ctx->getConstantInt(IceType_i32, Const->getValue() & Mask);
+  }
+  if (OperandX8632Mem *Mem = llvm::dyn_cast<OperandX8632Mem>(Operand)) {
+    return OperandX8632Mem::create(Func, IceType_i32, Mem->getBase(),
+                                   Mem->getOffset(), Mem->getIndex(),
+                                   Mem->getShift());
+  }
+  llvm_unreachable("Unsupported operand type");
+  return NULL;
+}
+
+Operand *TargetX8632::hiOperand(Operand *Operand) {
+  assert(Operand->getType() == IceType_i64);
+  if (Operand->getType() != IceType_i64)
+    return Operand;
+  if (Variable *Var = llvm::dyn_cast<Variable>(Operand)) {
+    split64(Var);
+    return Var->getHi();
+  }
+  if (ConstantInteger *Const = llvm::dyn_cast<ConstantInteger>(Operand)) {
+    return Ctx->getConstantInt(IceType_i32, Const->getValue() >> 32);
+  }
+  if (OperandX8632Mem *Mem = llvm::dyn_cast<OperandX8632Mem>(Operand)) {
+    Constant *Offset = Mem->getOffset();
+    if (Offset == NULL)
+      Offset = Ctx->getConstantInt(IceType_i32, 4);
+    else if (ConstantInteger *IntOffset =
+                 llvm::dyn_cast<ConstantInteger>(Offset)) {
+      Offset = Ctx->getConstantInt(IceType_i32, 4 + IntOffset->getValue());
+    } else if (ConstantRelocatable *SymOffset =
+                   llvm::dyn_cast<ConstantRelocatable>(Offset)) {
+      Offset = Ctx->getConstantSym(IceType_i32, 4 + SymOffset->getOffset(),
+                                   SymOffset->getName());
+    }
+    return OperandX8632Mem::create(Func, IceType_i32, Mem->getBase(), Offset,
+                                   Mem->getIndex(), Mem->getShift());
+  }
+  llvm_unreachable("Unsupported operand type");
+  return NULL;
+}
+
+llvm::SmallBitVector TargetX8632::getRegisterSet(RegSetMask Include,
+                                                 RegSetMask Exclude) const {
+  llvm::SmallBitVector Registers(Reg_NUM);
+
+#define X(val, init, name, name16, name8, scratch, preserved, stackptr,        \
+          frameptr, isI8, isInt, isFP)                                         \
+  if (scratch && (Include & RegSet_CallerSave))                                \
+    Registers[val] = true;                                                     \
+  if (preserved && (Include & RegSet_CalleeSave))                              \
+    Registers[val] = true;                                                     \
+  if (stackptr && (Include & RegSet_StackPointer))                             \
+    Registers[val] = true;                                                     \
+  if (frameptr && (Include & RegSet_FramePointer))                             \
+    Registers[val] = true;                                                     \
+  if (scratch && (Exclude & RegSet_CallerSave))                                \
+    Registers[val] = false;                                                    \
+  if (preserved && (Exclude & RegSet_CalleeSave))                              \
+    Registers[val] = false;                                                    \
+  if (stackptr && (Exclude & RegSet_StackPointer))                             \
+    Registers[val] = false;                                                    \
+  if (frameptr && (Exclude & RegSet_FramePointer))                             \
+    Registers[val] = false;
+
+  REGX8632_TABLE
+
+#undef X
+
+  return Registers;
+}
+
+void TargetX8632::lowerAlloca(const InstAlloca *Inst) {
+  IsEbpBasedFrame = true;
+  // TODO(sehr,stichnot): align allocated memory, keep stack aligned, minimize
+  // the number of adjustments of esp, etc.
+  Variable *esp = getPhysicalRegister(Reg_esp);
+  Operand *TotalSize = legalize(Inst->getSizeInBytes());
+  Variable *Dest = Inst->getDest();
+  _sub(esp, TotalSize);
+  _mov(Dest, esp);
+}
+
+void TargetX8632::lowerArithmetic(const InstArithmetic *Inst) {
+  Variable *Dest = Inst->getDest();
+  Operand *Src0 = legalize(Inst->getSrc(0));
+  Operand *Src1 = legalize(Inst->getSrc(1));
+  if (Dest->getType() == IceType_i64) {
+    Variable *DestLo = llvm::cast<Variable>(loOperand(Dest));
+    Variable *DestHi = llvm::cast<Variable>(hiOperand(Dest));
+    Operand *Src0Lo = loOperand(Src0);
+    Operand *Src0Hi = hiOperand(Src0);
+    Operand *Src1Lo = loOperand(Src1);
+    Operand *Src1Hi = hiOperand(Src1);
+    Variable *T_Lo = NULL, *T_Hi = NULL;
+    switch (Inst->getOp()) {
+    case InstArithmetic::Add:
+      _mov(T_Lo, Src0Lo);
+      _add(T_Lo, Src1Lo);
+      _mov(DestLo, T_Lo);
+      _mov(T_Hi, Src0Hi);
+      _adc(T_Hi, Src1Hi);
+      _mov(DestHi, T_Hi);
+      break;
+    case InstArithmetic::And:
+      _mov(T_Lo, Src0Lo);
+      _and(T_Lo, Src1Lo);
+      _mov(DestLo, T_Lo);
+      _mov(T_Hi, Src0Hi);
+      _and(T_Hi, Src1Hi);
+      _mov(DestHi, T_Hi);
+      break;
+    case InstArithmetic::Or:
+      _mov(T_Lo, Src0Lo);
+      _or(T_Lo, Src1Lo);
+      _mov(DestLo, T_Lo);
+      _mov(T_Hi, Src0Hi);
+      _or(T_Hi, Src1Hi);
+      _mov(DestHi, T_Hi);
+      break;
+    case InstArithmetic::Xor:
+      _mov(T_Lo, Src0Lo);
+      _xor(T_Lo, Src1Lo);
+      _mov(DestLo, T_Lo);
+      _mov(T_Hi, Src0Hi);
+      _xor(T_Hi, Src1Hi);
+      _mov(DestHi, T_Hi);
+      break;
+    case InstArithmetic::Sub:
+      _mov(T_Lo, Src0Lo);
+      _sub(T_Lo, Src1Lo);
+      _mov(DestLo, T_Lo);
+      _mov(T_Hi, Src0Hi);
+      _sbb(T_Hi, Src1Hi);
+      _mov(DestHi, T_Hi);
+      break;
+    case InstArithmetic::Mul: {
+      Variable *T_1 = NULL, *T_2 = NULL, *T_3 = NULL;
+      Variable *T_4Lo = makeReg(IceType_i32, Reg_eax);
+      Variable *T_4Hi = makeReg(IceType_i32, Reg_edx);
+      // gcc does the following:
+      // a=b*c ==>
+      //   t1 = b.hi; t1 *=(imul) c.lo
+      //   t2 = c.hi; t2 *=(imul) b.lo
+      //   t3:eax = b.lo
+      //   t4.hi:edx,t4.lo:eax = t3:eax *(mul) c.lo
+      //   a.lo = t4.lo
+      //   t4.hi += t1
+      //   t4.hi += t2
+      //   a.hi = t4.hi
+      _mov(T_1, Src0Hi);
+      _imul(T_1, Src1Lo);
+      _mov(T_2, Src1Hi);
+      _imul(T_2, Src0Lo);
+      _mov(T_3, Src0Lo, Reg_eax);
+      _mul(T_4Lo, T_3, Src1Lo);
+      // The mul instruction produces two dest variables, edx:eax.  We
+      // create a fake definition of edx to account for this.
+      Context.insert(InstFakeDef::create(Func, T_4Hi, T_4Lo));
+      _mov(DestLo, T_4Lo);
+      _add(T_4Hi, T_1);
+      _add(T_4Hi, T_2);
+      _mov(DestHi, T_4Hi);
+    } break;
+    case InstArithmetic::Shl: {
+      // TODO: Refactor the similarities between Shl, Lshr, and Ashr.
+      // gcc does the following:
+      // a=b<<c ==>
+      //   t1:ecx = c.lo & 0xff
+      //   t2 = b.lo
+      //   t3 = b.hi
+      //   t3 = shld t3, t2, t1
+      //   t2 = shl t2, t1
+      //   test t1, 0x20
+      //   je L1
+      //   use(t3)
+      //   t3 = t2
+      //   t2 = 0
+      // L1:
+      //   a.lo = t2
+      //   a.hi = t3
+      Variable *T_1 = NULL, *T_2 = NULL, *T_3 = NULL;
+      Constant *BitTest = Ctx->getConstantInt(IceType_i32, 0x20);
+      Constant *Zero = Ctx->getConstantInt(IceType_i32, 0);
+      InstX8632Label *Label = InstX8632Label::create(Func, this);
+      _mov(T_1, Src1Lo, Reg_ecx);
+      _mov(T_2, Src0Lo);
+      _mov(T_3, Src0Hi);
+      _shld(T_3, T_2, T_1);
+      _shl(T_2, T_1);
+      _test(T_1, BitTest);
+      _br(InstX8632Br::Br_e, Label);
+      // Because of the intra-block control flow, we need to fake a use
+      // of T_3 to prevent its earlier definition from being dead-code
+      // eliminated in the presence of its later definition.
+      Context.insert(InstFakeUse::create(Func, T_3));
+      _mov(T_3, T_2);
+      _mov(T_2, Zero);
+      Context.insert(Label);
+      _mov(DestLo, T_2);
+      _mov(DestHi, T_3);
+    } break;
+    case InstArithmetic::Lshr: {
+      // a=b>>c (unsigned) ==>
+      //   t1:ecx = c.lo & 0xff
+      //   t2 = b.lo
+      //   t3 = b.hi
+      //   t2 = shrd t2, t3, t1
+      //   t3 = shr t3, t1
+      //   test t1, 0x20
+      //   je L1
+      //   use(t2)
+      //   t2 = t3
+      //   t3 = 0
+      // L1:
+      //   a.lo = t2
+      //   a.hi = t3
+      Variable *T_1 = NULL, *T_2 = NULL, *T_3 = NULL;
+      Constant *BitTest = Ctx->getConstantInt(IceType_i32, 0x20);
+      Constant *Zero = Ctx->getConstantInt(IceType_i32, 0);
+      InstX8632Label *Label = InstX8632Label::create(Func, this);
+      _mov(T_1, Src1Lo, Reg_ecx);
+      _mov(T_2, Src0Lo);
+      _mov(T_3, Src0Hi);
+      _shrd(T_2, T_3, T_1);
+      _shr(T_3, T_1);
+      _test(T_1, BitTest);
+      _br(InstX8632Br::Br_e, Label);
+      // Because of the intra-block control flow, we need to fake a use
+      // of T_3 to prevent its earlier definition from being dead-code
+      // eliminated in the presence of its later definition.
+      Context.insert(InstFakeUse::create(Func, T_2));
+      _mov(T_2, T_3);
+      _mov(T_3, Zero);
+      Context.insert(Label);
+      _mov(DestLo, T_2);
+      _mov(DestHi, T_3);
+    } break;
+    case InstArithmetic::Ashr: {
+      // a=b>>c (signed) ==>
+      //   t1:ecx = c.lo & 0xff
+      //   t2 = b.lo
+      //   t3 = b.hi
+      //   t2 = shrd t2, t3, t1
+      //   t3 = sar t3, t1
+      //   test t1, 0x20
+      //   je L1
+      //   use(t2)
+      //   t2 = t3
+      //   t3 = sar t3, 0x1f
+      // L1:
+      //   a.lo = t2
+      //   a.hi = t3
+      Variable *T_1 = NULL, *T_2 = NULL, *T_3 = NULL;
+      Constant *BitTest = Ctx->getConstantInt(IceType_i32, 0x20);
+      Constant *SignExtend = Ctx->getConstantInt(IceType_i32, 0x1f);
+      InstX8632Label *Label = InstX8632Label::create(Func, this);
+      _mov(T_1, Src1Lo, Reg_ecx);
+      _mov(T_2, Src0Lo);
+      _mov(T_3, Src0Hi);
+      _shrd(T_2, T_3, T_1);
+      _sar(T_3, T_1);
+      _test(T_1, BitTest);
+      _br(InstX8632Br::Br_e, Label);
+      // Because of the intra-block control flow, we need to fake a use
+      // of T_3 to prevent its earlier definition from being dead-code
+      // eliminated in the presence of its later definition.
+      Context.insert(InstFakeUse::create(Func, T_2));
+      _mov(T_2, T_3);
+      _sar(T_3, SignExtend);
+      Context.insert(Label);
+      _mov(DestLo, T_2);
+      _mov(DestHi, T_3);
+    } break;
+    case InstArithmetic::Udiv: {
+      const SizeT MaxSrcs = 2;
+      InstCall *Call = makeHelperCall("__udivdi3", Dest, MaxSrcs);
+      Call->addArg(Inst->getSrc(0));
+      Call->addArg(Inst->getSrc(1));
+      lowerCall(Call);
+    } break;
+    case InstArithmetic::Sdiv: {
+      const SizeT MaxSrcs = 2;
+      InstCall *Call = makeHelperCall("__divdi3", Dest, MaxSrcs);
+      Call->addArg(Inst->getSrc(0));
+      Call->addArg(Inst->getSrc(1));
+      lowerCall(Call);
+    } break;
+    case InstArithmetic::Urem: {
+      const SizeT MaxSrcs = 2;
+      InstCall *Call = makeHelperCall("__umoddi3", Dest, MaxSrcs);
+      Call->addArg(Inst->getSrc(0));
+      Call->addArg(Inst->getSrc(1));
+      lowerCall(Call);
+    } break;
+    case InstArithmetic::Srem: {
+      const SizeT MaxSrcs = 2;
+      InstCall *Call = makeHelperCall("__moddi3", Dest, MaxSrcs);
+      Call->addArg(Inst->getSrc(0));
+      Call->addArg(Inst->getSrc(1));
+      lowerCall(Call);
+    } break;
+    case InstArithmetic::Fadd:
+    case InstArithmetic::Fsub:
+    case InstArithmetic::Fmul:
+    case InstArithmetic::Fdiv:
+    case InstArithmetic::Frem:
+      llvm_unreachable("FP instruction with i64 type");
+      break;
+    }
+  } else { // Dest->getType() != IceType_i64
+    Variable *T_edx = NULL;
+    Variable *T = NULL;
+    switch (Inst->getOp()) {
+    case InstArithmetic::Add:
+      _mov(T, Src0);
+      _add(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::And:
+      _mov(T, Src0);
+      _and(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Or:
+      _mov(T, Src0);
+      _or(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Xor:
+      _mov(T, Src0);
+      _xor(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Sub:
+      _mov(T, Src0);
+      _sub(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Mul:
+      // TODO: Optimize for llvm::isa<Constant>(Src1)
+      // TODO: Strength-reduce multiplications by a constant,
+      // particularly -1 and powers of 2.  Advanced: use lea to
+      // multiply by 3, 5, 9.
+      //
+      // The 8-bit version of imul only allows the form "imul r/m8"
+      // where T must be in eax.
+      if (Dest->getType() == IceType_i8)
+        _mov(T, Src0, Reg_eax);
+      else
+        _mov(T, Src0);
+      _imul(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Shl:
+      _mov(T, Src0);
+      if (!llvm::isa<Constant>(Src1))
+        Src1 = legalizeToVar(Src1, false, Reg_ecx);
+      _shl(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Lshr:
+      _mov(T, Src0);
+      if (!llvm::isa<Constant>(Src1))
+        Src1 = legalizeToVar(Src1, false, Reg_ecx);
+      _shr(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Ashr:
+      _mov(T, Src0);
+      if (!llvm::isa<Constant>(Src1))
+        Src1 = legalizeToVar(Src1, false, Reg_ecx);
+      _sar(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Udiv:
+      if (Dest->getType() == IceType_i8) {
+        Variable *T_ah = NULL;
+        Constant *Zero = Ctx->getConstantInt(IceType_i8, 0);
+        _mov(T, Src0, Reg_eax);
+        _mov(T_ah, Zero, Reg_ah);
+        _div(T, Src1, T_ah);
+        _mov(Dest, T);
+      } else {
+        Constant *Zero = Ctx->getConstantInt(IceType_i32, 0);
+        _mov(T, Src0, Reg_eax);
+        _mov(T_edx, Zero, Reg_edx);
+        _div(T, Src1, T_edx);
+        _mov(Dest, T);
+      }
+      break;
+    case InstArithmetic::Sdiv:
+      T_edx = makeReg(IceType_i32, Reg_edx);
+      _mov(T, Src0, Reg_eax);
+      _cdq(T_edx, T);
+      _idiv(T, Src1, T_edx);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Urem:
+      if (Dest->getType() == IceType_i8) {
+        Variable *T_ah = NULL;
+        Constant *Zero = Ctx->getConstantInt(IceType_i8, 0);
+        _mov(T, Src0, Reg_eax);
+        _mov(T_ah, Zero, Reg_ah);
+        _div(T_ah, Src1, T);
+        _mov(Dest, T_ah);
+      } else {
+        Constant *Zero = Ctx->getConstantInt(IceType_i32, 0);
+        _mov(T_edx, Zero, Reg_edx);
+        _mov(T, Src0, Reg_eax);
+        _div(T_edx, Src1, T);
+        _mov(Dest, T_edx);
+      }
+      break;
+    case InstArithmetic::Srem:
+      T_edx = makeReg(IceType_i32, Reg_edx);
+      _mov(T, Src0, Reg_eax);
+      _cdq(T_edx, T);
+      _idiv(T_edx, Src1, T);
+      _mov(Dest, T_edx);
+      break;
+    case InstArithmetic::Fadd:
+      _mov(T, Src0);
+      _addss(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Fsub:
+      _mov(T, Src0);
+      _subss(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Fmul:
+      _mov(T, Src0);
+      _mulss(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Fdiv:
+      _mov(T, Src0);
+      _divss(T, Src1);
+      _mov(Dest, T);
+      break;
+    case InstArithmetic::Frem: {
+      const SizeT MaxSrcs = 2;
+      Type Ty = Dest->getType();
+      InstCall *Call =
+          makeHelperCall(Ty == IceType_f32 ? "fmodf" : "fmod", Dest, MaxSrcs);
+      Call->addArg(Src0);
+      Call->addArg(Src1);
+      return lowerCall(Call);
+    } break;
+    }
+  }
+}
+
+void TargetX8632::lowerAssign(const InstAssign *Inst) {
+  Variable *Dest = Inst->getDest();
+  Operand *Src0 = Inst->getSrc(0);
+  assert(Dest->getType() == Src0->getType());
+  if (Dest->getType() == IceType_i64) {
+    Src0 = legalize(Src0);
+    Operand *Src0Lo = loOperand(Src0);
+    Operand *Src0Hi = hiOperand(Src0);
+    Variable *DestLo = llvm::cast<Variable>(loOperand(Dest));
+    Variable *DestHi = llvm::cast<Variable>(hiOperand(Dest));
+    Variable *T_Lo = NULL, *T_Hi = NULL;
+    _mov(T_Lo, Src0Lo);
+    _mov(DestLo, T_Lo);
+    _mov(T_Hi, Src0Hi);
+    _mov(DestHi, T_Hi);
+  } else {
+    const bool AllowOverlap = true;
+    // RI is either a physical register or an immediate.
+    Operand *RI = legalize(Src0, Legal_Reg | Legal_Imm, AllowOverlap);
+    _mov(Dest, RI);
+  }
+}
+
+void TargetX8632::lowerBr(const InstBr *Inst) {
+  if (Inst->isUnconditional()) {
+    _br(Inst->getTargetUnconditional());
+  } else {
+    Operand *Src0 = legalize(Inst->getCondition());
+    Constant *Zero = Ctx->getConstantInt(IceType_i32, 0);
+    _cmp(Src0, Zero);
+    _br(InstX8632Br::Br_ne, Inst->getTargetTrue(), Inst->getTargetFalse());
+  }
+}
+
+void TargetX8632::lowerCall(const InstCall *Instr) {
+  // Generate a sequence of push instructions, pushing right to left,
+  // keeping track of stack offsets in case a push involves a stack
+  // operand and we are using an esp-based frame.
+  uint32_t StackOffset = 0;
+  // TODO: If for some reason the call instruction gets dead-code
+  // eliminated after lowering, we would need to ensure that the
+  // pre-call push instructions and the post-call esp adjustment get
+  // eliminated as well.
+  for (SizeT NumArgs = Instr->getNumArgs(), i = 0; i < NumArgs; ++i) {
+    Operand *Arg = legalize(Instr->getArg(NumArgs - i - 1));
+    if (Arg->getType() == IceType_i64) {
+      _push(hiOperand(Arg));
+      _push(loOperand(Arg));
+    } else if (Arg->getType() == IceType_f64) {
+      // If the Arg turns out to be a memory operand, we need to push
+      // 8 bytes, which requires two push instructions.  This ends up
+      // being somewhat clumsy in the current IR, so we use a
+      // workaround.  Force the operand into a (xmm) register, and
+      // then push the register.  An xmm register push is actually not
+      // possible in x86, but the Push instruction emitter handles
+      // this by decrementing the stack pointer and directly writing
+      // the xmm register value.
+      Variable *T = NULL;
+      _mov(T, Arg);
+      _push(T);
+    } else {
+      _push(Arg);
+    }
+    StackOffset += typeWidthInBytesOnStack(Arg->getType());
+  }
+  // Generate the call instruction.  Assign its result to a temporary
+  // with high register allocation weight.
+  Variable *Dest = Instr->getDest();
+  Variable *eax = NULL; // doubles as RegLo as necessary
+  Variable *edx = NULL;
+  if (Dest) {
+    switch (Dest->getType()) {
+    case IceType_NUM:
+      llvm_unreachable("Invalid Call dest type");
+      break;
+    case IceType_void:
+      break;
+    case IceType_i1:
+    case IceType_i8:
+    case IceType_i16:
+    case IceType_i32:
+      eax = makeReg(Dest->getType(), Reg_eax);
+      break;
+    case IceType_i64:
+      eax = makeReg(IceType_i32, Reg_eax);
+      edx = makeReg(IceType_i32, Reg_edx);
+      break;
+    case IceType_f32:
+    case IceType_f64:
+      // Leave eax==edx==NULL, and capture the result with the fstp
+      // instruction.
+      break;
+    }
+  }
+  Operand *CallTarget = legalize(Instr->getCallTarget());
+  Inst *NewCall = InstX8632Call::create(Func, eax, CallTarget);
+  Context.insert(NewCall);
+  if (edx)
+    Context.insert(InstFakeDef::create(Func, edx));
+
+  // Add the appropriate offset to esp.
+  if (StackOffset) {
+    Variable *esp = Func->getTarget()->getPhysicalRegister(Reg_esp);
+    _add(esp, Ctx->getConstantInt(IceType_i32, StackOffset));
+  }
+
+  // Insert a register-kill pseudo instruction.
+  VarList KilledRegs;
+  for (SizeT i = 0; i < ScratchRegs.size(); ++i) {
+    if (ScratchRegs[i])
+      KilledRegs.push_back(Func->getTarget()->getPhysicalRegister(i));
+  }
+  Context.insert(InstFakeKill::create(Func, KilledRegs, NewCall));
+
+  // Generate a FakeUse to keep the call live if necessary.
+  if (Instr->hasSideEffects() && eax) {
+    Inst *FakeUse = InstFakeUse::create(Func, eax);
+    Context.insert(FakeUse);
+  }
+
+  // Generate Dest=eax assignment.
+  if (Dest && eax) {
+    if (edx) {
+      split64(Dest);
+      Variable *DestLo = Dest->getLo();
+      Variable *DestHi = Dest->getHi();
+      DestLo->setPreferredRegister(eax, false);
+      DestHi->setPreferredRegister(edx, false);
+      _mov(DestLo, eax);
+      _mov(DestHi, edx);
+    } else {
+      Dest->setPreferredRegister(eax, false);
+      _mov(Dest, eax);
+    }
+  }
+
+  // Special treatment for an FP function which returns its result in
+  // st(0).
+  if (Dest &&
+      (Dest->getType() == IceType_f32 || Dest->getType() == IceType_f64)) {
+    _fstp(Dest);
+    // If Dest ends up being a physical xmm register, the fstp emit
+    // code will route st(0) through a temporary stack slot.
+  }
+}
+
+void TargetX8632::lowerCast(const InstCast *Inst) {
+  // a = cast(b) ==> t=cast(b); a=t; (link t->b, link a->t, no overlap)
+  InstCast::OpKind CastKind = Inst->getCastKind();
+  Variable *Dest = Inst->getDest();
+  // Src0RM is the source operand legalized to physical register or memory, but
+  // not immediate, since the relevant x86 native instructions don't allow an
+  // immediate operand.  If the operand is an immediate, we could consider
+  // computing the strength-reduced result at translation time, but we're
+  // unlikely to see something like that in the bitcode that the optimizer
+  // wouldn't have already taken care of.
+  Operand *Src0RM = legalize(Inst->getSrc(0), Legal_Reg | Legal_Mem, true);
+  switch (CastKind) {
+  default:
+    Func->setError("Cast type not supported");
+    return;
+  case InstCast::Sext:
+    if (Dest->getType() == IceType_i64) {
+      // t1=movsx src; t2=t1; t2=sar t2, 31; dst.lo=t1; dst.hi=t2
+      Variable *DestLo = llvm::cast<Variable>(loOperand(Dest));
+      Variable *DestHi = llvm::cast<Variable>(hiOperand(Dest));
+      Variable *T_Lo = makeReg(DestLo->getType());
+      if (Src0RM->getType() == IceType_i32)
+        _mov(T_Lo, Src0RM);
+      else
+        _movsx(T_Lo, Src0RM);
+      _mov(DestLo, T_Lo);
+      Variable *T_Hi = NULL;
+      Constant *Shift = Ctx->getConstantInt(IceType_i32, 31);
+      _mov(T_Hi, T_Lo);
+      _sar(T_Hi, Shift);
+      _mov(DestHi, T_Hi);
+    } else {
+      // TODO: Sign-extend an i1 via "shl reg, 31; sar reg, 31", and
+      // also copy to the high operand of a 64-bit variable.
+      // t1 = movsx src; dst = t1
+      Variable *T = makeReg(Dest->getType());
+      _movsx(T, Src0RM);
+      _mov(Dest, T);
+    }
+    break;
+  case InstCast::Zext:
+    if (Dest->getType() == IceType_i64) {
+      // t1=movzx src; dst.lo=t1; dst.hi=0
+      Constant *Zero = Ctx->getConstantInt(IceType_i32, 0);
+      Variable *DestLo = llvm::cast<Variable>(loOperand(Dest));
+      Variable *DestHi = llvm::cast<Variable>(hiOperand(Dest));
+      Variable *Tmp = makeReg(DestLo->getType());
+      if (Src0RM->getType() == IceType_i32)
+        _mov(Tmp, Src0RM);
+      else
+        _movzx(Tmp, Src0RM);
+      _mov(DestLo, Tmp);
+      _mov(DestHi, Zero);
+    } else if (Src0RM->getType() == IceType_i1) {
+      // t = Src0RM; t &= 1; Dest = t
+      Operand *One = Ctx->getConstantInt(IceType_i32, 1);
+      Variable *T = makeReg(IceType_i32);
+      _movzx(T, Src0RM);
+      _and(T, One);
+      _mov(Dest, T);
+    } else {
+      // t1 = movzx src; dst = t1
+      Variable *T = makeReg(Dest->getType());
+      _movzx(T, Src0RM);
+      _mov(Dest, T);
+    }
+    break;
+  case InstCast::Trunc: {
+    if (Src0RM->getType() == IceType_i64)
+      Src0RM = loOperand(Src0RM);
+    // t1 = trunc Src0RM; Dest = t1
+    Variable *T = NULL;
+    _mov(T, Src0RM);
+    _mov(Dest, T);
+    break;
+  }
+  case InstCast::Fptrunc:
+  case InstCast::Fpext: {
+    // t1 = cvt Src0RM; Dest = t1
+    Variable *T = makeReg(Dest->getType());
+    _cvt(T, Src0RM);
+    _mov(Dest, T);
+    break;
+  }
+  case InstCast::Fptosi:
+    if (Dest->getType() == IceType_i64) {
+      // Use a helper for converting floating-point values to 64-bit
+      // integers.  SSE2 appears to have no way to convert from xmm
+      // registers to something like the edx:eax register pair, and
+      // gcc and clang both want to use x87 instructions complete with
+      // temporary manipulation of the status word.  This helper is
+      // not needed for x86-64.
+      split64(Dest);
+      const SizeT MaxSrcs = 1;
+      Type SrcType = Inst->getSrc(0)->getType();
+      InstCall *Call = makeHelperCall(
+          SrcType == IceType_f32 ? "cvtftosi64" : "cvtdtosi64", Dest, MaxSrcs);
+      // TODO: Call the correct compiler-rt helper function.
+      Call->addArg(Inst->getSrc(0));
+      lowerCall(Call);
+    } else {
+      // t1.i32 = cvt Src0RM; t2.dest_type = t1; Dest = t2.dest_type
+      Variable *T_1 = makeReg(IceType_i32);
+      Variable *T_2 = makeReg(Dest->getType());
+      _cvt(T_1, Src0RM);
+      _mov(T_2, T_1); // T_1 and T_2 may have different integer types
+      _mov(Dest, T_2);
+      T_2->setPreferredRegister(T_1, true);
+    }
+    break;
+  case InstCast::Fptoui:
+    if (Dest->getType() == IceType_i64 || Dest->getType() == IceType_i32) {
+      // Use a helper for both x86-32 and x86-64.
+      split64(Dest);
+      const SizeT MaxSrcs = 1;
+      Type DestType = Dest->getType();
+      Type SrcType = Src0RM->getType();
+      IceString DstSubstring = (DestType == IceType_i64 ? "64" : "32");
+      IceString SrcSubstring = (SrcType == IceType_f32 ? "f" : "d");
+      // Possibilities are cvtftoui32, cvtdtoui32, cvtftoui64, cvtdtoui64
+      IceString TargetString = "cvt" + SrcSubstring + "toui" + DstSubstring;
+      // TODO: Call the correct compiler-rt helper function.
+      InstCall *Call = makeHelperCall(TargetString, Dest, MaxSrcs);
+      Call->addArg(Inst->getSrc(0));
+      lowerCall(Call);
+      return;
+    } else {
+      // t1.i32 = cvt Src0RM; t2.dest_type = t1; Dest = t2.dest_type
+      Variable *T_1 = makeReg(IceType_i32);
+      Variable *T_2 = makeReg(Dest->getType());
+      _cvt(T_1, Src0RM);
+      _mov(T_2, T_1); // T_1 and T_2 may have different integer types
+      _mov(Dest, T_2);
+      T_2->setPreferredRegister(T_1, true);
+    }
+    break;
+  case InstCast::Sitofp:
+    if (Src0RM->getType() == IceType_i64) {
+      // Use a helper for x86-32.
+      const SizeT MaxSrcs = 1;
+      Type DestType = Dest->getType();
+      InstCall *Call = makeHelperCall(
+          DestType == IceType_f32 ? "cvtsi64tof" : "cvtsi64tod", Dest, MaxSrcs);
+      // TODO: Call the correct compiler-rt helper function.
+      Call->addArg(Inst->getSrc(0));
+      lowerCall(Call);
+      return;
+    } else {
+      // Sign-extend the operand.
+      // t1.i32 = movsx Src0RM; t2 = Cvt t1.i32; Dest = t2
+      Variable *T_1 = makeReg(IceType_i32);
+      Variable *T_2 = makeReg(Dest->getType());
+      if (Src0RM->getType() == IceType_i32)
+        _mov(T_1, Src0RM);
+      else
+        _movsx(T_1, Src0RM);
+      _cvt(T_2, T_1);
+      _mov(Dest, T_2);
+    }
+    break;
+  case InstCast::Uitofp:
+    if (Src0RM->getType() == IceType_i64 || Src0RM->getType() == IceType_i32) {
+      // Use a helper for x86-32 and x86-64.  Also use a helper for
+      // i32 on x86-32.
+      const SizeT MaxSrcs = 1;
+      Type DestType = Dest->getType();
+      IceString SrcSubstring = (Src0RM->getType() == IceType_i64 ? "64" : "32");
+      IceString DstSubstring = (DestType == IceType_f32 ? "f" : "d");
+      // Possibilities are cvtui32tof, cvtui32tod, cvtui64tof, cvtui64tod
+      IceString TargetString = "cvtui" + SrcSubstring + "to" + DstSubstring;
+      // TODO: Call the correct compiler-rt helper function.
+      InstCall *Call = makeHelperCall(TargetString, Dest, MaxSrcs);
+      Call->addArg(Inst->getSrc(0));
+      lowerCall(Call);
+      return;
+    } else {
+      // Zero-extend the operand.
+      // t1.i32 = movzx Src0RM; t2 = Cvt t1.i32; Dest = t2
+      Variable *T_1 = makeReg(IceType_i32);
+      Variable *T_2 = makeReg(Dest->getType());
+      if (Src0RM->getType() == IceType_i32)
+        _mov(T_1, Src0RM);
+      else
+        _movzx(T_1, Src0RM);
+      _cvt(T_2, T_1);
+      _mov(Dest, T_2);
+    }
+    break;
+  case InstCast::Bitcast:
+    if (Dest->getType() == Src0RM->getType()) {
+      InstAssign *Assign = InstAssign::create(Func, Dest, Src0RM);
+      lowerAssign(Assign);
+      return;
+    }
+    switch (Dest->getType()) {
+    default:
+      llvm_unreachable("Unexpected Bitcast dest type");
+    case IceType_i32:
+    case IceType_f32: {
+      Type DestType = Dest->getType();
+      Type SrcType = Src0RM->getType();
+      assert((DestType == IceType_i32 && SrcType == IceType_f32) ||
+             (DestType == IceType_f32 && SrcType == IceType_i32));
+      // a.i32 = bitcast b.f32 ==>
+      //   t.f32 = b.f32
+      //   s.f32 = spill t.f32
+      //   a.i32 = s.f32
+      Variable *T = NULL;
+      // TODO: Should be able to force a spill setup by calling legalize() with
+      // Legal_Mem and not Legal_Reg or Legal_Imm.
+      Variable *Spill = Func->makeVariable(SrcType, Context.getNode());
+      Spill->setWeight(RegWeight::Zero);
+      Spill->setPreferredRegister(Dest, true);
+      _mov(T, Src0RM);
+      _mov(Spill, T);
+      _mov(Dest, Spill);
+    } break;
+    case IceType_i64: {
+      assert(Src0RM->getType() == IceType_f64);
+      // a.i64 = bitcast b.f64 ==>
+      //   s.f64 = spill b.f64
+      //   t_lo.i32 = lo(s.f64)
+      //   a_lo.i32 = t_lo.i32
+      //   t_hi.i32 = hi(s.f64)
+      //   a_hi.i32 = t_hi.i32
+      Variable *Spill = Func->makeVariable(IceType_f64, Context.getNode());
+      Spill->setWeight(RegWeight::Zero);
+      Spill->setPreferredRegister(llvm::dyn_cast<Variable>(Src0RM), true);
+      _mov(Spill, Src0RM);
+
+      Variable *DestLo = llvm::cast<Variable>(loOperand(Dest));
+      Variable *DestHi = llvm::cast<Variable>(hiOperand(Dest));
+      Variable *T_Lo = makeReg(IceType_i32);
+      Variable *T_Hi = makeReg(IceType_i32);
+      VariableSplit *SpillLo =
+          VariableSplit::create(Func, Spill, VariableSplit::Low);
+      VariableSplit *SpillHi =
+          VariableSplit::create(Func, Spill, VariableSplit::High);
+
+      _mov(T_Lo, SpillLo);
+      _mov(DestLo, T_Lo);
+      _mov(T_Hi, SpillHi);
+      _mov(DestHi, T_Hi);
+    } break;
+    case IceType_f64: {
+      assert(Src0RM->getType() == IceType_i64);
+      // a.f64 = bitcast b.i64 ==>
+      //   t_lo.i32 = b_lo.i32
+      //   lo(s.f64) = t_lo.i32
+      //   FakeUse(s.f64)
+      //   t_hi.i32 = b_hi.i32
+      //   hi(s.f64) = t_hi.i32
+      //   a.f64 = s.f64
+      Variable *Spill = Func->makeVariable(IceType_f64, Context.getNode());
+      Spill->setWeight(RegWeight::Zero);
+      Spill->setPreferredRegister(Dest, true);
+
+      Context.insert(InstFakeDef::create(Func, Spill));
+
+      Variable *T_Lo = NULL, *T_Hi = NULL;
+      VariableSplit *SpillLo =
+          VariableSplit::create(Func, Spill, VariableSplit::Low);
+      VariableSplit *SpillHi =
+          VariableSplit::create(Func, Spill, VariableSplit::High);
+      _mov(T_Lo, loOperand(Src0RM));
+      _store(T_Lo, SpillLo);
+      _mov(T_Hi, hiOperand(Src0RM));
+      _store(T_Hi, SpillHi);
+      _mov(Dest, Spill);
+    } break;
+    }
+    break;
+  }
+}
+
+void TargetX8632::lowerFcmp(const InstFcmp *Inst) {
+  Operand *Src0 = Inst->getSrc(0);
+  Operand *Src1 = Inst->getSrc(1);
+  Variable *Dest = Inst->getDest();
+  // Lowering a = fcmp cond, b, c
+  //   ucomiss b, c       /* only if C1 != Br_None */
+  //                      /* but swap b,c order if SwapOperands==true */
+  //   mov a, <default>
+  //   j<C1> label        /* only if C1 != Br_None */
+  //   j<C2> label        /* only if C2 != Br_None */
+  //   FakeUse(a)         /* only if C1 != Br_None */
+  //   mov a, !<default>  /* only if C1 != Br_None */
+  //   label:             /* only if C1 != Br_None */
+  InstFcmp::FCond Condition = Inst->getCondition();
+  size_t Index = static_cast<size_t>(Condition);
+  assert(Index < TableFcmpSize);
+  if (TableFcmp[Index].SwapOperands) {
+    Operand *Tmp = Src0;
+    Src0 = Src1;
+    Src1 = Tmp;
+  }
+  bool HasC1 = (TableFcmp[Index].C1 != InstX8632Br::Br_None);
+  bool HasC2 = (TableFcmp[Index].C2 != InstX8632Br::Br_None);
+  if (HasC1) {
+    Src0 = legalize(Src0);
+    Operand *Src1RM = legalize(Src1, Legal_Reg | Legal_Mem);
+    Variable *T = NULL;
+    _mov(T, Src0);
+    _ucomiss(T, Src1RM);
+  }
+  Constant *Default =
+      Ctx->getConstantInt(IceType_i32, TableFcmp[Index].Default);
+  _mov(Dest, Default);
+  if (HasC1) {
+    InstX8632Label *Label = InstX8632Label::create(Func, this);
+    _br(TableFcmp[Index].C1, Label);
+    if (HasC2) {
+      _br(TableFcmp[Index].C2, Label);
+    }
+    Context.insert(InstFakeUse::create(Func, Dest));
+    Constant *NonDefault =
+        Ctx->getConstantInt(IceType_i32, !TableFcmp[Index].Default);
+    _mov(Dest, NonDefault);
+    Context.insert(Label);
+  }
+}
+
+void TargetX8632::lowerIcmp(const InstIcmp *Inst) {
+  Operand *Src0 = legalize(Inst->getSrc(0));
+  Operand *Src1 = legalize(Inst->getSrc(1));
+  Variable *Dest = Inst->getDest();
+
+  // a=icmp cond, b, c ==> cmp b,c; a=1; br cond,L1; FakeUse(a); a=0; L1:
+  Constant *Zero = Ctx->getConstantInt(IceType_i32, 0);
+  Constant *One = Ctx->getConstantInt(IceType_i32, 1);
+  if (Src0->getType() == IceType_i64) {
+    InstIcmp::ICond Condition = Inst->getCondition();
+    size_t Index = static_cast<size_t>(Condition);
+    assert(Index < TableIcmp64Size);
+    Operand *Src1LoRI = legalize(loOperand(Src1), Legal_Reg | Legal_Imm);
+    Operand *Src1HiRI = legalize(hiOperand(Src1), Legal_Reg | Legal_Imm);
+    if (Condition == InstIcmp::Eq || Condition == InstIcmp::Ne) {
+      InstX8632Label *Label = InstX8632Label::create(Func, this);
+      _mov(Dest, (Condition == InstIcmp::Eq ? Zero : One));
+      _cmp(loOperand(Src0), Src1LoRI);
+      _br(InstX8632Br::Br_ne, Label);
+      _cmp(hiOperand(Src0), Src1HiRI);
+      _br(InstX8632Br::Br_ne, Label);
+      Context.insert(InstFakeUse::create(Func, Dest));
+      _mov(Dest, (Condition == InstIcmp::Eq ? One : Zero));
+      Context.insert(Label);
+    } else {
+      InstX8632Label *LabelFalse = InstX8632Label::create(Func, this);
+      InstX8632Label *LabelTrue = InstX8632Label::create(Func, this);
+      _mov(Dest, One);
+      _cmp(hiOperand(Src0), Src1HiRI);
+      _br(TableIcmp64[Index].C1, LabelTrue);
+      _br(TableIcmp64[Index].C2, LabelFalse);
+      _cmp(loOperand(Src0), Src1LoRI);
+      _br(TableIcmp64[Index].C3, LabelTrue);
+      Context.insert(LabelFalse);
+      Context.insert(InstFakeUse::create(Func, Dest));
+      _mov(Dest, Zero);
+      Context.insert(LabelTrue);
+    }
+    return;
+  }
+
+  // If Src1 is an immediate, or known to be a physical register, we can
+  // allow Src0 to be a memory operand.  Otherwise, Src0 must be copied into
+  // a physical register.  (Actually, either Src0 or Src1 can be chosen for
+  // the physical register, but unfortunately we have to commit to one or
+  // the other before register allocation.)
+  bool IsSrc1ImmOrReg = false;
+  if (llvm::isa<Constant>(Src1)) {
+    IsSrc1ImmOrReg = true;
+  } else if (Variable *Var = llvm::dyn_cast<Variable>(Src1)) {
+    if (Var->hasReg())
+      IsSrc1ImmOrReg = true;
+  }
+
+  // cmp b, c
+  Operand *Src0New =
+      legalize(Src0, IsSrc1ImmOrReg ? Legal_All : Legal_Reg, true);
+  InstX8632Label *Label = InstX8632Label::create(Func, this);
+  _cmp(Src0New, Src1);
+  _mov(Dest, One);
+  _br(getIcmp32Mapping(Inst->getCondition()), Label);
+  Context.insert(InstFakeUse::create(Func, Dest));
+  _mov(Dest, Zero);
+  Context.insert(Label);
+}
+
+void TargetX8632::lowerLoad(const InstLoad *Inst) {
+  // A Load instruction can be treated the same as an Assign
+  // instruction, after the source operand is transformed into an
+  // OperandX8632Mem operand.  Note that the address mode
+  // optimization already creates an OperandX8632Mem operand, so it
+  // doesn't need another level of transformation.
+  Type Ty = Inst->getDest()->getType();
+  Operand *Src0 = Inst->getSourceAddress();
+  // Address mode optimization already creates an OperandX8632Mem
+  // operand, so it doesn't need another level of transformation.
+  if (!llvm::isa<OperandX8632Mem>(Src0)) {
+    Variable *Base = llvm::dyn_cast<Variable>(Src0);
+    Constant *Offset = llvm::dyn_cast<Constant>(Src0);
+    assert(Base || Offset);
+    Src0 = OperandX8632Mem::create(Func, Ty, Base, Offset);
+  }
+
+  InstAssign *Assign = InstAssign::create(Func, Inst->getDest(), Src0);
+  lowerAssign(Assign);
+}
+
+void TargetX8632::lowerPhi(const InstPhi * /*Inst*/) {
+  Func->setError("Phi found in regular instruction list");
+}
+
+void TargetX8632::lowerRet(const InstRet *Inst) {
+  Variable *Reg = NULL;
+  if (Inst->hasRetValue()) {
+    Operand *Src0 = legalize(Inst->getRetValue());
+    if (Src0->getType() == IceType_i64) {
+      Variable *eax = legalizeToVar(loOperand(Src0), false, Reg_eax);
+      Variable *edx = legalizeToVar(hiOperand(Src0), false, Reg_edx);
+      Reg = eax;
+      Context.insert(InstFakeUse::create(Func, edx));
+    } else if (Src0->getType() == IceType_f32 ||
+               Src0->getType() == IceType_f64) {
+      _fld(Src0);
+    } else {
+      _mov(Reg, Src0, Reg_eax);
+    }
+  }
+  _ret(Reg);
+  // Add a fake use of esp to make sure esp stays alive for the entire
+  // function.  Otherwise post-call esp adjustments get dead-code
+  // eliminated.  TODO: Are there more places where the fake use
+  // should be inserted?  E.g. "void f(int n){while(1) g(n);}" may not
+  // have a ret instruction.
+  Variable *esp = Func->getTarget()->getPhysicalRegister(Reg_esp);
+  Context.insert(InstFakeUse::create(Func, esp));
+}
+
+void TargetX8632::lowerSelect(const InstSelect *Inst) {
+  // a=d?b:c ==> cmp d,0; a=b; jne L1; FakeUse(a); a=c; L1:
+  Variable *Dest = Inst->getDest();
+  Operand *SrcT = Inst->getTrueOperand();
+  Operand *SrcF = Inst->getFalseOperand();
+  Operand *Condition = legalize(Inst->getCondition());
+  Constant *Zero = Ctx->getConstantInt(IceType_i32, 0);
+  InstX8632Label *Label = InstX8632Label::create(Func, this);
+
+  if (Dest->getType() == IceType_i64) {
+    Variable *DestLo = llvm::cast<Variable>(loOperand(Dest));
+    Variable *DestHi = llvm::cast<Variable>(hiOperand(Dest));
+    Operand *SrcLoRI = legalize(loOperand(SrcT), Legal_Reg | Legal_Imm, true);
+    Operand *SrcHiRI = legalize(hiOperand(SrcT), Legal_Reg | Legal_Imm, true);
+    _cmp(Condition, Zero);
+    _mov(DestLo, SrcLoRI);
+    _mov(DestHi, SrcHiRI);
+    _br(InstX8632Br::Br_ne, Label);
+    Context.insert(InstFakeUse::create(Func, DestLo));
+    Context.insert(InstFakeUse::create(Func, DestHi));
+    Operand *SrcFLo = loOperand(SrcF);
+    Operand *SrcFHi = hiOperand(SrcF);
+    SrcLoRI = legalize(SrcFLo, Legal_Reg | Legal_Imm, true);
+    SrcHiRI = legalize(SrcFHi, Legal_Reg | Legal_Imm, true);
+    _mov(DestLo, SrcLoRI);
+    _mov(DestHi, SrcHiRI);
+  } else {
+    _cmp(Condition, Zero);
+    SrcT = legalize(SrcT, Legal_Reg | Legal_Imm, true);
+    _mov(Dest, SrcT);
+    _br(InstX8632Br::Br_ne, Label);
+    Context.insert(InstFakeUse::create(Func, Dest));
+    SrcF = legalize(SrcF, Legal_Reg | Legal_Imm, true);
+    _mov(Dest, SrcF);
+  }
+
+  Context.insert(Label);
+}
+
+void TargetX8632::lowerStore(const InstStore *Inst) {
+  Operand *Value = Inst->getData();
+  Operand *Addr = Inst->getAddr();
+  OperandX8632Mem *NewAddr = llvm::dyn_cast<OperandX8632Mem>(Addr);
+  // Address mode optimization already creates an OperandX8632Mem
+  // operand, so it doesn't need another level of transformation.
+  if (!NewAddr) {
+    // The address will be either a constant (which represents a global
+    // variable) or a variable, so either the Base or Offset component
+    // of the OperandX8632Mem will be set.
+    Variable *Base = llvm::dyn_cast<Variable>(Addr);
+    Constant *Offset = llvm::dyn_cast<Constant>(Addr);
+    assert(Base || Offset);
+    NewAddr = OperandX8632Mem::create(Func, Value->getType(), Base, Offset);
+  }
+  NewAddr = llvm::cast<OperandX8632Mem>(legalize(NewAddr));
+
+  if (NewAddr->getType() == IceType_i64) {
+    Value = legalize(Value);
+    Operand *ValueHi = legalize(hiOperand(Value), Legal_Reg | Legal_Imm, true);
+    Operand *ValueLo = legalize(loOperand(Value), Legal_Reg | Legal_Imm, true);
+    _store(ValueHi, llvm::cast<OperandX8632Mem>(hiOperand(NewAddr)));
+    _store(ValueLo, llvm::cast<OperandX8632Mem>(loOperand(NewAddr)));
+  } else {
+    Value = legalize(Value, Legal_Reg | Legal_Imm, true);
+    _store(Value, NewAddr);
+  }
+}
+
+void TargetX8632::lowerSwitch(const InstSwitch *Inst) {
+  // This implements the most naive possible lowering.
+  // cmp a,val[0]; jeq label[0]; cmp a,val[1]; jeq label[1]; ... jmp default
+  Operand *Src0 = Inst->getComparison();
+  SizeT NumCases = Inst->getNumCases();
+  // OK, we'll be slightly less naive by forcing Src into a physical
+  // register if there are 2 or more uses.
+  if (NumCases >= 2)
+    Src0 = legalizeToVar(Src0, true);
+  else
+    Src0 = legalize(Src0, Legal_All, true);
+  for (SizeT I = 0; I < NumCases; ++I) {
+    Operand *Value = Ctx->getConstantInt(IceType_i32, Inst->getValue(I));
+    _cmp(Src0, Value);
+    _br(InstX8632Br::Br_e, Inst->getLabel(I));
+  }
+
+  _br(Inst->getLabelDefault());
+}
+
+void TargetX8632::lowerUnreachable(const InstUnreachable * /*Inst*/) {
+  const SizeT MaxSrcs = 0;
+  Variable *Dest = NULL;
+  InstCall *Call = makeHelperCall("ice_unreachable", Dest, MaxSrcs);
+  lowerCall(Call);
+}
+
+Operand *TargetX8632::legalize(Operand *From, LegalMask Allowed,
+                               bool AllowOverlap, int32_t RegNum) {
+  // Assert that a physical register is allowed.  To date, all calls
+  // to legalize() allow a physical register.  If a physical register
+  // needs to be explicitly disallowed, then new code will need to be
+  // written to force a spill.
+  assert(Allowed & Legal_Reg);
+  // If we're asking for a specific physical register, make sure we're
+  // not allowing any other operand kinds.  (This could be future
+  // work, e.g. allow the shl shift amount to be either an immediate
+  // or in ecx.)
+  assert(RegNum == Variable::NoRegister || Allowed == Legal_Reg);
+  if (OperandX8632Mem *Mem = llvm::dyn_cast<OperandX8632Mem>(From)) {
+    // Before doing anything with a Mem operand, we need to ensure
+    // that the Base and Index components are in physical registers.
+    Variable *Base = Mem->getBase();
+    Variable *Index = Mem->getIndex();
+    Variable *RegBase = NULL;
+    Variable *RegIndex = NULL;
+    if (Base) {
+      RegBase = legalizeToVar(Base, true);
+    }
+    if (Index) {
+      RegIndex = legalizeToVar(Index, true);
+    }
+    if (Base != RegBase || Index != RegIndex) {
+      From =
+          OperandX8632Mem::create(Func, Mem->getType(), RegBase,
+                                  Mem->getOffset(), RegIndex, Mem->getShift());
+    }
+
+    if (!(Allowed & Legal_Mem)) {
+      Variable *Reg = makeReg(From->getType(), RegNum);
+      _mov(Reg, From, RegNum);
+      From = Reg;
+    }
+    return From;
+  }
+  if (llvm::isa<Constant>(From)) {
+    if (!(Allowed & Legal_Imm)) {
+      Variable *Reg = makeReg(From->getType(), RegNum);
+      _mov(Reg, From);
+      From = Reg;
+    }
+    return From;
+  }
+  if (Variable *Var = llvm::dyn_cast<Variable>(From)) {
+    // We need a new physical register for the operand if:
+    //   Mem is not allowed and Var->getRegNum() is unknown, or
+    //   RegNum is required and Var->getRegNum() doesn't match.
+    if ((!(Allowed & Legal_Mem) && !Var->hasReg()) ||
+        (RegNum != Variable::NoRegister && RegNum != Var->getRegNum())) {
+      Variable *Reg = makeReg(From->getType(), RegNum);
+      if (RegNum == Variable::NoRegister) {
+        Reg->setPreferredRegister(Var, AllowOverlap);
+      }
+      _mov(Reg, From);
+      From = Reg;
+    }
+    return From;
+  }
+  llvm_unreachable("Unhandled operand kind in legalize()");
+  return From;
+}
+
+// Provide a trivial wrapper to legalize() for this common usage.
+Variable *TargetX8632::legalizeToVar(Operand *From, bool AllowOverlap,
+                                     int32_t RegNum) {
+  return llvm::cast<Variable>(legalize(From, Legal_Reg, AllowOverlap, RegNum));
+}
+
+Variable *TargetX8632::makeReg(Type Type, int32_t RegNum) {
+  Variable *Reg = Func->makeVariable(Type, Context.getNode());
+  if (RegNum == Variable::NoRegister)
+    Reg->setWeightInfinite();
+  else
+    Reg->setRegNum(RegNum);
+  return Reg;
+}
+
+void TargetX8632::postLower() {
+  if (Ctx->getOptLevel() != Opt_m1)
+    return;
+  // TODO: Avoid recomputing WhiteList every instruction.
+  llvm::SmallBitVector WhiteList = getRegisterSet(RegSet_All, RegSet_None);
+  // Make one pass to black-list pre-colored registers.  TODO: If
+  // there was some prior register allocation pass that made register
+  // assignments, those registers need to be black-listed here as
+  // well.
+  for (InstList::iterator I = Context.getCur(), E = Context.getEnd(); I != E;
+       ++I) {
+    const Inst *Inst = *I;
+    if (Inst->isDeleted())
+      continue;
+    if (llvm::isa<InstFakeKill>(Inst))
+      continue;
+    SizeT VarIndex = 0;
+    for (SizeT SrcNum = 0; SrcNum < Inst->getSrcSize(); ++SrcNum) {
+      Operand *Src = Inst->getSrc(SrcNum);
+      SizeT NumVars = Src->getNumVars();
+      for (SizeT J = 0; J < NumVars; ++J, ++VarIndex) {
+        const Variable *Var = Src->getVar(J);
+        if (!Var->hasReg())
+          continue;
+        WhiteList[Var->getRegNum()] = false;
+      }
+    }
+  }
+  // The second pass colors infinite-weight variables.
+  llvm::SmallBitVector AvailableRegisters = WhiteList;
+  for (InstList::iterator I = Context.getCur(), E = Context.getEnd(); I != E;
+       ++I) {
+    const Inst *Inst = *I;
+    if (Inst->isDeleted())
+      continue;
+    SizeT VarIndex = 0;
+    for (SizeT SrcNum = 0; SrcNum < Inst->getSrcSize(); ++SrcNum) {
+      Operand *Src = Inst->getSrc(SrcNum);
+      SizeT NumVars = Src->getNumVars();
+      for (SizeT J = 0; J < NumVars; ++J, ++VarIndex) {
+        Variable *Var = Src->getVar(J);
+        if (Var->hasReg())
+          continue;
+        if (!Var->getWeight().isInf())
+          continue;
+        llvm::SmallBitVector AvailableTypedRegisters =
+            AvailableRegisters & getRegisterSetForType(Var->getType());
+        if (!AvailableTypedRegisters.any()) {
+          // This is a hack in case we run out of physical registers
+          // due to an excessive number of "push" instructions from
+          // lowering a call.
+          AvailableRegisters = WhiteList;
+          AvailableTypedRegisters =
+              AvailableRegisters & getRegisterSetForType(Var->getType());
+        }
+        assert(AvailableTypedRegisters.any());
+        int32_t RegNum = AvailableTypedRegisters.find_first();
+        Var->setRegNum(RegNum);
+        AvailableRegisters[RegNum] = false;
+      }
+    }
+  }
+}
+
+} // end of namespace Ice
diff --git a/src/IceTargetLoweringX8632.def b/src/IceTargetLoweringX8632.def
new file mode 100644
index 0000000..b88091a
--- /dev/null
+++ b/src/IceTargetLoweringX8632.def
@@ -0,0 +1,52 @@
+//===- subzero/src/IceTargetLoweringX8632.def - x86-32 X-macros -*- C++ -*-===//
+//
+//                        The Subzero Code Generator
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines certain patterns for lowering to x86-32 target
+// instructions, in the form of x-macros.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SUBZERO_SRC_ICETARGETLOWERINGX8632_DEF
+#define SUBZERO_SRC_ICETARGETLOWERINGX8632_DEF
+
+#define FCMPX8632_TABLE                  \
+  /* val,  dflt, swap, C1,      C2 */    \
+  X(False, 0,    0,    Br_None, Br_None) \
+  X(Oeq,   0,    0,    Br_ne,   Br_p)    \
+  X(Ogt,   1,    0,    Br_a,    Br_None) \
+  X(Oge,   1,    0,    Br_ae,   Br_None) \
+  X(Olt,   1,    1,    Br_a,    Br_None) \
+  X(Ole,   1,    1,    Br_ae,   Br_None) \
+  X(One,   1,    0,    Br_ne,   Br_None) \
+  X(Ord,   1,    0,    Br_np,   Br_None) \
+  X(Ueq,   1,    0,    Br_e,    Br_None) \
+  X(Ugt,   1,    1,    Br_b,    Br_None) \
+  X(Uge,   1,    1,    Br_be,   Br_None) \
+  X(Ult,   1,    0,    Br_b,    Br_None) \
+  X(Ule,   1,    0,    Br_be,   Br_None) \
+  X(Une,   1,    0,    Br_ne,   Br_p)    \
+  X(Uno,   1,    0,    Br_p,    Br_None) \
+  X(True,  1,    0,    Br_None, Br_None) \
+//#define X(val, dflt, swap, C1, C2)
+
+#define ICMPX8632_TABLE                     \
+  /* val, C_32,  C1_64,   C2_64,   C3_64 */ \
+  X(Eq,   Br_e,  Br_None, Br_None, Br_None) \
+  X(Ne,   Br_ne, Br_None, Br_None, Br_None) \
+  X(Ugt,  Br_a,  Br_a,    Br_b,    Br_a)    \
+  X(Uge,  Br_ae, Br_a,    Br_b,    Br_ae)   \
+  X(Ult,  Br_b,  Br_b,    Br_a,    Br_b)    \
+  X(Ule,  Br_be, Br_b,    Br_a,    Br_be)   \
+  X(Sgt,  Br_g,  Br_g,    Br_l,    Br_a)    \
+  X(Sge,  Br_ge, Br_g,    Br_l,    Br_ae)   \
+  X(Slt,  Br_l,  Br_l,    Br_g,    Br_b)    \
+  X(Sle,  Br_le, Br_l,    Br_g,    Br_be)   \
+//#define X(val, C_32, C1_64, C2_64, C3_64)
+
+#endif // SUBZERO_SRC_ICETARGETLOWERINGX8632_DEF
diff --git a/src/IceTargetLoweringX8632.h b/src/IceTargetLoweringX8632.h
new file mode 100644
index 0000000..38184de
--- /dev/null
+++ b/src/IceTargetLoweringX8632.h
@@ -0,0 +1,268 @@
+//===- subzero/src/IceTargetLoweringX8632.h - x86-32 lowering ---*- C++ -*-===//
+//
+//                        The Subzero Code Generator
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file declares the TargetLoweringX8632 class, which
+// implements the TargetLowering interface for the x86-32
+// architecture.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SUBZERO_SRC_ICETARGETLOWERINGX8632_H
+#define SUBZERO_SRC_ICETARGETLOWERINGX8632_H
+
+#include "IceDefs.h"
+#include "IceTargetLowering.h"
+#include "IceInstX8632.h"
+
+namespace Ice {
+
+class TargetX8632 : public TargetLowering {
+public:
+  static TargetX8632 *create(Cfg *Func) { return new TargetX8632(Func); }
+
+  virtual void translateOm1();
+
+  virtual Variable *getPhysicalRegister(SizeT RegNum);
+  virtual IceString getRegName(SizeT RegNum, Type Ty) const;
+  virtual llvm::SmallBitVector getRegisterSet(RegSetMask Include,
+                                              RegSetMask Exclude) const;
+  virtual const llvm::SmallBitVector &getRegisterSetForType(Type Ty) const {
+    return TypeToRegisterSet[Ty];
+  }
+  virtual bool hasFramePointer() const { return IsEbpBasedFrame; }
+  virtual SizeT getFrameOrStackReg() const {
+    return IsEbpBasedFrame ? Reg_ebp : Reg_esp;
+  }
+  virtual size_t typeWidthInBytesOnStack(Type Ty) {
+    // Round up to the next multiple of 4 bytes.  In particular, i1,
+    // i8, and i16 are rounded up to 4 bytes.
+    return (typeWidthInBytes(Ty) + 3) & ~3;
+  }
+  virtual void emitVariable(const Variable *Var, const Cfg *Func) const;
+  virtual void addProlog(CfgNode *Node);
+  virtual void addEpilog(CfgNode *Node);
+  SizeT makeNextLabelNumber() { return NextLabelNumber++; }
+  // Ensure that a 64-bit Variable has been split into 2 32-bit
+  // Variables, creating them if necessary.  This is needed for all
+  // I64 operations, and it is needed for pushing F64 arguments for
+  // function calls using the 32-bit push instruction (though the
+  // latter could be done by directly writing to the stack).
+  void split64(Variable *Var);
+  void setArgOffsetAndCopy(Variable *Arg, Variable *FramePtr,
+                           int32_t BasicFrameOffset, int32_t &InArgsSizeBytes);
+  Operand *loOperand(Operand *Operand);
+  Operand *hiOperand(Operand *Operand);
+
+  enum Registers {
+#define X(val, init, name, name16, name8, scratch, preserved, stackptr,        \
+          frameptr, isI8, isInt, isFP)                                         \
+  val init,
+    REGX8632_TABLE
+#undef X
+        Reg_NUM
+  };
+
+protected:
+  TargetX8632(Cfg *Func);
+
+  virtual void postLower();
+
+  virtual void lowerAlloca(const InstAlloca *Inst);
+  virtual void lowerArithmetic(const InstArithmetic *Inst);
+  virtual void lowerAssign(const InstAssign *Inst);
+  virtual void lowerBr(const InstBr *Inst);
+  virtual void lowerCall(const InstCall *Inst);
+  virtual void lowerCast(const InstCast *Inst);
+  virtual void lowerFcmp(const InstFcmp *Inst);
+  virtual void lowerIcmp(const InstIcmp *Inst);
+  virtual void lowerLoad(const InstLoad *Inst);
+  virtual void lowerPhi(const InstPhi *Inst);
+  virtual void lowerRet(const InstRet *Inst);
+  virtual void lowerSelect(const InstSelect *Inst);
+  virtual void lowerStore(const InstStore *Inst);
+  virtual void lowerSwitch(const InstSwitch *Inst);
+  virtual void lowerUnreachable(const InstUnreachable *Inst);
+
+  // Operand legalization helpers.  To deal with address mode
+  // constraints, the helpers will create a new Operand and emit
+  // instructions that guarantee that the Operand kind is one of those
+  // indicated by the LegalMask (a bitmask of allowed kinds).  If the
+  // input Operand is known to already meet the constraints, it may be
+  // simply returned as the result, without creating any new
+  // instructions or operands.
+  enum OperandLegalization {
+    Legal_None = 0,
+    Legal_Reg = 1 << 0, // physical register, not stack location
+    Legal_Imm = 1 << 1,
+    Legal_Mem = 1 << 2, // includes [eax+4*ecx] as well as [esp+12]
+    Legal_All = ~Legal_None
+  };
+  typedef uint32_t LegalMask;
+  Operand *legalize(Operand *From, LegalMask Allowed = Legal_All,
+                    bool AllowOverlap = false,
+                    int32_t RegNum = Variable::NoRegister);
+  Variable *legalizeToVar(Operand *From, bool AllowOverlap = false,
+                          int32_t RegNum = Variable::NoRegister);
+
+  Variable *makeReg(Type Ty, int32_t RegNum = Variable::NoRegister);
+  InstCall *makeHelperCall(const IceString &Name, Variable *Dest,
+                           SizeT MaxSrcs) {
+    bool SuppressMangling = true;
+    Type Ty = Dest ? Dest->getType() : IceType_void;
+    Constant *CallTarget = Ctx->getConstantSym(Ty, 0, Name, SuppressMangling);
+    InstCall *Call = InstCall::create(Func, MaxSrcs, Dest, CallTarget);
+    return Call;
+  }
+
+  // The following are helpers that insert lowered x86 instructions
+  // with minimal syntactic overhead, so that the lowering code can
+  // look as close to assembly as practical.
+  void _adc(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Adc::create(Func, Dest, Src0));
+  }
+  void _add(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Add::create(Func, Dest, Src0));
+  }
+  void _addss(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Addss::create(Func, Dest, Src0));
+  }
+  void _and(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632And::create(Func, Dest, Src0));
+  }
+  void _br(InstX8632Br::BrCond Condition, CfgNode *TargetTrue,
+           CfgNode *TargetFalse) {
+    Context.insert(
+        InstX8632Br::create(Func, TargetTrue, TargetFalse, Condition));
+  }
+  void _br(CfgNode *Target) {
+    Context.insert(InstX8632Br::create(Func, Target));
+  }
+  void _br(InstX8632Br::BrCond Condition, CfgNode *Target) {
+    Context.insert(InstX8632Br::create(Func, Target, Condition));
+  }
+  void _br(InstX8632Br::BrCond Condition, InstX8632Label *Label) {
+    Context.insert(InstX8632Br::create(Func, Label, Condition));
+  }
+  void _cdq(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Cdq::create(Func, Dest, Src0));
+  }
+  void _cmp(Operand *Src0, Operand *Src1) {
+    Context.insert(InstX8632Icmp::create(Func, Src0, Src1));
+  }
+  void _cvt(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Cvt::create(Func, Dest, Src0));
+  }
+  void _div(Variable *Dest, Operand *Src0, Operand *Src1) {
+    Context.insert(InstX8632Div::create(Func, Dest, Src0, Src1));
+  }
+  void _divss(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Divss::create(Func, Dest, Src0));
+  }
+  void _fld(Operand *Src0) { Context.insert(InstX8632Fld::create(Func, Src0)); }
+  void _fstp(Variable *Dest) {
+    Context.insert(InstX8632Fstp::create(Func, Dest));
+  }
+  void _idiv(Variable *Dest, Operand *Src0, Operand *Src1) {
+    Context.insert(InstX8632Idiv::create(Func, Dest, Src0, Src1));
+  }
+  void _imul(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Imul::create(Func, Dest, Src0));
+  }
+  // If Dest=NULL is passed in, then a new variable is created, marked
+  // as infinite register allocation weight, and returned through the
+  // in/out Dest argument.
+  void _mov(Variable *&Dest, Operand *Src0,
+            int32_t RegNum = Variable::NoRegister) {
+    if (Dest == NULL) {
+      Dest = legalizeToVar(Src0, false, RegNum);
+    } else {
+      Context.insert(InstX8632Mov::create(Func, Dest, Src0));
+    }
+  }
+  void _movsx(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Movsx::create(Func, Dest, Src0));
+  }
+  void _movzx(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Movzx::create(Func, Dest, Src0));
+  }
+  void _mul(Variable *Dest, Variable *Src0, Operand *Src1) {
+    Context.insert(InstX8632Mul::create(Func, Dest, Src0, Src1));
+  }
+  void _mulss(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Mulss::create(Func, Dest, Src0));
+  }
+  void _or(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Or::create(Func, Dest, Src0));
+  }
+  void _pop(Variable *Dest) {
+    Context.insert(InstX8632Pop::create(Func, Dest));
+  }
+  void _push(Operand *Src0, bool SuppressStackAdjustment = false) {
+    Context.insert(InstX8632Push::create(Func, Src0, SuppressStackAdjustment));
+  }
+  void _ret(Variable *Src0 = NULL) {
+    Context.insert(InstX8632Ret::create(Func, Src0));
+  }
+  void _sar(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Sar::create(Func, Dest, Src0));
+  }
+  void _sbb(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Sbb::create(Func, Dest, Src0));
+  }
+  void _shl(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Shl::create(Func, Dest, Src0));
+  }
+  void _shld(Variable *Dest, Variable *Src0, Variable *Src1) {
+    Context.insert(InstX8632Shld::create(Func, Dest, Src0, Src1));
+  }
+  void _shr(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Shr::create(Func, Dest, Src0));
+  }
+  void _shrd(Variable *Dest, Variable *Src0, Variable *Src1) {
+    Context.insert(InstX8632Shrd::create(Func, Dest, Src0, Src1));
+  }
+  void _store(Operand *Value, OperandX8632 *Mem) {
+    Context.insert(InstX8632Store::create(Func, Value, Mem));
+  }
+  void _sub(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Sub::create(Func, Dest, Src0));
+  }
+  void _subss(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Subss::create(Func, Dest, Src0));
+  }
+  void _test(Operand *Src0, Operand *Src1) {
+    Context.insert(InstX8632Test::create(Func, Src0, Src1));
+  }
+  void _ucomiss(Operand *Src0, Operand *Src1) {
+    Context.insert(InstX8632Ucomiss::create(Func, Src0, Src1));
+  }
+  void _xor(Variable *Dest, Operand *Src0) {
+    Context.insert(InstX8632Xor::create(Func, Dest, Src0));
+  }
+
+  bool IsEbpBasedFrame;
+  int32_t FrameSizeLocals;
+  int32_t LocalsSizeBytes;
+  llvm::SmallBitVector TypeToRegisterSet[IceType_NUM];
+  llvm::SmallBitVector ScratchRegs;
+  llvm::SmallBitVector RegsUsed;
+  SizeT NextLabelNumber;
+  bool ComputedLiveRanges;
+  VarList PhysicalRegisters;
+  static IceString RegNames[];
+
+private:
+  TargetX8632(const TargetX8632 &) LLVM_DELETED_FUNCTION;
+  TargetX8632 &operator=(const TargetX8632 &) LLVM_DELETED_FUNCTION;
+  virtual ~TargetX8632() {}
+};
+
+} // end of namespace Ice
+
+#endif // SUBZERO_SRC_ICETARGETLOWERINGX8632_H
diff --git a/src/IceTypes.cpp b/src/IceTypes.cpp
index b54c0d7..84cf410 100644
--- a/src/IceTypes.cpp
+++ b/src/IceTypes.cpp
@@ -41,7 +41,7 @@
   if (Index < TypeAttributesSize) {
     Width = TypeAttributes[Index].TypeWidthInBytes;
   } else {
-    assert(0 && "Invalid type for typeWidthInBytes()");
+    llvm_unreachable("Invalid type for typeWidthInBytes()");
   }
   return Width;
 }
@@ -52,7 +52,7 @@
   if (Index < TypeAttributesSize) {
     Align = TypeAttributes[Index].TypeAlignInBytes;
   } else {
-    assert(0 && "Invalid type for typeAlignInBytes()");
+    llvm_unreachable("Invalid type for typeAlignInBytes()");
   }
   return Align;
 }
@@ -65,7 +65,7 @@
     Str << TypeAttributes[Index].DisplayString;
   } else {
     Str << "???";
-    assert(0 && "Invalid type for printing");
+    llvm_unreachable("Invalid type for printing");
   }
 
   return Str;
diff --git a/src/IceTypes.def b/src/IceTypes.def
index a54ab65..fc48b9d 100644
--- a/src/IceTypes.def
+++ b/src/IceTypes.def
@@ -25,7 +25,8 @@
   X(IceType_i32,  4,    1,     "i32")              \
   X(IceType_i64,  8,    1,     "i64")              \
   X(IceType_f32,  4,    4,     "float")            \
-  X(IceType_f64,  8,    8,     "double")
+  X(IceType_f64,  8,    8,     "double")           \
+  X(IceType_NUM,  0,    0,     "<invalid>")        \
 //#define X(tag, size, align, str)
 
 #endif // SUBZERO_SRC_ICETYPES_DEF
diff --git a/src/IceTypes.h b/src/IceTypes.h
index b3d28c3..21c399d 100644
--- a/src/IceTypes.h
+++ b/src/IceTypes.h
@@ -26,6 +26,20 @@
 #undef X
 };
 
+enum TargetArch {
+  Target_X8632,
+  Target_X8664,
+  Target_ARM32,
+  Target_ARM64
+};
+
+enum OptLevel {
+  Opt_m1,
+  Opt_0,
+  Opt_1,
+  Opt_2
+};
+
 size_t typeWidthInBytes(Type Ty);
 size_t typeAlignInBytes(Type Ty);
 
diff --git a/src/llvm2ice.cpp b/src/llvm2ice.cpp
index 7136331..08f90f8 100644
--- a/src/llvm2ice.cpp
+++ b/src/llvm2ice.cpp
@@ -165,8 +165,9 @@
     return Ice::IceType_void;
   }
 
-  // Given a LLVM instruction and an operand number, produce the Operand this
-  // refers to. If there's no such operand, return NULL.
+  // Given an LLVM instruction and an operand number, produce the
+  // Ice::Operand this refers to. If there's no such operand, return
+  // NULL.
   Ice::Operand *convertOperand(const Instruction *Inst, unsigned OpNum) {
     if (OpNum >= Inst->getNumOperands()) {
       return NULL;
@@ -189,10 +190,10 @@
           return Ctx->getConstantFloat(CFP->getValueAPF().convertToFloat());
         else if (Type == Ice::IceType_f64)
           return Ctx->getConstantDouble(CFP->getValueAPF().convertToDouble());
-        assert(0 && "Unexpected floating point type");
+        llvm_unreachable("Unexpected floating point type");
         return NULL;
       } else {
-        assert(0 && "Unhandled constant type");
+        llvm_unreachable("Unhandled constant type");
         return NULL;
       }
     } else {
@@ -534,7 +535,7 @@
     return Ice::InstAlloca::create(Func, ByteCount, Align, Dest);
   }
 
-  Ice::Inst *convertUnreachableInstruction(const UnreachableInst *Inst) {
+  Ice::Inst *convertUnreachableInstruction(const UnreachableInst * /*Inst*/) {
     return Ice::InstUnreachable::create(Func);
   }
 
@@ -576,12 +577,35 @@
         clEnumValN(Ice::IceV_Timing, "time", "Pass timing details"),
         clEnumValN(Ice::IceV_All, "all", "Use all verbose options"),
         clEnumValN(Ice::IceV_None, "none", "No verbosity"), clEnumValEnd));
+static cl::opt<Ice::TargetArch> TargetArch(
+    "target", cl::desc("Target architecture:"), cl::init(Ice::Target_X8632),
+    cl::values(
+        clEnumValN(Ice::Target_X8632, "x8632", "x86-32"),
+        clEnumValN(Ice::Target_X8632, "x86-32", "x86-32 (same as x8632)"),
+        clEnumValN(Ice::Target_X8632, "x86_32", "x86-32 (same as x8632)"),
+        clEnumValN(Ice::Target_X8664, "x8664", "x86-64"),
+        clEnumValN(Ice::Target_X8664, "x86-64", "x86-64 (same as x8664)"),
+        clEnumValN(Ice::Target_X8664, "x86_64", "x86-64 (same as x8664)"),
+        clEnumValN(Ice::Target_ARM32, "arm", "arm32"),
+        clEnumValN(Ice::Target_ARM32, "arm32", "arm32 (same as arm)"),
+        clEnumValN(Ice::Target_ARM64, "arm64", "arm64"), clEnumValEnd));
+static cl::opt<Ice::OptLevel>
+OptLevel(cl::desc("Optimization level"), cl::init(Ice::Opt_m1),
+         cl::value_desc("level"),
+         cl::values(clEnumValN(Ice::Opt_m1, "Om1", "-1"),
+                    clEnumValN(Ice::Opt_m1, "O-1", "-1"),
+                    clEnumValN(Ice::Opt_0, "O0", "0"),
+                    clEnumValN(Ice::Opt_1, "O1", "1"),
+                    clEnumValN(Ice::Opt_2, "O2", "2"), clEnumValEnd));
 static cl::opt<std::string> IRFilename(cl::Positional, cl::desc("<IR file>"),
                                        cl::init("-"));
 static cl::opt<std::string> OutputFilename("o",
                                            cl::desc("Override output filename"),
                                            cl::init("-"),
                                            cl::value_desc("filename"));
+static cl::opt<std::string> LogFilename("log", cl::desc("Set log filename"),
+                                        cl::init("-"),
+                                        cl::value_desc("filename"));
 static cl::opt<std::string>
 TestPrefix("prefix", cl::desc("Prepend a prefix to symbol names for testing"),
            cl::init(""), cl::value_desc("prefix"));
@@ -605,6 +629,8 @@
     cl::init(LLVMFormat));
 
 int main(int argc, char **argv) {
+  int ExitStatus = 0;
+
   cl::ParseCommandLineOptions(argc, argv);
 
   // Parse the input LLVM IR file into a module.
@@ -637,8 +663,14 @@
   raw_os_ostream *Os =
       new raw_os_ostream(OutputFilename == "-" ? std::cout : Ofs);
   Os->SetUnbuffered();
+  std::ofstream Lfs;
+  if (LogFilename != "-") {
+    Lfs.open(LogFilename.c_str(), std::ofstream::out);
+  }
+  raw_os_ostream *Ls = new raw_os_ostream(LogFilename == "-" ? std::cout : Lfs);
+  Ls->SetUnbuffered();
 
-  Ice::GlobalContext Ctx(Os, Os, VMask, TestPrefix);
+  Ice::GlobalContext Ctx(Ls, Os, VMask, TargetArch, OptLevel, TestPrefix);
 
   for (Module::const_iterator I = Mod->begin(), E = Mod->end(); I != E; ++I) {
     if (I->empty())
@@ -658,8 +690,28 @@
 
     if (DisableTranslation) {
       Func->dump();
+    } else {
+      Ice::Timer TTranslate;
+      Func->translate();
+      if (SubzeroTimingEnabled) {
+        std::cerr << "[Subzero timing] Translate function "
+                  << Func->getFunctionName() << ": "
+                  << TTranslate.getElapsedSec() << " sec\n";
+      }
+      if (Func->hasError()) {
+        errs() << "ICE translation error: " << Func->getError() << "\n";
+        ExitStatus = 1;
+      }
+
+      Ice::Timer TEmit;
+      Func->emit();
+      if (SubzeroTimingEnabled) {
+        std::cerr << "[Subzero timing] Emit function "
+                  << Func->getFunctionName() << ": " << TEmit.getElapsedSec()
+                  << " sec\n";
+      }
     }
   }
 
-  return 0;
+  return ExitStatus;
 }
diff --git a/tests_lit/llvm2ice_tests/64bit.pnacl.ll b/tests_lit/llvm2ice_tests/64bit.pnacl.ll
index 761f388..1ca8077 100644
--- a/tests_lit/llvm2ice_tests/64bit.pnacl.ll
+++ b/tests_lit/llvm2ice_tests/64bit.pnacl.ll
@@ -1,5 +1,10 @@
-; RUIN: %llvm2ice --verbose none %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; This tries to be a comprehensive test of i64 operations, in
+; particular the patterns for lowering i64 operations into constituent
+; i32 operations on x86-32.
+
+; RUIN: %llvm2ice -O2 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice -Om1 --verbose none %s | FileCheck --check-prefix=OPTM1 %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -40,6 +45,24 @@
 ; CHECK-NEXT: push
 ; CHECK-NEXT: push
 ; CHECK-NEXT: call    ignore64BitArgNoInline
+;
+; OPTM1: pass64BitArg:
+; OPTM1:      push    123
+; OPTM1-NEXT: push
+; OPTM1-NEXT: push
+; OPTM1-NEXT: call    ignore64BitArgNoInline
+; OPTM1:      push
+; OPTM1-NEXT: push
+; OPTM1-NEXT: push    123
+; OPTM1-NEXT: push
+; OPTM1-NEXT: push
+; OPTM1-NEXT: call    ignore64BitArgNoInline
+; OPTM1:      push
+; OPTM1-NEXT: push
+; OPTM1-NEXT: push    123
+; OPTM1-NEXT: push
+; OPTM1-NEXT: push
+; OPTM1-NEXT: call    ignore64BitArgNoInline
 
 declare i32 @ignore64BitArgNoInline(i64, i32, i64)
 
@@ -55,6 +78,14 @@
 ; CHECK-NEXT: push    ecx
 ; CHECK-NEXT: push    eax
 ; CHECK-NEXT: call    ignore64BitArgNoInline
+;
+; OPTM1: pass64BitConstArg:
+; OPTM1:      push    3735928559
+; OPTM1-NEXT: push    305419896
+; OPTM1-NEXT: push    123
+; OPTM1-NEXT: push    dword ptr [
+; OPTM1-NEXT: push    dword ptr [
+; OPTM1-NEXT: call    ignore64BitArgNoInline
 
 define internal i64 @return64BitArg(i64 %a) {
 entry:
@@ -64,6 +95,11 @@
 ; CHECK: mov     {{.*}}, dword ptr [esp+4]
 ; CHECK: mov     {{.*}}, dword ptr [esp+8]
 ; CHECK: ret
+;
+; OPTM1: return64BitArg:
+; OPTM1: mov     {{.*}}, dword ptr [esp+4]
+; OPTM1: mov     {{.*}}, dword ptr [esp+8]
+; OPTM1: ret
 
 define internal i64 @return64BitConst() {
 entry:
@@ -73,6 +109,11 @@
 ; CHECK: mov     eax, 305419896
 ; CHECK: mov     edx, 3735928559
 ; CHECK: ret
+;
+; OPTM1: return64BitConst:
+; OPTM1: mov     eax, 305419896
+; OPTM1: mov     edx, 3735928559
+; OPTM1: ret
 
 define internal i64 @add64BitSigned(i64 %a, i64 %b) {
 entry:
@@ -83,6 +124,11 @@
 ; CHECK: add
 ; CHECK: adc
 ; CHECK: ret
+;
+; OPTM1: add64BitSigned:
+; OPTM1: add
+; OPTM1: adc
+; OPTM1: ret
 
 define internal i64 @add64BitUnsigned(i64 %a, i64 %b) {
 entry:
@@ -93,6 +139,11 @@
 ; CHECK: add
 ; CHECK: adc
 ; CHECK: ret
+;
+; OPTM1: add64BitUnsigned:
+; OPTM1: add
+; OPTM1: adc
+; OPTM1: ret
 
 define internal i64 @sub64BitSigned(i64 %a, i64 %b) {
 entry:
@@ -103,6 +154,11 @@
 ; CHECK: sub
 ; CHECK: sbb
 ; CHECK: ret
+;
+; OPTM1: sub64BitSigned:
+; OPTM1: sub
+; OPTM1: sbb
+; OPTM1: ret
 
 define internal i64 @sub64BitUnsigned(i64 %a, i64 %b) {
 entry:
@@ -113,6 +169,11 @@
 ; CHECK: sub
 ; CHECK: sbb
 ; CHECK: ret
+;
+; OPTM1: sub64BitUnsigned:
+; OPTM1: sub
+; OPTM1: sbb
+; OPTM1: ret
 
 define internal i64 @mul64BitSigned(i64 %a, i64 %b) {
 entry:
@@ -126,6 +187,14 @@
 ; CHECK: add
 ; CHECK: add
 ; CHECK: ret
+;
+; OPTM1: mul64BitSigned:
+; OPTM1: imul
+; OPTM1: imul
+; OPTM1: mul
+; OPTM1: add
+; OPTM1: add
+; OPTM1: ret
 
 define internal i64 @mul64BitUnsigned(i64 %a, i64 %b) {
 entry:
@@ -139,6 +208,14 @@
 ; CHECK: add
 ; CHECK: add
 ; CHECK: ret
+;
+; OPTM1: mul64BitUnsigned:
+; OPTM1: imul
+; OPTM1: imul
+; OPTM1: mul
+; OPTM1: add
+; OPTM1: add
+; OPTM1: ret
 
 define internal i64 @div64BitSigned(i64 %a, i64 %b) {
 entry:
@@ -148,6 +225,10 @@
 ; CHECK: div64BitSigned:
 ; CHECK: call    __divdi3
 ; CHECK: ret
+;
+; OPTM1: div64BitSigned:
+; OPTM1: call    __divdi3
+; OPTM1: ret
 
 define internal i64 @div64BitUnsigned(i64 %a, i64 %b) {
 entry:
@@ -157,6 +238,10 @@
 ; CHECK: div64BitUnsigned:
 ; CHECK: call    __udivdi3
 ; CHECK: ret
+;
+; OPTM1: div64BitUnsigned:
+; OPTM1: call    __udivdi3
+; OPTM1: ret
 
 define internal i64 @rem64BitSigned(i64 %a, i64 %b) {
 entry:
@@ -166,6 +251,10 @@
 ; CHECK: rem64BitSigned:
 ; CHECK: call    __moddi3
 ; CHECK: ret
+;
+; OPTM1: rem64BitSigned:
+; OPTM1: call    __moddi3
+; OPTM1: ret
 
 define internal i64 @rem64BitUnsigned(i64 %a, i64 %b) {
 entry:
@@ -175,6 +264,10 @@
 ; CHECK: rem64BitUnsigned:
 ; CHECK: call    __umoddi3
 ; CHECK: ret
+;
+; OPTM1: rem64BitUnsigned:
+; OPTM1: call    __umoddi3
+; OPTM1: ret
 
 define internal i64 @shl64BitSigned(i64 %a, i64 %b) {
 entry:
@@ -186,6 +279,12 @@
 ; CHECK: shl e
 ; CHECK: test {{.*}}, 32
 ; CHECK: je
+;
+; OPTM1: shl64BitSigned:
+; OPTM1: shld
+; OPTM1: shl e
+; OPTM1: test {{.*}}, 32
+; OPTM1: je
 
 define internal i64 @shl64BitUnsigned(i64 %a, i64 %b) {
 entry:
@@ -197,6 +296,12 @@
 ; CHECK: shl e
 ; CHECK: test {{.*}}, 32
 ; CHECK: je
+;
+; OPTM1: shl64BitUnsigned:
+; OPTM1: shld
+; OPTM1: shl e
+; OPTM1: test {{.*}}, 32
+; OPTM1: je
 
 define internal i64 @shr64BitSigned(i64 %a, i64 %b) {
 entry:
@@ -209,6 +314,13 @@
 ; CHECK: test {{.*}}, 32
 ; CHECK: je
 ; CHECK: sar {{.*}}, 31
+;
+; OPTM1: shr64BitSigned:
+; OPTM1: shrd
+; OPTM1: sar
+; OPTM1: test {{.*}}, 32
+; OPTM1: je
+; OPTM1: sar {{.*}}, 31
 
 define internal i64 @shr64BitUnsigned(i64 %a, i64 %b) {
 entry:
@@ -220,6 +332,12 @@
 ; CHECK: shr
 ; CHECK: test {{.*}}, 32
 ; CHECK: je
+;
+; OPTM1: shr64BitUnsigned:
+; OPTM1: shrd
+; OPTM1: shr
+; OPTM1: test {{.*}}, 32
+; OPTM1: je
 
 define internal i64 @and64BitSigned(i64 %a, i64 %b) {
 entry:
@@ -229,6 +347,10 @@
 ; CHECK: and64BitSigned:
 ; CHECK: and
 ; CHECK: and
+;
+; OPTM1: and64BitSigned:
+; OPTM1: and
+; OPTM1: and
 
 define internal i64 @and64BitUnsigned(i64 %a, i64 %b) {
 entry:
@@ -238,6 +360,10 @@
 ; CHECK: and64BitUnsigned:
 ; CHECK: and
 ; CHECK: and
+;
+; OPTM1: and64BitUnsigned:
+; OPTM1: and
+; OPTM1: and
 
 define internal i64 @or64BitSigned(i64 %a, i64 %b) {
 entry:
@@ -247,6 +373,10 @@
 ; CHECK: or64BitSigned:
 ; CHECK: or
 ; CHECK: or
+;
+; OPTM1: or64BitSigned:
+; OPTM1: or
+; OPTM1: or
 
 define internal i64 @or64BitUnsigned(i64 %a, i64 %b) {
 entry:
@@ -256,6 +386,10 @@
 ; CHECK: or64BitUnsigned:
 ; CHECK: or
 ; CHECK: or
+;
+; OPTM1: or64BitUnsigned:
+; OPTM1: or
+; OPTM1: or
 
 define internal i64 @xor64BitSigned(i64 %a, i64 %b) {
 entry:
@@ -265,6 +399,10 @@
 ; CHECK: xor64BitSigned:
 ; CHECK: xor
 ; CHECK: xor
+;
+; OPTM1: xor64BitSigned:
+; OPTM1: xor
+; OPTM1: xor
 
 define internal i64 @xor64BitUnsigned(i64 %a, i64 %b) {
 entry:
@@ -274,6 +412,10 @@
 ; CHECK: xor64BitUnsigned:
 ; CHECK: xor
 ; CHECK: xor
+;
+; OPTM1: xor64BitUnsigned:
+; OPTM1: xor
+; OPTM1: xor
 
 define internal i32 @trunc64To32Signed(i64 %a) {
 entry:
@@ -283,6 +425,10 @@
 ; CHECK: trunc64To32Signed:
 ; CHECK: mov     eax, dword ptr [esp+4]
 ; CHECK-NEXT: ret
+;
+; OPTM1: trunc64To32Signed:
+; OPTM1: mov     eax, dword ptr [esp+
+; OPTM1: ret
 
 define internal i32 @trunc64To16Signed(i64 %a) {
 entry:
@@ -294,6 +440,11 @@
 ; CHECK:      mov     eax, dword ptr [esp+4]
 ; CHECK-NEXT: movsx  eax, ax
 ; CHECK-NEXT: ret
+;
+; OPTM1: trunc64To16Signed:
+; OPTM1:      mov     eax, dword ptr [esp+
+; OPTM1: movsx  eax,
+; OPTM1: ret
 
 define internal i32 @trunc64To8Signed(i64 %a) {
 entry:
@@ -305,6 +456,11 @@
 ; CHECK:      mov     eax, dword ptr [esp+4]
 ; CHECK-NEXT: movsx  eax, al
 ; CHECK-NEXT: ret
+;
+; OPTM1: trunc64To8Signed:
+; OPTM1:      mov     eax, dword ptr [esp+
+; OPTM1: movsx  eax,
+; OPTM1: ret
 
 define internal i32 @trunc64To32Unsigned(i64 %a) {
 entry:
@@ -314,6 +470,10 @@
 ; CHECK: trunc64To32Unsigned:
 ; CHECK: mov     eax, dword ptr [esp+4]
 ; CHECK-NEXT: ret
+;
+; OPTM1: trunc64To32Unsigned:
+; OPTM1: mov     eax, dword ptr [esp+
+; OPTM1: ret
 
 define internal i32 @trunc64To16Unsigned(i64 %a) {
 entry:
@@ -325,6 +485,11 @@
 ; CHECK:      mov     eax, dword ptr [esp+4]
 ; CHECK-NEXT: movzx  eax, ax
 ; CHECK-NEXT: ret
+;
+; OPTM1: trunc64To16Unsigned:
+; OPTM1:      mov     eax, dword ptr [esp+
+; OPTM1: movzx  eax,
+; OPTM1: ret
 
 define internal i32 @trunc64To8Unsigned(i64 %a) {
 entry:
@@ -336,6 +501,11 @@
 ; CHECK:      mov     eax, dword ptr [esp+4]
 ; CHECK-NEXT: movzx  eax, al
 ; CHECK-NEXT: ret
+;
+; OPTM1: trunc64To8Unsigned:
+; OPTM1:      mov     eax, dword ptr [esp+
+; OPTM1: movzx  eax,
+; OPTM1: ret
 
 define internal i32 @trunc64To1(i64 %a) {
 entry:
@@ -348,6 +518,11 @@
 ; CHECK:      mov     eax, dword ptr [esp+4]
 ; CHECK:      and     eax, 1
 ; CHECK-NEXT: ret
+;
+; OPTM1: trunc64To1:
+; OPTM1:      mov     eax, dword ptr [esp+
+; OPTM1:      and     eax, 1
+; OPTM1: ret
 
 define internal i64 @sext32To64(i32 %a) {
 entry:
@@ -357,6 +532,10 @@
 ; CHECK: sext32To64:
 ; CHECK: mov
 ; CHECK: sar {{.*}}, 31
+;
+; OPTM1: sext32To64:
+; OPTM1: mov
+; OPTM1: sar {{.*}}, 31
 
 define internal i64 @sext16To64(i32 %a) {
 entry:
@@ -367,6 +546,10 @@
 ; CHECK: sext16To64:
 ; CHECK: movsx
 ; CHECK: sar {{.*}}, 31
+;
+; OPTM1: sext16To64:
+; OPTM1: movsx
+; OPTM1: sar {{.*}}, 31
 
 define internal i64 @sext8To64(i32 %a) {
 entry:
@@ -377,6 +560,10 @@
 ; CHECK: sext8To64:
 ; CHECK: movsx
 ; CHECK: sar {{.*}}, 31
+;
+; OPTM1: sext8To64:
+; OPTM1: movsx
+; OPTM1: sar {{.*}}, 31
 
 define internal i64 @zext32To64(i32 %a) {
 entry:
@@ -386,6 +573,10 @@
 ; CHECK: zext32To64:
 ; CHECK: mov
 ; CHECK: mov {{.*}}, 0
+;
+; OPTM1: zext32To64:
+; OPTM1: mov
+; OPTM1: mov {{.*}}, 0
 
 define internal i64 @zext16To64(i32 %a) {
 entry:
@@ -396,6 +587,10 @@
 ; CHECK: zext16To64:
 ; CHECK: movzx
 ; CHECK: mov {{.*}}, 0
+;
+; OPTM1: zext16To64:
+; OPTM1: movzx
+; OPTM1: mov {{.*}}, 0
 
 define internal i64 @zext8To64(i32 %a) {
 entry:
@@ -406,6 +601,10 @@
 ; CHECK: zext8To64:
 ; CHECK: movzx
 ; CHECK: mov {{.*}}, 0
+;
+; OPTM1: zext8To64:
+; OPTM1: movzx
+; OPTM1: mov {{.*}}, 0
 
 define internal i64 @zext1To64(i32 %a) {
 entry:
@@ -416,6 +615,10 @@
 ; CHECK: zext1To64:
 ; CHECK: movzx
 ; CHECK: mov {{.*}}, 0
+;
+; OPTM1: zext1To64:
+; OPTM1: movzx
+; OPTM1: mov {{.*}}, 0
 
 define internal void @icmpEq64(i64 %a, i64 %b, i64 %c, i64 %d) {
 entry:
@@ -444,6 +647,14 @@
 ; CHECK: jne
 ; CHECK: jne
 ; CHECK: call
+;
+; OPTM1: icmpEq64:
+; OPTM1: jne
+; OPTM1: jne
+; OPTM1: call
+; OPTM1: jne
+; OPTM1: jne
+; OPTM1: call
 
 declare void @func()
 
@@ -474,6 +685,14 @@
 ; CHECK: jne
 ; CHECK: jne
 ; CHECK: call
+;
+; OPTM1: icmpNe64:
+; OPTM1: jne
+; OPTM1: jne
+; OPTM1: call
+; OPTM1: jne
+; OPTM1: jne
+; OPTM1: call
 
 define internal void @icmpGt64(i64 %a, i64 %b, i64 %c, i64 %d) {
 entry:
@@ -504,6 +723,16 @@
 ; CHECK: jl
 ; CHECK: ja
 ; CHECK: call
+;
+; OPTM1: icmpGt64:
+; OPTM1: ja
+; OPTM1: jb
+; OPTM1: ja
+; OPTM1: call
+; OPTM1: jg
+; OPTM1: jl
+; OPTM1: ja
+; OPTM1: call
 
 define internal void @icmpGe64(i64 %a, i64 %b, i64 %c, i64 %d) {
 entry:
@@ -534,6 +763,16 @@
 ; CHECK: jl
 ; CHECK: jae
 ; CHECK: call
+;
+; OPTM1: icmpGe64:
+; OPTM1: ja
+; OPTM1: jb
+; OPTM1: jae
+; OPTM1: call
+; OPTM1: jg
+; OPTM1: jl
+; OPTM1: jae
+; OPTM1: call
 
 define internal void @icmpLt64(i64 %a, i64 %b, i64 %c, i64 %d) {
 entry:
@@ -564,6 +803,16 @@
 ; CHECK: jg
 ; CHECK: jb
 ; CHECK: call
+;
+; OPTM1: icmpLt64:
+; OPTM1: jb
+; OPTM1: ja
+; OPTM1: jb
+; OPTM1: call
+; OPTM1: jl
+; OPTM1: jg
+; OPTM1: jb
+; OPTM1: call
 
 define internal void @icmpLe64(i64 %a, i64 %b, i64 %c, i64 %d) {
 entry:
@@ -594,6 +843,16 @@
 ; CHECK: jg
 ; CHECK: jbe
 ; CHECK: call
+;
+; OPTM1: icmpLe64:
+; OPTM1: jb
+; OPTM1: ja
+; OPTM1: jbe
+; OPTM1: call
+; OPTM1: jl
+; OPTM1: jg
+; OPTM1: jbe
+; OPTM1: call
 
 define internal i32 @icmpEq64Bool(i64 %a, i64 %b) {
 entry:
@@ -604,6 +863,10 @@
 ; CHECK: icmpEq64Bool:
 ; CHECK: jne
 ; CHECK: jne
+;
+; OPTM1: icmpEq64Bool:
+; OPTM1: jne
+; OPTM1: jne
 
 define internal i32 @icmpNe64Bool(i64 %a, i64 %b) {
 entry:
@@ -614,6 +877,10 @@
 ; CHECK: icmpNe64Bool:
 ; CHECK: jne
 ; CHECK: jne
+;
+; OPTM1: icmpNe64Bool:
+; OPTM1: jne
+; OPTM1: jne
 
 define internal i32 @icmpSgt64Bool(i64 %a, i64 %b) {
 entry:
@@ -627,6 +894,13 @@
 ; CHECK: jl
 ; CHECK: cmp
 ; CHECK: ja
+;
+; OPTM1: icmpSgt64Bool:
+; OPTM1: cmp
+; OPTM1: jg
+; OPTM1: jl
+; OPTM1: cmp
+; OPTM1: ja
 
 define internal i32 @icmpUgt64Bool(i64 %a, i64 %b) {
 entry:
@@ -640,6 +914,13 @@
 ; CHECK: jb
 ; CHECK: cmp
 ; CHECK: ja
+;
+; OPTM1: icmpUgt64Bool:
+; OPTM1: cmp
+; OPTM1: ja
+; OPTM1: jb
+; OPTM1: cmp
+; OPTM1: ja
 
 define internal i32 @icmpSge64Bool(i64 %a, i64 %b) {
 entry:
@@ -653,6 +934,13 @@
 ; CHECK: jl
 ; CHECK: cmp
 ; CHECK: jae
+;
+; OPTM1: icmpSge64Bool:
+; OPTM1: cmp
+; OPTM1: jg
+; OPTM1: jl
+; OPTM1: cmp
+; OPTM1: jae
 
 define internal i32 @icmpUge64Bool(i64 %a, i64 %b) {
 entry:
@@ -666,6 +954,13 @@
 ; CHECK: jb
 ; CHECK: cmp
 ; CHECK: jae
+;
+; OPTM1: icmpUge64Bool:
+; OPTM1: cmp
+; OPTM1: ja
+; OPTM1: jb
+; OPTM1: cmp
+; OPTM1: jae
 
 define internal i32 @icmpSlt64Bool(i64 %a, i64 %b) {
 entry:
@@ -679,6 +974,13 @@
 ; CHECK: jg
 ; CHECK: cmp
 ; CHECK: jb
+;
+; OPTM1: icmpSlt64Bool:
+; OPTM1: cmp
+; OPTM1: jl
+; OPTM1: jg
+; OPTM1: cmp
+; OPTM1: jb
 
 define internal i32 @icmpUlt64Bool(i64 %a, i64 %b) {
 entry:
@@ -692,6 +994,13 @@
 ; CHECK: ja
 ; CHECK: cmp
 ; CHECK: jb
+;
+; OPTM1: icmpUlt64Bool:
+; OPTM1: cmp
+; OPTM1: jb
+; OPTM1: ja
+; OPTM1: cmp
+; OPTM1: jb
 
 define internal i32 @icmpSle64Bool(i64 %a, i64 %b) {
 entry:
@@ -705,6 +1014,13 @@
 ; CHECK: jg
 ; CHECK: cmp
 ; CHECK: jbe
+;
+; OPTM1: icmpSle64Bool:
+; OPTM1: cmp
+; OPTM1: jl
+; OPTM1: jg
+; OPTM1: cmp
+; OPTM1: jbe
 
 define internal i32 @icmpUle64Bool(i64 %a, i64 %b) {
 entry:
@@ -718,6 +1034,13 @@
 ; CHECK: ja
 ; CHECK: cmp
 ; CHECK: jbe
+;
+; OPTM1: icmpUle64Bool:
+; OPTM1: cmp
+; OPTM1: jb
+; OPTM1: ja
+; OPTM1: cmp
+; OPTM1: jbe
 
 define internal i64 @load64(i32 %a) {
 entry:
@@ -729,6 +1052,10 @@
 ; CHECK: mov e[[REGISTER:[a-z]+]], dword ptr [esp+4]
 ; CHECK-NEXT: mov {{.*}}, dword ptr [e[[REGISTER]]]
 ; CHECK-NEXT: mov {{.*}}, dword ptr [e[[REGISTER]]+4]
+;
+; OPTM1: load64:
+; OPTM1: mov e{{..}}, dword ptr [e{{..}}]
+; OPTM1: mov e{{..}}, dword ptr [e{{..}}+4]
 
 define internal void @store64(i32 %a, i64 %value) {
 entry:
@@ -740,6 +1067,10 @@
 ; CHECK: mov e[[REGISTER:[a-z]+]], dword ptr [esp+4]
 ; CHECK: mov dword ptr [e[[REGISTER]]+4],
 ; CHECK: mov dword ptr [e[[REGISTER]]],
+;
+; OPTM1: store64:
+; OPTM1: mov dword ptr [e[[REGISTER:[a-z]+]]+4],
+; OPTM1: mov dword ptr [e[[REGISTER]]],
 
 define internal void @store64Const(i32 %a) {
 entry:
@@ -751,6 +1082,10 @@
 ; CHECK: mov e[[REGISTER:[a-z]+]], dword ptr [esp+4]
 ; CHECK: mov dword ptr [e[[REGISTER]]+4], 3735928559
 ; CHECK: mov dword ptr [e[[REGISTER]]], 305419896
+;
+; OPTM1: store64Const:
+; OPTM1: mov dword ptr [e[[REGISTER:[a-z]+]]+4], 3735928559
+; OPTM1: mov dword ptr [e[[REGISTER]]], 305419896
 
 define internal i64 @select64VarVar(i64 %a, i64 %b) {
 entry:
@@ -766,6 +1101,15 @@
 ; CHECK: jb
 ; CHECK: cmp
 ; CHECK: jne
+;
+; OPTM1: select64VarVar:
+; OPTM1: cmp
+; OPTM1: jb
+; OPTM1: ja
+; OPTM1: cmp
+; OPTM1: jb
+; OPTM1: cmp
+; OPTM1: jne
 
 define internal i64 @select64VarConst(i64 %a, i64 %b) {
 entry:
@@ -781,6 +1125,15 @@
 ; CHECK: jb
 ; CHECK: cmp
 ; CHECK: jne
+;
+; OPTM1: select64VarConst:
+; OPTM1: cmp
+; OPTM1: jb
+; OPTM1: ja
+; OPTM1: cmp
+; OPTM1: jb
+; OPTM1: cmp
+; OPTM1: jne
 
 define internal i64 @select64ConstVar(i64 %a, i64 %b) {
 entry:
@@ -796,6 +1149,15 @@
 ; CHECK: jb
 ; CHECK: cmp
 ; CHECK: jne
+;
+; OPTM1: select64ConstVar:
+; OPTM1: cmp
+; OPTM1: jb
+; OPTM1: ja
+; OPTM1: cmp
+; OPTM1: jb
+; OPTM1: cmp
+; OPTM1: jne
 
 ; ERRORS-NOT: ICE translation error
 ; DUMP-NOT: SZ
diff --git a/tests_lit/llvm2ice_tests/alloc.ll b/tests_lit/llvm2ice_tests/alloc.ll
index b2f90e4..d9da106 100644
--- a/tests_lit/llvm2ice_tests/alloc.ll
+++ b/tests_lit/llvm2ice_tests/alloc.ll
@@ -1,5 +1,9 @@
-; RUIN: %llvm2ice --verbose none %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; This is a basic test of the alloca instruction - one test for alloca
+; of a fixed size, and one test for variable size.
+
+; RUIN: %llvm2ice -O2 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice -Om1 --verbose none %s | FileCheck --check-prefix=OPTM1 %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -10,11 +14,18 @@
   %__2 = ptrtoint i8* %array to i32
   call void @f1(i32 %__2)
   ret void
-  ; CHECK:      sub     esp, 400
-  ; CHECK-NEXT: mov     eax, esp
-  ; CHECK-NEXT: push    eax
-  ; CHECK-NEXT: call    f1
 }
+; CHECK: fixed_400:
+; CHECK:      sub     esp, 400
+; CHECK-NEXT: mov     eax, esp
+; CHECK-NEXT: push    eax
+; CHECK-NEXT: call    f1
+;
+; OPTM1: fixed_400:
+; OPTM1:      sub     esp, 400
+; OPTM1-NEXT: mov     {{.*}}, esp
+; OPTM1:      push
+; OPTM1-NEXT: call    f1
 
 declare void @f1(i32)
 
@@ -24,12 +35,18 @@
   %__2 = ptrtoint i8* %array to i32
   call void @f2(i32 %__2)
   ret void
-  ; CHECK:      mov     eax, dword ptr [ebp+8]
-  ; CHECK-NEXT: sub     esp, eax
-  ; CHECK-NEXT: mov     eax, esp
-  ; CHECK-NEXT: push    eax
-  ; CHECK-NEXT: call    f2
 }
+; CHECK: variable_n:
+; CHECK:      mov     eax, dword ptr [ebp+8]
+; CHECK-NEXT: sub     esp, eax
+; CHECK-NEXT: mov     eax, esp
+; CHECK-NEXT: push    eax
+; CHECK-NEXT: call    f2
+;
+; OPTM1: variable_n:
+; OPTM1:      mov     {{.*}}, esp
+; OPTM1:      push
+; OPTM1-NEXT: call    f2
 
 declare void @f2(i32)
 
diff --git a/tests_lit/llvm2ice_tests/arith-opt.ll b/tests_lit/llvm2ice_tests/arith-opt.ll
index f080125..fff8e3f 100644
--- a/tests_lit/llvm2ice_tests/arith-opt.ll
+++ b/tests_lit/llvm2ice_tests/arith-opt.ll
@@ -1,5 +1,8 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; This is a very early test that just checks the representation of i32
+; arithmetic instructions.  No assembly tests are done.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/arithmetic-chain.ll b/tests_lit/llvm2ice_tests/arithmetic-chain.ll
index 2609ef7..01ec91a 100644
--- a/tests_lit/llvm2ice_tests/arithmetic-chain.ll
+++ b/tests_lit/llvm2ice_tests/arithmetic-chain.ll
@@ -1,5 +1,9 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; This is a very early test that just checks the representation of
+; arithmetic instructions, i64, variables, and constants.  No assembly
+; tests are done.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/bitcast.ll b/tests_lit/llvm2ice_tests/bitcast.ll
index c180c87..8c9a936 100644
--- a/tests_lit/llvm2ice_tests/bitcast.ll
+++ b/tests_lit/llvm2ice_tests/bitcast.ll
@@ -1,5 +1,7 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Trivial smoke test of bitcast between integer and FP types.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -7,24 +9,28 @@
 define internal i32 @cast_f2i(float %f) {
 entry:
   %v0 = bitcast float %f to i32
+; CHECK: bitcast
   ret i32 %v0
 }
 
 define internal float @cast_i2f(i32 %i) {
 entry:
   %v0 = bitcast i32 %i to float
+; CHECK: bitcast
   ret float %v0
 }
 
 define internal i64 @cast_d2ll(double %d) {
 entry:
   %v0 = bitcast double %d to i64
+; CHECK: bitcast
   ret i64 %v0
 }
 
 define internal double @cast_ll2d(i64 %ll) {
 entry:
   %v0 = bitcast i64 %ll to double
+; CHECK: bitcast
   ret double %v0
 }
 
diff --git a/tests_lit/llvm2ice_tests/bool-opt.ll b/tests_lit/llvm2ice_tests/bool-opt.ll
index 3078615..1ccdeb6 100644
--- a/tests_lit/llvm2ice_tests/bool-opt.ll
+++ b/tests_lit/llvm2ice_tests/bool-opt.ll
@@ -1,5 +1,7 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Trivial smoke test of icmp without fused branch opportunity.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/branch-simple.ll b/tests_lit/llvm2ice_tests/branch-simple.ll
index 201238e..6702790 100644
--- a/tests_lit/llvm2ice_tests/branch-simple.ll
+++ b/tests_lit/llvm2ice_tests/branch-simple.ll
@@ -1,5 +1,8 @@
-; RUIN: %llvm2ice %s -verbose inst | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Trivial smoke test of compare and branch, with multiple basic
+; blocks.
+
+; RUN: %llvm2ice %s --verbose inst | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/call.ll b/tests_lit/llvm2ice_tests/call.ll
index c029789..41dcf09 100644
--- a/tests_lit/llvm2ice_tests/call.ll
+++ b/tests_lit/llvm2ice_tests/call.ll
@@ -1,5 +1,8 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Simple smoke test of the call instruction.  The assembly checks
+; currently only verify the function labels.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -49,6 +52,7 @@
 
 define void @call_void(i32 %n) {
 ; CHECK: define void @call_void
+
 entry:
   %cmp2 = icmp sgt i32 %n, 0
   br i1 %cmp2, label %if.then, label %if.end
diff --git a/tests_lit/llvm2ice_tests/callindirect.pnacl.ll b/tests_lit/llvm2ice_tests/callindirect.pnacl.ll
index 10b0aba..0112a5b 100644
--- a/tests_lit/llvm2ice_tests/callindirect.pnacl.ll
+++ b/tests_lit/llvm2ice_tests/callindirect.pnacl.ll
@@ -1,5 +1,10 @@
-; RUIN: %llvm2ice --verbose none %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Test of multiple indirect calls to the same target.  Each call
+; should be to the same operand, whether it's in a register or on the
+; stack.
+
+; RUIN: %llvm2ice -O2 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice -Om1 --verbose none %s | FileCheck --check-prefix=OPTM1 %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -24,6 +29,12 @@
 ; CHECK: call [[REGISTER]]
 ; CHECK: call [[REGISTER]]
 ; CHECK: call [[REGISTER]]
+;
+; OPTM1: call [[TARGET:.+]]
+; OPTM1: call [[TARGET]]
+; OPTM1: call [[TARGET]]
+; OPTM1: call [[TARGET]]
+; OPTM1: call [[TARGET]]
 
 ; ERRORS-NOT: ICE translation error
 ; DUMP-NOT: SZ
diff --git a/tests_lit/llvm2ice_tests/casts.ll b/tests_lit/llvm2ice_tests/casts.ll
index 0849fb2..a75cd51 100644
--- a/tests_lit/llvm2ice_tests/casts.ll
+++ b/tests_lit/llvm2ice_tests/casts.ll
@@ -1,5 +1,5 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; RUIN: %llvm2ice -O2 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/cmp-opt.ll b/tests_lit/llvm2ice_tests/cmp-opt.ll
index 124630d..0dd2a80 100644
--- a/tests_lit/llvm2ice_tests/cmp-opt.ll
+++ b/tests_lit/llvm2ice_tests/cmp-opt.ll
@@ -1,5 +1,8 @@
-; RUIN: %llvm2ice %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Simple test of non-fused compare/branch.
+
+; RUIN: %llvm2ice -O2 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice -Om1 --verbose none %s | FileCheck --check-prefix=OPTM1 %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -27,8 +30,6 @@
 
 declare void @use(i1 zeroext)
 
-; ERRORS-NOT: ICE translation error
-
 ; CHECK:      .globl testBool
 ; Two bool computations
 ; CHECK:      cmp
@@ -40,4 +41,18 @@
 ; CHECK:      cmp
 ; CHECK:      call
 ; CHECK:      ret
+;
+; OPTM1:      .globl testBool
+; Two bool computations
+; OPTM1:      cmp
+; OPTM1:      cmp
+; Test first bool
+; OPTM1:      cmp
+; OPTM1:      call
+; Test second bool
+; OPTM1:      cmp
+; OPTM1:      call
+; OPTM1:      ret
+
+; ERRORS-NOT: ICE translation error
 ; DUMP-NOT: SZ
diff --git a/tests_lit/llvm2ice_tests/convert.ll b/tests_lit/llvm2ice_tests/convert.ll
index b5655cc..1b61db3 100644
--- a/tests_lit/llvm2ice_tests/convert.ll
+++ b/tests_lit/llvm2ice_tests/convert.ll
@@ -1,5 +1,8 @@
-; RUIN: %llvm2ice %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Simple test of signed and unsigned integer conversions.
+
+; RUIN: %llvm2ice -O2 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice -Om1 --verbose none %s | FileCheck --check-prefix=OPTM1 %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -27,16 +30,27 @@
   %__7 = bitcast [8 x i8]* @i64v to i64*
   store i64 %v3, i64* %__7, align 1
   ret void
-  ; CHECK: mov al, byte ptr [
-  ; CHECK-NEXT: movsx cx, al
-  ; CHECK-NEXT: mov word ptr [
-  ; CHECK-NEXT: movsx ecx, al
-  ; CHECK-NEXT: mov dword ptr [
-  ; CHECK-NEXT: movsx ecx, al
-  ; CHECK-NEXT: sar eax, 31
-  ; CHECK-NEXT: mov dword ptr [i64v+4],
-  ; CHECK-NEXT: mov dword ptr [i64v],
 }
+; CHECK: from_int8:
+; CHECK: mov al, byte ptr [
+; CHECK-NEXT: movsx cx, al
+; CHECK-NEXT: mov word ptr [
+; CHECK-NEXT: movsx ecx, al
+; CHECK-NEXT: mov dword ptr [
+; CHECK-NEXT: movsx ecx, al
+; CHECK-NEXT: sar eax, 31
+; CHECK-NEXT: mov dword ptr [i64v+4],
+; CHECK-NEXT: mov dword ptr [i64v],
+;
+; OPTM1: from_int8:
+; OPTM1: mov {{.*}}, byte ptr [
+; OPTM1: movsx
+; OPTM1: mov word ptr [
+; OPTM1: movsx
+; OPTM1: mov dword ptr [
+; OPTM1: movsx
+; OPTM1: sar {{.*}}, 31
+; OPTM1: i64v
 
 define void @from_int16() {
 entry:
@@ -52,16 +66,26 @@
   %__7 = bitcast [8 x i8]* @i64v to i64*
   store i64 %v3, i64* %__7, align 1
   ret void
-  ; CHECK: mov ax, word ptr [
-  ; CHECK-NEXT: mov cx, ax
-  ; CHECK-NEXT: mov byte ptr [
-  ; CHECK-NEXT: movsx ecx, ax
-  ; CHECK-NEXT: mov dword ptr [
-  ; CHECK-NEXT: movsx ecx, ax
-  ; CHECK-NEXT: sar eax, 31
-  ; CHECK-NEXT: mov dword ptr [i64v+4],
-  ; CHECK-NEXT: mov dword ptr [i64v],
 }
+; CHECK: from_int16:
+; CHECK: mov ax, word ptr [
+; CHECK-NEXT: mov cx, ax
+; CHECK-NEXT: mov byte ptr [
+; CHECK-NEXT: movsx ecx, ax
+; CHECK-NEXT: mov dword ptr [
+; CHECK-NEXT: movsx ecx, ax
+; CHECK-NEXT: sar eax, 31
+; CHECK-NEXT: mov dword ptr [i64v+4],
+; CHECK-NEXT: mov dword ptr [i64v],
+;
+; OPTM1: from_int16:
+; OPTM1: mov {{.*}}, word ptr [
+; OPTM1: i8v
+; OPTM1: movsx
+; OPTM1: i32v
+; OPTM1: movsx
+; OPTM1: sar {{.*}}, 31
+; OPTM1: i64v
 
 define void @from_int32() {
 entry:
@@ -77,16 +101,24 @@
   %__7 = bitcast [8 x i8]* @i64v to i64*
   store i64 %v3, i64* %__7, align 1
   ret void
-  ; CHECK: mov eax, dword ptr [
-  ; CHECK-NEXT: mov ecx, eax
-  ; CHECK-NEXT: mov byte ptr [
-  ; CHECK-NEXT: mov ecx, eax
-  ; CHECK-NEXT: mov word ptr [
-  ; CHECK-NEXT: mov ecx, eax
-  ; CHECK-NEXT: sar eax, 31
-  ; CHECK-NEXT: mov dword ptr [i64v+4],
-  ; CHECK-NEXT: mov dword ptr [i64v],
 }
+; CHECK: from_int32:
+; CHECK: mov eax, dword ptr [
+; CHECK-NEXT: mov ecx, eax
+; CHECK-NEXT: mov byte ptr [
+; CHECK-NEXT: mov ecx, eax
+; CHECK-NEXT: mov word ptr [
+; CHECK-NEXT: mov ecx, eax
+; CHECK-NEXT: sar eax, 31
+; CHECK-NEXT: mov dword ptr [i64v+4],
+; CHECK-NEXT: mov dword ptr [i64v],
+;
+; OPTM1: from_int32:
+; OPTM1: i32v
+; OPTM1: i8v
+; OPTM1: i16v
+; OPTM1: sar {{.*}}, 31
+; OPTM1: i64v
 
 define void @from_int64() {
 entry:
@@ -102,13 +134,20 @@
   %__7 = bitcast [4 x i8]* @i32v to i32*
   store i32 %v3, i32* %__7, align 1
   ret void
-  ; CHECK: mov eax, dword ptr [
-  ; CHECK-NEXT: mov ecx, eax
-  ; CHECK-NEXT: mov byte ptr [
-  ; CHECK-NEXT: mov ecx, eax
-  ; CHECK-NEXT: mov word ptr [
-  ; CHECK-NEXT: mov dword ptr [
 }
+; CHECK: from_int64:
+; CHECK: mov eax, dword ptr [
+; CHECK-NEXT: mov ecx, eax
+; CHECK-NEXT: mov byte ptr [
+; CHECK-NEXT: mov ecx, eax
+; CHECK-NEXT: mov word ptr [
+; CHECK-NEXT: mov dword ptr [
+;
+; OPTM1: from_int64:
+; OPTM1: i64v
+; OPTM1: i8v
+; OPTM1: i16v
+; OPTM1: i32v
 
 define void @from_uint8() {
 entry:
@@ -124,16 +163,27 @@
   %__7 = bitcast [8 x i8]* @i64v to i64*
   store i64 %v3, i64* %__7, align 1
   ret void
-  ; CHECK: mov al, byte ptr [
-  ; CHECK-NEXT: movzx cx, al
-  ; CHECK-NEXT: mov word ptr [
-  ; CHECK-NEXT: movzx ecx, al
-  ; CHECK-NEXT: mov dword ptr [
-  ; CHECK-NEXT: movzx eax, al
-  ; CHECK-NEXT: mov ecx, 0
-  ; CHECK-NEXT: mov dword ptr [i64v+4],
-  ; CHECK-NEXT: mov dword ptr [i64v],
 }
+; CHECK: from_uint8:
+; CHECK: mov al, byte ptr [
+; CHECK-NEXT: movzx cx, al
+; CHECK-NEXT: mov word ptr [
+; CHECK-NEXT: movzx ecx, al
+; CHECK-NEXT: mov dword ptr [
+; CHECK-NEXT: movzx eax, al
+; CHECK-NEXT: mov ecx, 0
+; CHECK-NEXT: mov dword ptr [i64v+4],
+; CHECK-NEXT: mov dword ptr [i64v],
+;
+; OPTM1: from_uint8:
+; OPTM1: u8v
+; OPTM1: movzx
+; OPTM1: i16v
+; OPTM1: movzx
+; OPTM1: i32v
+; OPTM1: movzx
+; OPTM1: mov {{.*}}, 0
+; OPTM1: i64v
 
 define void @from_uint16() {
 entry:
@@ -149,16 +199,26 @@
   %__7 = bitcast [8 x i8]* @i64v to i64*
   store i64 %v3, i64* %__7, align 1
   ret void
-  ; CHECK: mov ax, word ptr [
-  ; CHECK-NEXT: mov cx, ax
-  ; CHECK-NEXT: mov byte ptr [
-  ; CHECK-NEXT: movzx ecx, ax
-  ; CHECK-NEXT: mov dword ptr [
-  ; CHECK-NEXT: movzx eax, ax
-  ; CHECK-NEXT: mov ecx, 0
-  ; CHECK-NEXT: mov dword ptr [i64v+4],
-  ; CHECK-NEXT: mov dword ptr [i64v],
 }
+; CHECK: from_uint16:
+; CHECK: mov ax, word ptr [
+; CHECK-NEXT: mov cx, ax
+; CHECK-NEXT: mov byte ptr [
+; CHECK-NEXT: movzx ecx, ax
+; CHECK-NEXT: mov dword ptr [
+; CHECK-NEXT: movzx eax, ax
+; CHECK-NEXT: mov ecx, 0
+; CHECK-NEXT: mov dword ptr [i64v+4],
+; CHECK-NEXT: mov dword ptr [i64v],
+;
+; OPTM1: from_uint16:
+; OPTM1: u16v
+; OPTM1: i8v
+; OPTM1: movzx
+; OPTM1: i32v
+; OPTM1: movzx
+; OPTM1: mov {{.*}}, 0
+; OPTM1: i64v
 
 define void @from_uint32() {
 entry:
@@ -174,15 +234,23 @@
   %__7 = bitcast [8 x i8]* @i64v to i64*
   store i64 %v3, i64* %__7, align 1
   ret void
-  ; CHECK: mov eax, dword ptr [
-  ; CHECK-NEXT: mov ecx, eax
-  ; CHECK-NEXT: mov byte ptr [
-  ; CHECK-NEXT: mov ecx, eax
-  ; CHECK-NEXT: mov word ptr [
-  ; CHECK-NEXT: mov ecx, 0
-  ; CHECK-NEXT: mov dword ptr [i64v+4],
-  ; CHECK-NEXT: mov dword ptr [i64v],
 }
+; CHECK: from_uint32:
+; CHECK: mov eax, dword ptr [
+; CHECK-NEXT: mov ecx, eax
+; CHECK-NEXT: mov byte ptr [
+; CHECK-NEXT: mov ecx, eax
+; CHECK-NEXT: mov word ptr [
+; CHECK-NEXT: mov ecx, 0
+; CHECK-NEXT: mov dword ptr [i64v+4],
+; CHECK-NEXT: mov dword ptr [i64v],
+;
+; OPTM1: from_uint32:
+; OPTM1: u32v
+; OPTM1: i8v
+; OPTM1: i16v
+; OPTM1: mov {{.*}}, 0
+; OPTM1: i64v
 
 define void @from_uint64() {
 entry:
@@ -198,13 +266,20 @@
   %__7 = bitcast [4 x i8]* @i32v to i32*
   store i32 %v3, i32* %__7, align 1
   ret void
-  ; CHECK: mov eax, dword ptr [
-  ; CHECK-NEXT: mov ecx, eax
-  ; CHECK-NEXT: mov byte ptr [
-  ; CHECK-NEXT: mov ecx, eax
-  ; CHECK-NEXT: mov word ptr [
-  ; CHECK-NEXT: mov dword ptr [
 }
+; CHECK: from_uint64:
+; CHECK: mov eax, dword ptr [
+; CHECK-NEXT: mov ecx, eax
+; CHECK-NEXT: mov byte ptr [
+; CHECK-NEXT: mov ecx, eax
+; CHECK-NEXT: mov word ptr [
+; CHECK-NEXT: mov dword ptr [
+;
+; OPTM1: from_uint64:
+; OPTM1: u64v
+; OPTM1: i8v
+; OPTM1: i16v
+; OPTM1: i32v
 
 ; ERRORS-NOT: ICE translation error
 ; DUMP-NOT: SZ
diff --git a/tests_lit/llvm2ice_tests/empty-func.ll b/tests_lit/llvm2ice_tests/empty-func.ll
index 98367bc..32d1704 100644
--- a/tests_lit/llvm2ice_tests/empty-func.ll
+++ b/tests_lit/llvm2ice_tests/empty-func.ll
@@ -1,5 +1,7 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Trivial test of a trivial function.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/fp.pnacl.ll b/tests_lit/llvm2ice_tests/fp.pnacl.ll
index a9951d9..c31725a 100644
--- a/tests_lit/llvm2ice_tests/fp.pnacl.ll
+++ b/tests_lit/llvm2ice_tests/fp.pnacl.ll
@@ -1,5 +1,11 @@
-; RUIN: %llvm2ice --verbose none %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; This tries to be a comprehensive test of f32 and f64 operations.
+; The CHECK lines are only checking for basic instruction patterns
+; that should be present regardless of the optimization level, so
+; there are no special OPTM1 match lines.
+
+; RUIN: %llvm2ice -O2 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice -Om1 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/fpconst.pnacl.ll b/tests_lit/llvm2ice_tests/fpconst.pnacl.ll
index 1e40f76..6ca41e4 100644
--- a/tests_lit/llvm2ice_tests/fpconst.pnacl.ll
+++ b/tests_lit/llvm2ice_tests/fpconst.pnacl.ll
@@ -1,7 +1,3 @@
-; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
-; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
-; RUN:                           | FileCheck --check-prefix=DUMP %s
-
 ; This is a smoke test for floating-point constant pooling.  It tests
 ; pooling of various float and double constants (including positive
 ; and negative NaN) within functions and across functions.  Note that
@@ -10,6 +6,11 @@
 ; number in a reasonable number of digits".  See
 ; http://llvm.org/docs/LangRef.html#simple-constants .
 
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
+; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
+; RUN:                           | FileCheck --check-prefix=DUMP %s
+
 @__init_array_start = internal constant [0 x i8] zeroinitializer, align 4
 @__fini_array_start = internal constant [0 x i8] zeroinitializer, align 4
 @__tls_template_start = internal constant [0 x i8] zeroinitializer, align 8
diff --git a/tests_lit/llvm2ice_tests/global.ll b/tests_lit/llvm2ice_tests/global.ll
index 00ed03e..ba5471e 100644
--- a/tests_lit/llvm2ice_tests/global.ll
+++ b/tests_lit/llvm2ice_tests/global.ll
@@ -1,5 +1,8 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Trivial test of the use of internal versus external global
+; functions.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 
 ; Note: We don't run this test using a PNaCl bitcode file, because
diff --git a/tests_lit/llvm2ice_tests/icmp-simple.ll b/tests_lit/llvm2ice_tests/icmp-simple.ll
index dc97a9b..f6ca8d5 100644
--- a/tests_lit/llvm2ice_tests/icmp-simple.ll
+++ b/tests_lit/llvm2ice_tests/icmp-simple.ll
@@ -1,5 +1,7 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Trivial structural test of 64-bit icmp instructions.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/load.ll b/tests_lit/llvm2ice_tests/load.ll
index 1a1d6a3..6b7b77f 100644
--- a/tests_lit/llvm2ice_tests/load.ll
+++ b/tests_lit/llvm2ice_tests/load.ll
@@ -1,5 +1,7 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Simple test of the load instruction.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -10,7 +12,8 @@
   %iv = load i64* %__1, align 1
   ret void
 
-; CHECK:      %__1 = i32 %addr_arg
+; CHECK:       Initial CFG
+; CHECK:       %__1 = i32 %addr_arg
 ; CHECK-NEXT:  %iv = load i64* {{.*}}, align 1
 ; CHECK-NEXT:  ret void
 }
@@ -21,6 +24,7 @@
   %iv = load i32* %__1, align 1
   ret void
 
+; CHECK:       Initial CFG
 ; CHECK:       %__1 = i32 %addr_arg
 ; CHECK-NEXT:  %iv = load i32* {{.*}}, align 1
 ; CHECK-NEXT:  ret void
@@ -32,6 +36,7 @@
   %iv = load i16* %__1, align 1
   ret void
 
+; CHECK:       Initial CFG
 ; CHECK:       %__1 = i32 %addr_arg
 ; CHECK-NEXT:  %iv = load i16* {{.*}}, align 1
 ; CHECK-NEXT:  ret void
@@ -43,6 +48,7 @@
   %iv = load i8* %__1, align 1
   ret void
 
+; CHECK:       Initial CFG
 ; CHECK:       %__1 = i32 %addr_arg
 ; CHECK-NEXT:  %iv = load i8* {{.*}}, align 1
 ; CHECK-NEXT:  ret void
diff --git a/tests_lit/llvm2ice_tests/mangle.ll b/tests_lit/llvm2ice_tests/mangle.ll
new file mode 100644
index 0000000..c76a23d
--- /dev/null
+++ b/tests_lit/llvm2ice_tests/mangle.ll
@@ -0,0 +1,105 @@
+; Tests the Subzero "name mangling" when using the "llvm2ice --prefix"
+; option.
+
+; RUN: %llvm2ice --verbose none %s | FileCheck %s
+; RUN: %llvm2ice --verbose none --prefix Subzero %s | FileCheck --check-prefix=MANGLE %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
+; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
+; RUN:                           | FileCheck --check-prefix=DUMP %s
+
+define internal void @FuncC(i32 %i) {
+entry:
+  ret void
+}
+; FuncC is a C symbol that isn't recognized as a C++ mangled symbol.
+; CHECK: FuncC:
+; MANGLE: SubzeroFuncC
+
+define internal void @_ZN13TestNamespace4FuncEi(i32 %i) {
+entry:
+  ret void
+}
+; This is Func(int) nested inside namespace TestNamespace.
+; CHECK: _ZN13TestNamespace4FuncEi:
+; MANGLE: _ZN7Subzero13TestNamespace4FuncEi:
+
+define internal void @_ZN13TestNamespace15NestedNamespace4FuncEi(i32 %i) {
+entry:
+  ret void
+}
+; This is Func(int) nested inside two namespaces.
+; CHECK: _ZN13TestNamespace15NestedNamespace4FuncEi:
+; MANGLE: _ZN7Subzero13TestNamespace15NestedNamespace4FuncEi:
+
+define internal void @_Z13FuncCPlusPlusi(i32 %i) {
+entry:
+  ret void
+}
+; This is a non-nested, mangled C++ symbol.
+; CHECK: _Z13FuncCPlusPlusi:
+; MANGLE: _ZN7Subzero13FuncCPlusPlusEi:
+
+define internal void @_ZN12_GLOBAL__N_18FuncAnonEi(i32 %i) {
+entry:
+  ret void
+}
+; This is FuncAnon(int) nested inside an anonymous namespace.
+; CHECK: _ZN12_GLOBAL__N_18FuncAnonEi:
+; MANGLE: _ZN7Subzero12_GLOBAL__N_18FuncAnonEi:
+
+; Now for the illegitimate examples.
+
+; Test for _ZN with no suffix.  Don't crash, prepend Subzero.
+define internal void @_ZN(i32 %i) {
+entry:
+  ret void
+}
+; MANGLE: Subzero_ZN:
+
+; Test for _Z<len><str> where <len> is smaller than it should be.
+define internal void @_Z12FuncCPlusPlusi(i32 %i) {
+entry:
+  ret void
+}
+; MANGLE: _ZN7Subzero12FuncCPlusPluEsi:
+
+; Test for _Z<len><str> where <len> is slightly larger than it should be.
+define internal void @_Z14FuncCPlusPlusi(i32 %i) {
+entry:
+  ret void
+}
+; MANGLE: _ZN7Subzero14FuncCPlusPlusiE:
+
+; Test for _Z<len><str> where <len> is much larger than it should be.
+define internal void @_Z114FuncCPlusPlusi(i32 %i) {
+entry:
+  ret void
+}
+; MANGLE: Subzero_Z114FuncCPlusPlusi:
+
+; Test for _Z<len><str> where we try to overflow the uint32_t holding <len>.
+define internal void @_Z4294967296FuncCPlusPlusi(i32 %i) {
+entry:
+  ret void
+}
+; MANGLE: Subzero_Z4294967296FuncCPlusPlusi:
+
+; Test for _Z<len><str> where <len> is 0.
+define internal void @_Z0FuncCPlusPlusi(i32 %i) {
+entry:
+  ret void
+}
+; MANGLE: _ZN7Subzero0EFuncCPlusPlusi:
+
+; Test for _Z<len><str> where <len> is -1.  LLVM explicitly allows the
+; '-' character in identifiers.
+
+define internal void @_Z-1FuncCPlusPlusi(i32 %i) {
+entry:
+  ret void
+}
+; MANGLE: Subzero_Z-1FuncCPlusPlusi:
+
+; ERRORS-NOT: ICE translation error
+; DUMP-NOT: SZ
diff --git a/tests_lit/llvm2ice_tests/return-int-arg.ll b/tests_lit/llvm2ice_tests/return-int-arg.ll
index 1e2d8b2..67e7715 100644
--- a/tests_lit/llvm2ice_tests/return-int-arg.ll
+++ b/tests_lit/llvm2ice_tests/return-int-arg.ll
@@ -1,5 +1,7 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Simple test of functions returning one of its arguments.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/select-opt.ll b/tests_lit/llvm2ice_tests/select-opt.ll
index 9bb9701..c0358fb 100644
--- a/tests_lit/llvm2ice_tests/select-opt.ll
+++ b/tests_lit/llvm2ice_tests/select-opt.ll
@@ -1,5 +1,11 @@
-; RUIN: %llvm2ice %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Simple test of the select instruction.  The CHECK lines are only
+; checking for basic instruction patterns that should be present
+; regardless of the optimization level, so there are no special OPTM1
+; match lines.
+
+; RUIN: %llvm2ice -O2 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice -Om1 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/shift.ll b/tests_lit/llvm2ice_tests/shift.ll
index 674f4db..c1a071f 100644
--- a/tests_lit/llvm2ice_tests/shift.ll
+++ b/tests_lit/llvm2ice_tests/shift.ll
@@ -1,5 +1,9 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; This is a test of C-level conversion operations that clang lowers
+; into pairs of shifts.
+
+; RUIN: %llvm2ice -O2 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice -Om1 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -17,9 +21,10 @@
   %__4 = bitcast [4 x i8]* @i1 to i32*
   store i32 %v1, i32* %__4, align 1
   ret void
-  ; CHECK: shl eax, 24
-  ; CHECK-NEXT: sar eax, 24
 }
+; CHECK: conv1:
+; CHECK: shl {{.*}}, 24
+; CHECK: sar {{.*}}, 24
 
 define void @conv2() {
 entry:
@@ -30,9 +35,10 @@
   %__4 = bitcast [4 x i8]* @i2 to i32*
   store i32 %v1, i32* %__4, align 1
   ret void
-  ; CHECK: shl eax, 16
-  ; CHECK-NEXT: sar eax, 16
 }
+; CHECK: conv2:
+; CHECK: shl {{.*}}, 16
+; CHECK: sar {{.*}}, 16
 
 ; ERRORS-NOT: ICE translation error
 ; DUMP-NOT: SZ
diff --git a/tests_lit/llvm2ice_tests/simple-arith.ll b/tests_lit/llvm2ice_tests/simple-arith.ll
deleted file mode 100644
index 8b32a94..0000000
--- a/tests_lit/llvm2ice_tests/simple-arith.ll
+++ /dev/null
@@ -1,35 +0,0 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
-; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
-; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
-; RUN:                           | FileCheck --check-prefix=DUMP %s
-
-define i64 @add_args_i64(i64 %arg1, i64 %arg2) {
-entry:
-  %add = add i64 %arg2, %arg1
-  ret i64 %add
-}
-
-; Checks for verbose instruction output
-
-; CHECK: define i64 @add_args
-; CHECK: %add = add i64 %arg2, %arg1
-; CHECK-NEXT: ret i64 %add
-
-define i32 @add_args_i32(i32 %arg1, i32 %arg2) {
-entry:
-  %add = add i32 %arg2, %arg1
-  ret i32 %add
-}
-
-; Checks for emitted assembly
-
-; CHECK:      .globl add_args_i32
-; CHECK:      mov eax, dword ptr [esp+4]
-; CHECK-NEXT: mov ecx, dword ptr [esp+8]
-; CHECK-NEXT: add ecx, eax
-; CHECK-NEXT: mov eax, ecx
-; CHECK-NEXT: ret
-
-; ERRORS-NOT: ICE translation error
-; DUMP-NOT: SZ
diff --git a/tests_lit/llvm2ice_tests/simple-cond.ll b/tests_lit/llvm2ice_tests/simple-cond.ll
deleted file mode 100644
index 2161a59..0000000
--- a/tests_lit/llvm2ice_tests/simple-cond.ll
+++ /dev/null
@@ -1,32 +0,0 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
-; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
-; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
-; RUN:                           | FileCheck --check-prefix=DUMP %s
-
-define internal i32 @simple_cond(i32 %a, i32 %n) {
-entry:
-  %cmp = icmp slt i32 %n, 0
-; CHECK:  %cmp = icmp slt i32 %n, 0
-  br i1 %cmp, label %if.then, label %if.else
-; CHECK-NEXT:  br i1 %cmp, label %if.then, label %if.else
-
-if.then:
-  %sub = sub i32 1, %n
-  br label %if.end
-
-if.else:
-  %gep_array = mul i32 %n, 4
-  %gep = add i32 %a, %gep_array
-  %__6 = inttoptr i32 %gep to i32*
-  %v0 = load i32* %__6, align 1
-  br label %if.end
-
-if.end:
-  %result.0 = phi i32 [ %sub, %if.then ], [ %v0, %if.else ]
-; CHECK: %result.0 = phi i32 [ %sub, %if.then ], [ %v0, %if.else ]
-  ret i32 %result.0
-}
-
-; ERRORS-NOT: ICE translation error
-; DUMP-NOT: SZ
diff --git a/tests_lit/llvm2ice_tests/simple-loop.ll b/tests_lit/llvm2ice_tests/simple-loop.ll
index a35e5b3..b983d14c 100644
--- a/tests_lit/llvm2ice_tests/simple-loop.ll
+++ b/tests_lit/llvm2ice_tests/simple-loop.ll
@@ -1,5 +1,9 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; This tests a simple loop that sums the elements of an input array.
+; The O2 check patterns represent the best code currently achieved.
+
+; RUIN: %llvm2ice -O2 --verbose none %s | FileCheck %s
+; RUN: %llvm2ice -Om1 --verbose none %s | FileCheck --check-prefix=OPTM1 %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -26,17 +30,7 @@
   ret i32 %sum.0.lcssa
 }
 
-; Checks for verbose instruction output
-
-; CHECK:  br i1 %cmp4, label %for.body, label %for.end
-; CHECK-NEXT: for.body
-; CHECK:  %i.06 = phi i32 [ %inc, %for.body ], [ 0, %entry ]
-; CHECK-NEXT:  %sum.05 = phi i32 [ %add, %for.body ], [ 0, %entry ]
-
-; Checks for emitted assembly
-
 ; CHECK:      .globl simple_loop
-
 ; CHECK:      mov ecx, dword ptr [esp+{{[0-9]+}}]
 ; CHECK:      cmp ecx, 0
 ; CHECK-NEXT: jg {{.*}}for.body
@@ -49,6 +43,13 @@
 ; CHECK-NEXT: mov [[ICMPREG:[a-z]+]], [[IREG]]
 ; CHECK:      cmp [[ICMPREG]], ecx
 ; CHECK-NEXT: jl {{.*}}for.body
+;
+; There's nothing remarkable under Om1 to test for, since Om1 generates
+; such atrocious code (by design).
+; OPTM1:      .globl simple_loop
+; OPTM1:      cmp {{.*}}, 0
+; OPTM1:      jg
+; OPTM1:      ret
 
 ; ERRORS-NOT: ICE translation error
 ; DUMP-NOT: SZ
diff --git a/tests_lit/llvm2ice_tests/store.ll b/tests_lit/llvm2ice_tests/store.ll
index 59e1c2b..789a1de 100644
--- a/tests_lit/llvm2ice_tests/store.ll
+++ b/tests_lit/llvm2ice_tests/store.ll
@@ -1,5 +1,7 @@
-; RUIN: %llvm2ice %s -verbose inst | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; Simple test of the store instruction.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -10,6 +12,7 @@
   store i64 1, i64* %__1, align 1
   ret void
 
+; CHECK:       Initial CFG
 ; CHECK:       %__1 = i32 %addr_arg
 ; CHECK-NEXT:  store i64 1, {{.*}}, align 1
 ; CHECK-NEXT:  ret void
@@ -21,6 +24,7 @@
   store i32 1, i32* %__1, align 1
   ret void
 
+; CHECK:       Initial CFG
 ; CHECK:       %__1 = i32 %addr_arg
 ; CHECK-NEXT:  store i32 1, {{.*}}, align 1
 ; CHECK-NEXT:  ret void
@@ -32,6 +36,7 @@
   store i16 1, i16* %__1, align 1
   ret void
 
+; CHECK:       Initial CFG
 ; CHECK:       %__1 = i32 %addr_arg
 ; CHECK-NEXT:  store i16 1, {{.*}}, align 1
 ; CHECK-NEXT:  ret void
@@ -43,6 +48,7 @@
   store i8 1, i8* %__1, align 1
   ret void
 
+; CHECK:       Initial CFG
 ; CHECK:       %__1 = i32 %addr_arg
 ; CHECK-NEXT:  store i8 1, {{.*}}, align 1
 ; CHECK-NEXT:  ret void
diff --git a/tests_lit/llvm2ice_tests/struct-arith.pnacl.ll b/tests_lit/llvm2ice_tests/struct-arith.pnacl.ll
index 3e88b0f..da922d0 100644
--- a/tests_lit/llvm2ice_tests/struct-arith.pnacl.ll
+++ b/tests_lit/llvm2ice_tests/struct-arith.pnacl.ll
@@ -1,14 +1,12 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; This test is lowered from C code that does some simple aritmetic
+; with struct members.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
 
-; This file is lowered from C code that does some simple aritmetic with
-; struct members. It's also built with the PNaCl toolchain so this is the
-; stable ABI subset of LLVM IR (structs are gone, pointers turned into i32,
-; geps gone, etc.)
-
 define internal i32 @compute_important_function(i32 %v1, i32 %v2) {
 entry:
   %__2 = inttoptr i32 %v1 to i32*
diff --git a/tests_lit/llvm2ice_tests/switch-opt.ll b/tests_lit/llvm2ice_tests/switch-opt.ll
index 827dae7..19187e4 100644
--- a/tests_lit/llvm2ice_tests/switch-opt.ll
+++ b/tests_lit/llvm2ice_tests/switch-opt.ll
@@ -1,5 +1,9 @@
-; RUIN: %llvm2ice %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; This tests a switch statement, including multiple branches to the
+; same label which also results in phi instructions with multiple
+; entries for the same incoming edge.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
diff --git a/tests_lit/llvm2ice_tests/unreachable.ll b/tests_lit/llvm2ice_tests/unreachable.ll
index 25a3dd6..ab54704 100644
--- a/tests_lit/llvm2ice_tests/unreachable.ll
+++ b/tests_lit/llvm2ice_tests/unreachable.ll
@@ -1,5 +1,7 @@
-; RUIN: %llvm2ice -verbose inst %s | FileCheck %s
-; RUIN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
+; This tests the basic structure of the Unreachable instruction.
+
+; RUN: %llvm2ice --verbose inst %s | FileCheck %s
+; RUN: %llvm2ice --verbose none %s | FileCheck --check-prefix=ERRORS %s
 ; RUN: %llvm2iceinsts %s | %szdiff %s | FileCheck --check-prefix=DUMP %s
 ; RUN: %llvm2iceinsts --pnacl %s | %szdiff %s \
 ; RUN:                           | FileCheck --check-prefix=DUMP %s
@@ -11,6 +13,7 @@
 
 abort:                                            ; preds = %entry
   unreachable
+; CHECK: unreachable
 
 return:                                           ; preds = %entry
   %div = sdiv i32 %num, %den