Google Git
Sign in
swiftshader / SwiftShader / b73b76017789049c23b27c681ffde654d6c04918
commitb73b76017789049c23b27c681ffde654d6c04918[log] [tgz]
authorBen Clayton <bclayton@google.com>Mon Jul 29 13:56:13 2019 +0100
committerBen Clayton <bclayton@google.com>Mon Jul 29 13:56:13 2019 +0100
tree67ee2c5d4a58ba10cde6f5718a5d80010eda9ff4
parentcc5697f9685479191c012cb919c97f5eedd1e041 [diff]
Squashed 'third_party/SPIRV-Tools/' changes from d14db341b..9559cdbdf

9559cdbdf Fix #2609 - Handle out-of-bounds scalar replacements. (#2767)
f54b8653d Limit fuzzer tests so that they take less time to run (#2763)
bb0e2f65b Fix check for unreachable blocks in merge-return (#2762)
1a89ac8b2 Transformation and fuzzer pass to add dead continues (#2758)
65f49dfc3 Remove unneeded future imports (#2739)
c7fcb8c3b Process OpDecorateId in ADCE (#2761)
fb83b6fbb Record correct dominators in merge return (#2760)
c9190a54d SSA rewriter: Don't use trivial phis (#2757)
aea4e6b1b Fix block depth rule priority (#2755)
a94ddc267 Case validation with repeated labels (#2689)
3855447d9 Bindless Instrument: Make init check depend solely on input_init_enabled (#2753)
11516c0b9 Validate storage class OpenCL environment rules for atomics (#2750)
bac82f49a Allow LOD ops in compute shaders with derivative group execution modes (#2752)
76b75c40a Document opt::Instruction::InsertBefore methods (#2751)
aa9e8f538 Revert "Do not inline OpKill Instructions (#2713)" (#2749)
58e2ec25b For Vulkan, disallow structures containing opaque types (#2546)
230c9e437 Fix bug in merge return (#2734)
1fedf72e5 Allow ray tracing shaders in inst bindle check pass. (#2733)
032adc4d7 Correctly implement WebGPU related flag exclusions (#2737)
92c41ff1e Remove Common Uniform Elimination Pass (#2731)
59de04ad6 BUILD.gn: Add deps and move files for `gn check` (#2735)
55adf4cf7 Update execution scope rules for WebGPU (#2730)
1a2de48a1 Extra small storage validation (#2732)
327963765 Add validation for SPV_EXT_demote_to_helper_invocation (#2707)
508151250 BUILD.gn: Add targets to build all command-line tools (#2727)
5ce8cf781 Change the order branches are simplified in dead branch elim (#2728)
cd153db8e Add —preserve-bindings and —preserve-spec-constants (#2693)
86e45efe1 Handle decorations better in some optimizations (#2716)
3a252a267 Update memory scope rules for WebGPU (#2725)
0c4feb643 Remove extra semis (#2717)
456cc598a Validate usage of 8- and 16-bit types with only storage capabilities (#2704)
b8ab80843 Shrinker for spirv-fuzz (#2708)
37e8f7994 Perform merge return with single return in loop. (#2714)
fe7cc9c61 Do not inline OpKill Instructions (#2713)
5a93e0739 Refactor reducer options (#2709)
a6bfc26e5 Fix BUILD.gn for Fuchsia platform build. (#2692)
9702d47c6 Validate that in OpenGL env block variables have Binding (#2685)
e6e3e2ccc Update type for loaded builtin GlobalInvocationID in pass instrumentation (#2705)
6ccb52b86 Warn when input facts are invalid. (#2699)
88183041d Got rid of redundant declaration. (#2698)
efde68236 Disallow movement of unreachable blocks. (#2700)
dfcb5a1e1 Refactor fuzzer transformations (#2694)
888aeef8a Fix Component decoration validation for arrays (#2697)
df86bb44f Replace global static map with an array of pairs (#2691)
7c294608c Basic validation for Component decorations (#2679)
69b945992 Add infrastructure for maintaining and using DEPS (#2684)
2b84d25f1 Fix store to uniform Vulkan check (#2688)
bec7e0393 Add all accepted target environments to the tools' help texts (#2687)
51b0d5ce5 Represent uniform facts via descriptor set and binding. (#2681)
fa981bc24 Roll external/spirv-headers/ 8b911bd2b..de99d4d83 (8 commits) (#2682)
a132c9b64 Whitelist SPV_GOOGLE_user_type. (#2673)
001e823b6 Add fuzzer pass to obfuscate constants. (#2671)
2090d7a2d Handle volatile memory semantics in upgrade (#2674)
3d5fb7b90 Validate Volatile memory semantics bit (#2672)
400dbde0b Disallow stores to UBOs (#2651)
6cc2c8f4a Another fix uint -> uint32_t (#2676)
59983a601 Validate variable initializer type (#2668)
9477c91de Fix uint -> uint32_t in fuzz.cpp (#2675)
42830e5a6 Add replayer tool for spirv-fuzz. (#2664)
b4bf7bcf0 Add validation for Subgroup builtins (#2637)
9c0830133 Add constant == uniform facts. (#2660)
208d3132e Cast __LINE__ to size_t (#2661)
a8ae579f7 Add transformation to replace a boolean constant with a numeric comparison (#2659)
0755d6ce8 Add builtin validation for SPV_NV_shader_sm_builtins (#2656)
43fb2403a Instrument: Fix code for version 2 output format. (#2655)
08cc49ec5 Fix bug in 'split blocks', and add tests for fuzzer. (#2658)
d01a3c3b4 Optimizer: Handle array type with OpSpecConstantOp length (#2652)
4a00a80c4 Add fuzzer pass to add dead breaks. (#2654)
620197bd6 Add fuzzer pass that adds useful constructs to a module (#2647)
2c0111e6e Add validation for SPV_EXT_fragment_shader_interlock (#2650)
699e167d7 Remove asserts from GetUnderlyingType (#2646)
7919b877c Close opened file handles. (#2644)
f99d7ad5c Validate OpenCL rules for ImageRead and OpImageSampleExplicitLod (#2643)
209ff0ce9 Add spirv-fuzz pass to permute blocks. (#2642)
e7866de4b Linker: Better type comparison for OpTypeArray and OpTypeForwardPointer (#2580)
0125b28ed Add compact ids to WebGPU <-> Vulkan transformations (#2639)
3d62cb814 Instrument: Add version 2 of record formats (#2630)
1b71e4533 Add "split block" transformation. (#2633)
f05181234 Add WebGPU specific fuzzer for validation (#2628)
5a06fa466 Add fuzzer for Vulkan->WebGPU spirv-opt passes (#2626)
78b2b1866  Add fuzzer for WebGPU->Vulkan spirv-opt passes (#2625)
6c7db9c63 Handle nested breaks from switches. (#2624)
37ae8671a Add spirv-fuzz tool. (#2631)
fe9f87013 Add library for spirv-fuzz (#2618)
42abaa099 Remove MarkV and Stats code. (#2576)
3b5ab540c linker: Add tests for various type comparisons (#2613)
4c73ebc46 Enable the base branch to be overridden in the code format check script (#2607)
b8fe7211c Allow arrays of out per-primitive builtins for mesh shaders (#2617)
07a101971 Validate OpenCL environment rules for OpImageWrite (#2619)
b0504239a Added an external dependency on protobufs, included when SPIRV_BUILD_FUZZER is defined, so that they can be used by the (upcoming) spirv-fuzz tool.  Also updated the kokoro build scripts, for relevant targets, to clone an appropriate tag of the protobufs repo, and to pass -DSPIRV_BUILD_FUZZER to the configurations for which we intend to ultimately build spirv-fuzz. (#2616)
4557d0858 Add in individual flags for Vulkan <-> WebGPU passes (#2615)
13f61bf85 Update vloadn and vstoren validation to match the OpenCL Extended Instruction Set Specification (#2599)
d9c00e1d2 Add folding rules for OpQuantizeToF16 (#2614)
713da30b6 Disallow merge targeting block with OpLoopMerge (#2610)
60aaafbc7 Allows breaks selection breaks to switches (#2605)
0982f0212 Using the instruction folder to fold OpSpecConstantOp (#2598)
9f035269d Validate OpenCL environment rules for OpTypeImage (#2606)
47741f050 Validate OpenCL memory and addressing model environment rules (#2589)
ff4feb44b Validate construct exits (#2459)
9dfd4b835 Bindless Validation: Instrument descriptor-based loads and stores (#2583)
7e7745fce Validate loop merge (#2579)
fc7b5d8c6 Mem model spv 1.4 (#2565)
84aa4946e Start SPIRV-Tools v2019.4
ce19e217b Finalize SPIRV-Tools v2019.3
84503583c Handle id overflow in sroa better. (#2582)
f815e6fe5 Update CHANGES
e935dac9e Make pointers to isomorphic type interchangeable with option. (#2570)
2947e88f7 Update instrumentation passes to handle 1.4 interfaces (#2573)
06ce59b0b Instrument: Fix load type of pre-existing builtin (#2575)
87c4ef8a9 Do not fold floating point if float controls used (#2569)
45fb69666 Use last version (#2578)
f6d9a1784 Add pass to fix some invalid unreachable blocks for WebGPU (#2563)
89fe836fe Fix clang-tidy warning about definition/declaration mismatch. (#2571)
f2803c4a7 VK_KHR_uniform_buffer_standard_layout validation (#2562)
cc3e93c4e Add tests for folding 1.4 selects (#2568)
ea5e1b62e Update priv-to-local for SPIR-V 1.4 (#2567)
d0a1f5a05 spvtest::Validate::CompileFailure: Don't leak the diagnostic (#2564)
b74d92a8c ADCE support for SPIR-V 1.4 entry points (#2561)
2b65a71d4 Fix use of 'is' operator for comparison (#2547)
63f57d95d Support SPIR-V 1.4 (#2550)
106c98d0f Validate sign of int types. (#2549)
eef11cdb7 Update README with links to build artifacts. (#2548)
5fc5303ee [spirv] Package and upload builds. (#2544)
6d04da22c Fix up type mismatches. (#2545)
c8b09744c Add validation specific to OpExecutionModeId (#2536)
a5da68d44 Remove stale comment (#2542)
32af42616 Change implementation of post order CFG traversal (#2543)
64faf6d9c Fix undefined bit shift in sroa. (#2532)
dca3ea5e1 fix example.cpp (#2540)
2de3e3c02 Add spirv-lesspipe.sh into SPIRV_SKIP_EXECUTABLES umbrella (https://github.com/KhronosGroup/SPIRV-Tools/issues/2497) (#2504)
fb08224f8 Fix spirv-headers link in the README. (#2516)
b68af7ca8 Add support for Private & Output to initializer decompose flag (#2537)
736376dbf Remove Acquire, Release, and Relaxed from allowed Mem Sem bits for WebGPU (#2526)
07c4dd4b9 Reduce runtime of array layout checks (#2534)
7aad9653f Remove legacy utility functions (#2530)
d754b7059 Shorten names of cmake targets (#2531)
ac878fcbd Remove unreachable block validation (#2525)
21712068f Validate that SPIR-V binary is encoded as little endian for WebGPU (#2523)
3aad3e922 Change validation of memory semantics for OpAtomics* in WebGPU (#2519)
048dcd38c Implement WebGPU->Vulkan initializer conversion for 'Function' variables (#2513)
3335c6114 reduce: Add two branch reduction passes (#2507)
102e430a8 Add pass to legalize OpVectorShuffle for WebGPU (#2509)
07ac7dee5 SPIRV-Tools requires python3 (#2510)
98b3f26c2 Gate formatless checks on Vulkan env (#2486)
2b46521cd Add -fsantize to link flags. (#2506)
82ebbbba1 README: fix formatting of requiring Python 'future' (#2500)
9047de51c Accept OpBitCast in fix storage class. (#2505)
d90aae9a5 reduce: miscellaneous fixes (#2494)
7ce37d66a Fix use of Logf to avoid format security warning (#2498)
0cb2d4079 Add WebGPU->Vulkan and Vulkan->WebGPU flags in spirv-opt (#2496)
9766b22b3 spirv-opt: Behave a bit better in the face of unknown instructions (#2487)
3a0bc9e72 Add fix storage class code. (#2434)
e8c2d95ed Fix webgpu header file name in BUILD.gn (#2493)
26c1b8878 Update CHANGES
236bdc006 Change prioritization of unreachable merge and continue (#2460)
12e4a7b64 Handle variable pointer in some optimizations (#2490)
01964e325 Add pass to generate needed initializers for WebGPU (#2481)
4bd106b08 Handle dead infinite loops in DCE (#2471)
8129cf2f9 Remove merge assert in block calculation (#2489)
e2ddb9371 reduce: add remove_selection_reduction_opportunity (#2485)
c9874e509 Fix merge return in the face of breaks (#2466)
0300a464a Maintain inst to block mapping in merge return (#2469)
320a7de5c Validate that OpUnreacahble is not statically reachable (#2473)
fcb845310 reduce: fix loop to selection pass for loops with combined header/continue block (#2480)
2ff54e34e Handle function decls in Structured CFG analysis (#2474)
42e6f1aa6 Add option to validate after each pass (#2462)
fb0753640 reduce: fix loop to selection dominance query (#2477)
7d1b176c1 Improve reducer algorithm and other changes (#2472)
ffbecae56 Check OpSampledImage is only passed into valid instructions (#2467)
2d52cbee4 Add some val options to reduce (#2401)
1f60f9896 reduce: remove unreferenced blocks pass (#2398)
08b54d9e4 Convert sampled consumers to being Instructions instead of IDs (#2464)
e1a76269b Bindless Validation: Descriptor Initialization Check (#2419)
9244e6ff6 Reverting commit da5a780ff9fff7e226ca84728075adabc4d1608c
da5a780ff Variable pointers cannot be an operand to OpArrayLength
2ac348b5c Repair test for unused storage buffer with descriptor (#2436)
e54552214 Add --strip-atomic-counter-memory (#2413)
bdcb15516 Relax function call parameter check (#2448)
5186ffedb Remove duplicates from list of interface IDs in OpEntryPoint instruction (#2449)
6df8a917a Add validation of storage classes for WebGPU (#2446)
a5c06c903 Validator: no Storage comparison for pointer param (#2428)
9d29c37ac Removing decorations when doing constant propagation. (#2444)
b75f4362f Add validation for ExecutionMode in WebGPU (#2443)
b1ff15f22 Add missing DepthGreater case to Fragment only check (#2440)
b12e7338e Implement WebGPU specific CFG validation (#2386)
a2ef7be24 Add Linux ASAN bot configs. (#2427)
07f80c4df Fix python scripts to work with python3 (#2426)
86f6ac086 Add a test for two back-edges in a continue (#2418)
5fb83a970 Allow NonWritable to target struct members. (#2420)
32b0f6739 Use correct option in spvTextToBinary. (#2416)
d800bbbac Handle back edges better in dead branch elim. (#2417)
002ef361c Add validation for SPV_NV_cooperative_matrix (#2404)
fc3897b5f Validate: (data) Block can't appear within a Block (#2410)
37b584a73 Fixed undefined reference to 'clock_gettime' by linking rt library (#2409)
a006cbc1d Non memory object as parameters. (#2415)
4c43afcad It is invalid to apply both Restrict and Aliased to the same <id> (#2408)
fde69dcd8 Fix OpDot folding of half float vectors. (#2411)
8eddde2e7 Don't change type of input and output var in dead member elim (#2412)
76730a46a In Vulkan, disallow BufferBlock on StorageBuffer variables  (#2380)
5994ae2a0 Start SPIRV-Tools 2019.3
61dfd8492 Finalize SPIRV-Tools 2019.2
bf23ed887  OpAtomicLoad, OpAtomicStore, OpAtomicExchange can operate on int or float value. Except for Vulkan environment that only operates on int value. (#2385)
80496f42a Update CHANGES
2f84b5de9 Bindless: Fix computation of set and binding for runtime bounds check (#2384)
528fea2b1 Fixup unused variables (#2402)
4b42cd19d BUILD.gn: Only build tests in Chromium. (#2392)
6d20f6257 Refactor webgpu-mode pass ran tests to be parameterized (#2395)
78ac954c4 Mark type id of unknown instructions at fully used. (#2399)
9540f2d98 Instrumentation: Fix instruction index when multiple functions (#2389)
1b0047f21 Add pass to remove dead members. (#2379)
0167a20b0 Move usage detection to after all instructions are registered (#2378)
0187c190b Fix BUILD.gn (#2381)
354205b3d Don't merge unreachable blocks (#2375)
40a7940e0 Fix merge blocks opportunity to check if still enabled (#2370)
12b3d7e9d Add strip-debug to webgpu-mode passes (#2368)
a0ff7c512 Add better error message to ValidateInstructions (#2365)
34c5ac614 Fixes #2358.  Added to the reducer the ability to remove a function t… (#2361)
10a7def6c Smoketest should use KhronosGroup glslang (#2363)
39bfb6b97 Make spvParseTargetEnv public (#2362)
cf2114613 Expand bindless bounds checking to runtime-sized descriptor arrays (#2316)
9b6ba4d1c Allow arrayed storage images for NonWritable decoration (#2358)
117a1fd11 Validate variable pointer related function call rules (#2270)
0f4bf0720 Add flatten-decorations flag to webgpu-mode flags (#2348)
368567bc5 Convert deprecated GoogleTest fixture to the new version (#2357)
adbbe2024 Mention the reducer tool and location in the README. (#2341)
e3c64a374 Do not build spirv-reduce on iOS/tvOS/watchOS - it requires std::system which is not available on those platforms (#2355)
37861ac10 Merge blocks in reducer (#2353)
453b7c85c Fix up some test (#2351)
846d12afe Add whitelist for decorations in WebGPU (#2346)
63e032f91 Remove unused lambda capture (#2350)
3b6fee3da Fixes #2338.  Added functionality to remove OpPhi instructions (replacing their uses) when merging blocks (#2339)
2acbf488b Add WebGPU specific validation for WorkgroupSize BuiltIn decoration (#2334)
e2f462262 Add WebGPU specific validation for multiple BuiltIn decorations (#2333)
3d2afb78c Add whitelist of allowed BuiltIn decorations for WebGPU (#2337)
d17fcf8ab Add WebGPU validation for LocalInvocationIndex BuiltIn decoration (#2335)
837153ccd Add WebGPU specific validation for FragDepth BuiltIn decoration (#2332)
0c14583f1 Add WebGPU specific validation for FragCoord BuiltIn decoration (#2331)
b6698e0d8 Add WebGPU specific validation for FrontFacing BuiltIn decoration (#2330)
734def144 Add WebGPU specific validation for InstanceIndex BuiltIn decoration (#2329)
b947ecfe7 Add WebGPU specific validation for VertexIndex BuiltIn decoration (#2328)
464111eae Remove use of deprecated googletest macro (#2286)
7f1a020ab Fix test_val_limits MSVC build (#2347)
5d6b4c4b1 Move CodeGenerator to be accessible by other validation tests (#2343)
cf7834623 Add source/opt/block_merge_util.* to BUILD.gn (#2344)
7f3679a8b Validate NonWritable decoration (#2263)
9ab1c0ddd Remove code sinking for -O. (#2340)
98c67d385 Fixed names in ifdefs and GetName functions that had been forgotten in a previous refactoring.  Also shortened names of test files as those files test both the new 'finder' classes introduced in the refactoring, as well as the 'reduction pass' class; the shorter names capture both. (#2336)
3345fe6a9 Extracted block merging functionality into its own utility file (#2325)
cf011f990 More layout check fixes (#2315)
e2279da71 Remove the static maps from CheckDecorationsCompatibility (#2327)
8915a7c8f spirv-val: Emit an error when an OpSwitch target is not an OpLabel (#2298)
1e3c589a6 Add WebGPU specific validation for Position BuiltIn decoration (#2309)
20b2e2b9f Add SpirvTools::IsValid(). (#2326)
86d0d9be2 Refactored reducer so that the 'finding' functionality of a reduction pass are separated from the generic functionality for tracking progress of a pass.  With this change, we now have a ReductionOpportunityFinder abstract class, with many subclasses for each type of reduction, and just one ReductionPass class, which has an associated finder. (#2321)
b1be6763f Add helper for 'is Vulkan or WebGPU' (#2324)
3d7102424 Added some documentation about the reducer. (#2318)
4a405eda5 Fix layout checks for nested struct in relaxed layout; and descriptor arrays (#2312)
3a3ad2ec5 Add utility to generate a logging string for a given environment (#2314)
a64c651e1 Fix Constants Analyses bug inserted by #2302 (#2306)
eab06d669 Check forward reference in OpTypeArray. (#2307)
8df947d2d Handle instructions not in blocks in code sinking. (#2308)

git-subtree-dir: third_party/SPIRV-Tools
git-subtree-split: 9559cdbdf011c487f67f89e2d694bd4a18d5c1e0
516 files changed
tree: 67ee2c5d4a58ba10cde6f5718a5d80010eda9ff4
  1. android_test/
  2. build_overrides/
  3. cmake/
  4. examples/
  5. external/
  6. include/
  7. kokoro/
  8. source/
  9. test/
  10. tools/
  11. utils/
  12. .appveyor.yml
  13. .clang-format
  14. .gitignore
  15. .gn
  16. Android.mk
  17. BUILD.gn
  18. CHANGES
  19. CMakeLists.txt
  20. CODE_OF_CONDUCT.md
  21. codereview.settings
  22. CONTRIBUTING.md
  23. DEPS
  24. downloads.md
  25. LICENSE
  26. PRESUBMIT.py
  27. projects.md
  28. README.md
  29. syntax.md
README.md

SPIR-V Tools

Overview

The SPIR-V Tools project provides an API and commands for processing SPIR-V modules.

The project includes an assembler, binary module parser, disassembler, validator, and optimizer for SPIR-V. Except for the optimizer, all are based on a common static library. The library contains all of the implementation details, and is used in the standalone tools whilst also enabling integration into other code bases directly. The optimizer implementation resides in its own library, which depends on the core library.

The interfaces have stabilized: We don't anticipate making a breaking change for existing features.

SPIR-V is defined by the Khronos Group Inc. See the SPIR-V Registry for the SPIR-V specification, headers, and XML registry.

Downloads

Build status Linux Build Status MacOS Build Status Windows Build Status

More downloads

Versioning SPIRV-Tools

See CHANGES for a high level summary of recent changes, by version.

SPIRV-Tools project version numbers are of the form vyear.index and with an optional -dev suffix to indicate work in progress. For exampe, the following versions are ordered from oldest to newest:

  • v2016.0
  • v2016.1-dev
  • v2016.1
  • v2016.2-dev
  • v2016.2

Use the --version option on each command line tool to see the software version. An API call reports the software version as a C-style string.

Supported features

Assembler, binary parser, and disassembler

  • Support for SPIR-V 1.0, 1.1, 1.2, and 1.3
    • Based on SPIR-V syntax described by JSON grammar files in the SPIRV-Headers repository.
  • Support for extended instruction sets:
    • GLSL std450 version 1.0 Rev 3
    • OpenCL version 1.0 Rev 2
  • Assembler only does basic syntax checking. No cross validation of IDs or types is performed, except to check literal arguments to OpConstant, OpSpecConstant, and OpSwitch.

See syntax.md for the assembly language syntax.

Validator

The validator checks validation rules described by the SPIR-V specification.

Khronos recommends that tools that create or transform SPIR-V modules use the validator to ensure their outputs are valid, and that tools that consume SPIR-V modules optionally use the validator to protect themselves from bad inputs. This is especially encouraged for debug and development scenarios.

The validator has one-sided error: it will only return an error when it has implemented a rule check and the module violates that rule.

The validator is incomplete. See the CHANGES file for reports on completed work, and the Validator sub-project for planned and in-progress work.

Note: The validator checks some Universal Limits, from section 2.17 of the SPIR-V spec. The validator will fail on a module that exceeds those minimum upper bound limits. It is future work to parameterize the validator to allow larger limits accepted by a more than minimally capable SPIR-V consumer.

Optimizer

Note: The optimizer is still under development.

Currently supported optimizations:

  • General
    • Strip debug info
  • Specialization Constants
    • Set spec constant default value
    • Freeze spec constant
    • Fold OpSpecConstantOp and OpSpecConstantComposite
    • Unify constants
    • Eliminate dead constant
  • Code Reduction
    • Inline all function calls exhaustively
    • Convert local access chains to inserts/extracts
    • Eliminate local load/store in single block
    • Eliminate local load/store with single store
    • Eliminate local load/store with multiple stores
    • Eliminate local extract from insert
    • Eliminate dead instructions (aggressive)
    • Eliminate dead branches
    • Merge single successor / single predecessor block pairs
    • Eliminate common uniform loads
    • Remove duplicates: Capabilities, extended instruction imports, types, and decorations.

For the latest list with detailed documentation, please refer to include/spirv-tools/optimizer.hpp.

For suggestions on using the code reduction options, please refer to this white paper.

Linker

Note: The linker is still under development.

Current features:

  • Combine multiple SPIR-V binary modules together.
  • Combine into a library (exports are retained) or an executable (no symbols are exported).

See the CHANGES file for reports on completed work, and the General sub-project for planned and in-progress work.

Reducer

Note: The reducer is still under development.

The reducer simplifies and shrinks a SPIR-V module with respect to a user-supplied interestingness function. For example, given a large SPIR-V module that cause some SPIR-V compiler to fail with a given fatal error message, the reducer could be used to look for a smaller version of the module that causes the compiler to fail with the same fatal error message.

To suggest an additional capability for the reducer, file an issue with “Reducer:” as the start of its title.

Extras

  • Utility filters
  • Build target spirv-tools-vimsyntax generates file spvasm.vim. Copy that file into your $HOME/.vim/syntax directory to get SPIR-V assembly syntax highlighting in Vim. This build target is not built by default.

Contributing

The SPIR-V Tools project is maintained by members of the The Khronos Group Inc., and is hosted at https://github.com/KhronosGroup/SPIRV-Tools.

Consider joining the public_spirv_tools_dev@khronos.org mailing list, via https://www.khronos.org/spir/spirv-tools-mailing-list/. The mailing list is used to discuss development plans for the SPIRV-Tools as an open source project. Once discussion is resolved, specific work is tracked via issues and sometimes in one of the projects.

(To provide feedback on the SPIR-V specification, file an issue on the SPIRV-Headers GitHub repository.)

See projects.md to see how we use the GitHub Project feature to organize planned and in-progress work.

Contributions via merge request are welcome. Changes should:

We intend to maintain a linear history on the GitHub master branch.

Source code organization

  • example: demo code of using SPIRV-Tools APIs
  • external/googletest: Intended location for the googletest sources, not provided
  • external/effcee: Location of Effcee sources, if the effcee library is not already configured by an enclosing project.
  • external/re2: Location of RE2 sources, if the re2 library is not already configured by an enclosing project. (The Effcee project already requires RE2.)
  • include/: API clients should add this directory to the include search path
  • external/spirv-headers: Intended location for SPIR-V headers, not provided
  • include/spirv-tools/libspirv.h: C API public interface
  • source/: API implementation
  • test/: Tests, using the googletest framework
  • tools/: Command line executables

Example of getting sources, assuming SPIRV-Tools is configured as a standalone project:

git clone https://github.com/KhronosGroup/SPIRV-Tools.git   spirv-tools
git clone https://github.com/KhronosGroup/SPIRV-Headers.git spirv-tools/external/spirv-headers
git clone https://github.com/google/googletest.git          spirv-tools/external/googletest
git clone https://github.com/google/effcee.git              spirv-tools/external/effcee
git clone https://github.com/google/re2.git                 spirv-tools/external/re2

Tests

The project contains a number of tests, used to drive development and ensure correctness. The tests are written using the googletest framework. The googletest source is not provided with this project. There are two ways to enable tests:

  • If SPIR-V Tools is configured as part of an enclosing project, then the enclosing project should configure googletest before configuring SPIR-V Tools.
  • If SPIR-V Tools is configured as a standalone project, then download the googletest source into the <spirv-dir>/external/googletest directory before configuring and building the project.

Note: You must use a version of googletest that includes a fix for googletest issue 610. The fix is included on the googletest master branch any time after 2015-11-10. In particular, googletest must be newer than version 1.7.0.

Dependency on Effcee

Some tests depend on the Effcee library for stateful matching. Effcee itself depends on RE2.

  • If SPIRV-Tools is configured as part of a larger project that already uses Effcee, then that project should include Effcee before SPIRV-Tools.
  • Otherwise, SPIRV-Tools expects Effcee sources to appear in external/effcee and RE2 sources to appear in external/re2.

Build

Instead of building manually, you can also download the binaries for your platform directly from the master-tot release on GitHub. Those binaries are automatically uploaded by the buildbots after successful testing and they always reflect the current top of the tree of the master branch.

The project uses CMake to generate platform-specific build configurations. Assume that <spirv-dir> is the root directory of the checked out code:

cd <spirv-dir>
git clone https://github.com/KhronosGroup/SPIRV-Headers.git external/spirv-headers
git clone https://github.com/google/effcee.git external/effcee
git clone https://github.com/google/re2.git external/re2
git clone https://github.com/google/googletest.git external/googletest # optional

mkdir build && cd build
cmake [-G <platform-generator>] <spirv-dir>

Note: The script utils/git-sync-deps can be used to checkout and/or update the contents of the repos under external/ instead of manually maintaining them.

Once the build files have been generated, build using your preferred development environment.

Tools you'll need

For building and testing SPIRV-Tools, the following tools should be installed regardless of your OS:

  • CMake: for generating compilation targets. Version 2.8.12 or later.
  • Python 3: for utility scripts and running the test suite.

SPIRV-Tools is regularly tested with the the following compilers:

On Linux

  • GCC version 4.8.5
  • Clang version 3.8

On MacOS

  • AppleClang 10.0

On Windows

  • Visual Studio 2015
  • Visual Studio 2017

Other compilers or later versions may work, but they are not tested.

CMake options

The following CMake options are supported:

  • SPIRV_COLOR_TERMINAL={ON|OFF}, default ON - Enables color console output.
  • SPIRV_SKIP_TESTS={ON|OFF}, default OFF- Build only the library and the command line tools. This will prevent the tests from being built.
  • SPIRV_SKIP_EXECUTABLES={ON|OFF}, default OFF- Build only the library, not the command line tools and tests.
  • SPIRV_USE_SANITIZER=<sanitizer>, default is no sanitizing - On UNIX platforms with an appropriate version of clang this option enables the use of the sanitizers documented here. This should only be used with a debug build.
  • SPIRV_WARN_EVERYTHING={ON|OFF}, default OFF - On UNIX platforms enable more strict warnings. The code might not compile with this option enabled. For Clang, enables -Weverything. For GCC, enables -Wpedantic. See CMakeLists.txt for details.
  • SPIRV_WERROR={ON|OFF}, default ON - Forces a compilation error on any warnings encountered by enabling the compiler-specific compiler front-end option. No compiler front-end options are enabled when this option is OFF.

Additionally, you can pass additional C preprocessor definitions to SPIRV-Tools via setting SPIRV_TOOLS_EXTRA_DEFINITIONS. For example, by setting it to /D_ITERATOR_DEBUG_LEVEL=0 on Windows, you can disable checked iterators and iterator debugging.

Android

SPIR-V Tools supports building static libraries libSPIRV-Tools.a and libSPIRV-Tools-opt.a for Android:

cd <spirv-dir>

export ANDROID_NDK=/path/to/your/ndk

mkdir build && cd build
mkdir libs
mkdir app

$ANDROID_NDK/ndk-build -C ../android_test     \
                      NDK_PROJECT_PATH=.      \
                      NDK_LIBS_OUT=`pwd`/libs \
                      NDK_APP_OUT=`pwd`/app

Updating DEPS

Occasionally the entries in DEPS will need to be updated. This is done on demand when there is a request to do this, often due to downstream breakages. There is a script utils/roll_deps.sh provided, which will generate a patch with the updated DEPS values. This will still need to be tested in your checkout to confirm that there are no integration issues that need to be resolved.

Library

Usage

The internals of the library use C++11 features, and are exposed via both a C and C++ API.

In order to use the library from an application, the include path should point to <spirv-dir>/include, which will enable the application to include the header <spirv-dir>/include/spirv-tools/libspirv.h{|pp} then linking against the static library in <spirv-build-dir>/source/libSPIRV-Tools.a or <spirv-build-dir>/source/SPIRV-Tools.lib. For optimization, the header file is <spirv-dir>/include/spirv-tools/optimizer.hpp, and the static library is <spirv-build-dir>/source/libSPIRV-Tools-opt.a or <spirv-build-dir>/source/SPIRV-Tools-opt.lib.

  • SPIRV-Tools CMake target: Creates the static library:
    • <spirv-build-dir>/source/libSPIRV-Tools.a on Linux and OS X.
    • <spirv-build-dir>/source/libSPIRV-Tools.lib on Windows.
  • SPIRV-Tools-opt CMake target: Creates the static library:
    • <spirv-build-dir>/source/libSPIRV-Tools-opt.a on Linux and OS X.
    • <spirv-build-dir>/source/libSPIRV-Tools-opt.lib on Windows.

Entry points

The interfaces are still under development, and are expected to change.

There are five main entry points into the library in the C interface:

  • spvTextToBinary: An assembler, translating text to a binary SPIR-V module.
  • spvBinaryToText: A disassembler, translating a binary SPIR-V module to text.
  • spvBinaryParse: The entry point to a binary parser API. It issues callbacks for the header and each parsed instruction. The disassembler is implemented as a client of spvBinaryParse.
  • spvValidate implements the validator functionality. Incomplete
  • spvValidateBinary implements the validator functionality. Incomplete

The C++ interface is comprised of three classes, SpirvTools, Optimizer and Linker, all in the spvtools namespace.

  • SpirvTools provides Assemble, Disassemble, and Validate methods.
  • Optimizer provides methods for registering and running optimization passes.
  • Linker provides methods for combining together multiple binaries.

Command line tools

Command line tools, which wrap the above library functions, are provided to assemble or disassemble shader files. It's a convention to name SPIR-V assembly and binary files with suffix .spvasm and .spv, respectively.

Assembler tool

The assembler reads the assembly language text, and emits the binary form.

The standalone assembler is the exectuable called spirv-as, and is located in <spirv-build-dir>/tools/spirv-as. The functionality of the assembler is implemented by the spvTextToBinary library function.

  • spirv-as - the standalone assembler
    • <spirv-dir>/tools/as

Use option -h to print help.

Disassembler tool

The disassembler reads the binary form, and emits assembly language text.

The standalone disassembler is the executable called spirv-dis, and is located in <spirv-build-dir>/tools/spirv-dis. The functionality of the disassembler is implemented by the spvBinaryToText library function.

  • spirv-dis - the standalone disassembler
    • <spirv-dir>/tools/dis

Use option -h to print help.

The output includes syntax colouring when printing to the standard output stream, on Linux, Windows, and OS X.

Linker tool

The linker combines multiple SPIR-V binary modules together, resulting in a single binary module as output.

This is a work in progress. The linker does not support OpenCL program linking options related to math flags. (See section 5.6.5.2 in OpenCL 1.2)

  • spirv-link - the standalone linker
    • <spirv-dir>/tools/link

Optimizer tool

The optimizer processes a SPIR-V binary module, applying transformations in the specified order.

This is a work in progress, with initially only few available transformations.

  • spirv-opt - the standalone optimizer
    • <spirv-dir>/tools/opt

Validator tool

Warning: This functionality is under development, and is incomplete.

The standalone validator is the executable called spirv-val, and is located in <spirv-build-dir>/tools/spirv-val. The functionality of the validator is implemented by the spvValidate library function.

The validator operates on the binary form.

  • spirv-val - the standalone validator
    • <spirv-dir>/tools/val

Reducer tool

The reducer shrinks a SPIR-V binary module, guided by a user-supplied interestingness test.

This is a work in progress, with initially only shrinks a module in a few ways.

  • spirv-reduce - the standalone reducer
    • <spirv-dir>/tools/reduce

Run spirv-reduce --help to see how to specify interestingness.

Control flow dumper tool

The control flow dumper prints the control flow graph for a SPIR-V module as a GraphViz graph.

This is experimental.

  • spirv-cfg - the control flow graph dumper
    • <spirv-dir>/tools/cfg

Utility filters

  • spirv-lesspipe.sh - Automatically disassembles .spv binary files for the less program, on compatible systems. For example, set the LESSOPEN environment variable as follows, assuming both spirv-lesspipe.sh and spirv-dis are on your executable search path:

     export LESSOPEN='| spirv-lesspipe.sh "%s"'

    Then you page through a disassembled module as follows:

    less foo.spv
    • The spirv-lesspipe.sh script will pass through any extra arguments to spirv-dis. So, for example, you can turn off colours and friendly ID naming as follows:
      export LESSOPEN='| spirv-lesspipe.sh "%s" --no-color --raw-id'

  • vim-spirv - A vim plugin which supports automatic disassembly of .spv files using the :edit command and assembly using the :write command. The plugin also provides additional features which include; syntax highlighting; highlighting of all ID's matching the ID under the cursor; and highlighting errors where the Instruction operand of OpExtInst is used without an appropriate OpExtInstImport.

  • 50spirv-tools.el - Automatically disassembles ‘.spv’ binary files when loaded into the emacs text editor, and re-assembles them when saved, provided any modifications to the file are valid. This functionality must be explicitly requested by defining the symbol SPIRV_TOOLS_INSTALL_EMACS_HELPERS as follows:

    cmake -DSPIRV_TOOLS_INSTALL_EMACS_HELPERS=true ...

    In addition, this helper is only installed if the directory /etc/emacs/site-start.d exists, which is typically true if emacs is installed on the system.

    Note that symbol IDs are not currently preserved through a load/edit/save operation. This may change if the ability is added to spirv-as.

Tests

Tests are only built when googletest is found. Use ctest to run all the tests.

Future Work

See the projects pages for more information.

Assembler and disassembler

  • The disassembler could emit helpful annotations in comments. For example:
    • Use variable name information from debug instructions to annotate key operations on variables.
    • Show control flow information by annotating OpLabel instructions with that basic block's predecessors.
  • Error messages could be improved.

Validator

This is a work in progress.

Linker

  • The linker could accept math transformations such as allowing MADs, or other math flags passed at linking-time in OpenCL.
  • Linkage attributes can not be applied through a group.
  • Check decorations of linked functions attributes.
  • Remove dead instructions, such as OpName targeting imported symbols.

Licence

Full license terms are in LICENSE

Copyright (c) 2015-2016 The Khronos Group Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.