Skip to content

Commit

Permalink
Implement Linker (module combiner)
Browse files Browse the repository at this point in the history
Add extra iterators for ir::Module's sections
Add extra getters to ir::Function
Add a const version of BasicBlock::GetLabelInst()

Use the max of all inputs' version as version

Split debug in debug1 and debug2
- Debug1 instructions have to be placed before debug2 instructions.

Error out if different addressing or memory models are found

Exit early if no binaries were given

Error out if entry points are redeclared

Implement copy ctors for Function and BasicBlock
- Visual Studio ends up generating copy constructors that call deleted
  functions while compiling the linker code, while GCC and clang do not.
  So explicitly write those functions to avoid Visual Studio messing up.

Move removing duplicate capabilities to its own pass

Add functions running on all IDs present in an instruction

Remove duplicate SpvOpExtInstImport

Give default options value for link functions

Remove linkage capability if not making a library

Check types before allowing to link

Detect if two types/variables/functions have different decorations

Remove decorations of imported variables/functions and their types

Add a DecorationManager

Add a method for removing all decorations of id

Add methods for removing operands from instructions

Error out if one of the modules has a non-zero schema

Update README.md to talk about the linker

Do not freak out if an imported built-in variable has no export
  • Loading branch information
pierremoreau authored and dneto0 committed Oct 6, 2017
1 parent 4b1577a commit 86627f7
Show file tree
Hide file tree
Showing 43 changed files with 2,892 additions and 47 deletions.
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ if(ENABLE_SPIRV_TOOLS_INSTALL)
${CMAKE_CURRENT_SOURCE_DIR}/include/spirv-tools/libspirv.h
${CMAKE_CURRENT_SOURCE_DIR}/include/spirv-tools/libspirv.hpp
${CMAKE_CURRENT_SOURCE_DIR}/include/spirv-tools/optimizer.hpp
${CMAKE_CURRENT_SOURCE_DIR}/include/spirv-tools/linker.hpp
DESTINATION
${CMAKE_INSTALL_INCLUDEDIR}/spirv-tools/)
endif(ENABLE_SPIRV_TOOLS_INSTALL)
Expand Down
41 changes: 39 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,12 +109,28 @@ Currently supported optimizations:
* Eliminate dead branches
* Merge single successor / single predecessor block pairs
* Eliminate common uniform loads
* Remove duplicates: Capabilities, extended instruction imports, types, and
decorations.

For the latest list with detailed documentation, please refer to
[`include/spirv-tools/optimizer.hpp`](include/spirv-tools/optimizer.hpp).

For suggestions on using the code reduction options, please refer to this [white paper](https://www.lunarg.com/shader-compiler-technologies/white-paper-spirv-opt/).


### Linker

*Note:* The linker is still under development.

Current features:
* Combine multiple SPIR-V binary modules together.
* Combine into a library (exports are retained) or an executable (no symbols
are exported).

See the [CHANGES](CHANGES) file for reports on completed work, and the [General
sub-project](https://github.com/KhronosGroup/SPIRV-Tools/projects/2) for
planned and in-progress work.

### Extras

* [Utility filters](#utility-filters)
Expand Down Expand Up @@ -255,10 +271,11 @@ There are five main entry points into the library in the C interface:
* `spvValidate` implements the validator functionality. *Incomplete*
* `spvValidateBinary` implements the validator functionality. *Incomplete*

The C++ interface is comprised of two classes, `SpirvTools` and `Optimizer`,
both in the `spvtools` namespace.
The C++ interface is comprised of three classes, `SpirvTools`, `Optimizer` and
`Linker`, all in the `spvtools` namespace.
* `SpirvTools` provides `Assemble`, `Disassemble`, and `Validate` methods.
* `Optimizer` provides methods for registering and running optimization passes.
* `Linker` provides methods for combining together multiple binaries.

## Command line tools

Expand Down Expand Up @@ -295,6 +312,18 @@ Use option `-h` to print help.
The output includes syntax colouring when printing to the standard output stream,
on Linux, Windows, and OS X.

### Linker tool

The linker combines multiple SPIR-V binary modules together, resulting in a single
binary module as output.

This is a work in progress.
The linker does not support OpenCL program linking options related to math
flags. (See section 5.6.5.2 in OpenCL 1.2)

* `spirv-link` - the standalone linker
* `<spirv-dir>/tools/link`

### Optimizer tool

The optimizer processes a SPIR-V binary module, applying transformations
Expand Down Expand Up @@ -395,6 +424,14 @@ for more information._
This is a work in progress.
### Linker
* The linker could accept math transformations such as allowing MADs, or other
math flags passed at linking-time in OpenCL.
* Linkage attributes can not be applied through a group.
* Check decorations of linked functions attributes.
* Remove dead instructions, such as OpName targeting imported symbols.
## Licence
<a name="license"></a>
Full license terms are in [LICENSE](LICENSE)
Expand Down
98 changes: 98 additions & 0 deletions include/spirv-tools/linker.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
// Copyright (c) 2017 Pierre Moreau
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#ifndef SPIRV_TOOLS_LINKER_HPP_
#define SPIRV_TOOLS_LINKER_HPP_

#include <cstdint>

#include <memory>
#include <vector>

#include "libspirv.hpp"

namespace spvtools {

class LinkerOptions {
public:
LinkerOptions() : createLibrary_(false) {}

// Returns whether a library or an executable should be produced by the
// linking phase.
//
// All exported symbols are kept when creating a library, whereas they will
// be removed when creating an executable.
// The returned value will be true if creating a library, and false if
// creating an executable.
bool GetCreateLibrary() const { return createLibrary_; }
// Sets whether a library or an executable should be produced.
void SetCreateLibrary(bool create_library) {
createLibrary_ = create_library;
}

private:
bool createLibrary_;
};

class Linker {
public:
// Constructs an instance targeting the given environment |env|.
//
// The constructed instance will have an empty message consumer, which just
// ignores all messages from the library. Use SetMessageConsumer() to supply
// one if messages are of concern.
explicit Linker(spv_target_env env);

// Disables copy/move constructor/assignment operations.
Linker(const Linker&) = delete;
Linker(Linker&&) = delete;
Linker& operator=(const Linker&) = delete;
Linker& operator=(Linker&&) = delete;

// Destructs this instance.
~Linker();

// Sets the message consumer to the given |consumer|. The |consumer| will be
// invoked once for each message communicated from the library.
void SetMessageConsumer(MessageConsumer consumer);

// Links one or more SPIR-V modules into a new SPIR-V module. That is,
// combine several SPIR-V modules into one, resolving link dependencies
// between them.
//
// At least one binary has to be provided in |binaries|. Those binaries do
// not have to be valid, but they should be at least parseable.
// The functions can fail due to the following:
// * No input modules were given;
// * One or more of those modules were not parseable;
// * The input modules used different addressing or memory models;
// * The ID or global variable number limit were exceeded;
// * Some entry points were defined multiple times;
// * Some imported symbols did not have an exported counterpart;
// * Possibly other reasons.
spv_result_t Link(const std::vector<std::vector<uint32_t>>& binaries,
std::vector<uint32_t>& linked_binary,
const LinkerOptions& options = LinkerOptions()) const;
spv_result_t Link(const uint32_t* const* binaries, const size_t* binary_sizes,
size_t num_binaries, std::vector<uint32_t>& linked_binary,
const LinkerOptions& options = LinkerOptions()) const;

private:
struct Impl; // Opaque struct for holding the data fields used by this class.
std::unique_ptr<Impl> impl_; // Unique pointer to implementation data.
};

} // namespace spvtools

#endif // SPIRV_TOOLS_LINKER_HPP_
3 changes: 3 additions & 0 deletions include/spirv-tools/optimizer.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,9 @@ Optimizer::PassToken CreateAggressiveDCEPass();
// The pass remaps result ids to a compact and gapless range starting from %1.
Optimizer::PassToken CreateCompactIdsPass();

// Creates a remove duplicate capabilities pass.
Optimizer::PassToken CreateRemoveDuplicatesPass();

} // namespace spvtools

#endif // SPIRV_TOOLS_OPTIMIZER_HPP_
1 change: 1 addition & 0 deletions source/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ set_property(TARGET spirv-tools-build-version PROPERTY FOLDER "SPIRV-Tools build

add_subdirectory(comp)
add_subdirectory(opt)
add_subdirectory(link)

set(SPIRV_SOURCES
${spirv-tools_SOURCE_DIR}/include/spirv-tools/libspirv.h
Expand Down
35 changes: 35 additions & 0 deletions source/link/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Copyright (c) 2017 Pierre Moreau

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
add_library(SPIRV-Tools-link
linker.cpp
)

spvtools_default_compile_options(SPIRV-Tools-link)
target_include_directories(SPIRV-Tools-link
PUBLIC ${spirv-tools_SOURCE_DIR}/include
PUBLIC ${SPIRV_HEADER_INCLUDE_DIR}
PRIVATE ${spirv-tools_BINARY_DIR}
)
# We need the IR functionnalities from the optimizer
target_link_libraries(SPIRV-Tools-link
PUBLIC SPIRV-Tools-opt)

set_property(TARGET SPIRV-Tools-link PROPERTY FOLDER "SPIRV-Tools libraries")

if(ENABLE_SPIRV_TOOLS_INSTALL)
install(TARGETS SPIRV-Tools-link
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
endif(ENABLE_SPIRV_TOOLS_INSTALL)
Loading

0 comments on commit 86627f7

Please sign in to comment.