Skip to content

Commit

Permalink
docs: Add solver and failures doc; misc others
Browse files Browse the repository at this point in the history
  • Loading branch information
sdboyer committed Jan 17, 2018
1 parent bb08cee commit b2c090f
Show file tree
Hide file tree
Showing 6 changed files with 146 additions and 13 deletions.
11 changes: 9 additions & 2 deletions docs/Gopkg.toml.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@ The `Gopkg.toml` file is initially generated by `dep init`, and is primarily han
* [`constraints`](#constraint) and [`overrides`](#override) allow the user to specify which versions of dependencies are acceptable, and where they should be retrieved from.
* [`required`](#required) and [`ignored`](#ignored) allow the user to manipulate the import graph by including or excluding import paths, respectively.
* [`metadata`](#metadata) is a user-defined map of key-value pairs that dep will preserve and ignore.
* [`prune`](#prune) settings govern what files and directories can be deemed unnecessary, and thus automatically removed from `vendor/`.

Note that because of TOML does not adhere to a tree structure, the `required` and `ignored` fields must be declared before any `[[constraint]]` or `[[override]]`.
Note that because TOML does not adhere to a tree structure, the `required` and `ignored` fields must be declared before any `[[constraint]]` or `[[override]]`.

---

Expand Down Expand Up @@ -123,7 +124,13 @@ system2-data = "value that is used by another system"



## `version`
## Version rules

Version rules can be used in either `[[constraint]]` or `[[override]]` stanzas. There are three types of version rules - `version`, `branch`, and `revision`. At most one of the three types can be specified.

TODOOOOOOOOOOOOO

### `version`

`version` is a property of `constraint`s and `override`s. It is used to specify version constraint of a specific dependency.

Expand Down
10 changes: 5 additions & 5 deletions docs/ensure-mechanics.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ For any of the paths where `dep ensure -add` needs to run the solving function i

![Model modifications made by -add](img/required-arrows.png)

Import path arguments that need to be added are injected via the `required` list, and if an explicit version requirement was specified, the equivalent of a `[[constraint]]` is created.
Import path arguments that need to be added are injected via the `required` list, and if an explicit version requirement was specified, the equivalent of a `[[constraint]]` is created.

Though these rules may ultimately be persisted if solving succeeds, they are ephemeral at least until solving succeeds. And, from the solver's perspective, the ephemeral rules are indistinguishable from rules sourced directly from disk. Thus, to the solver, `dep ensure -add [email protected]` is identical to modifying `Gopkg.toml` by adding `"foo"` to the `required` list, plus a `[[constraint]]` stanza with `version = "v1.0.0"`, then running `dep ensure`.

Expand All @@ -131,9 +131,9 @@ First, to solidify an implication in the discussion of [functional optimizations

![Pre-existing lock feeds back into solving function](img/lock-back.png)

Injecting `Gopkg.lock` into the solver is a necessity. We want the solver to preserve previously-selected versions by default, but unless `Gopkg.lock` is injected somewhere, then the solver can't know what versions it's trying to preserve.
Injecting `Gopkg.lock` into the solver is a necessity. If we want the solver to preserve previously-selected versions by default, then the solver has to learn about the existing `Gopkg.lock` from somewhere. Otherwise, it wouldn't know what to preserve!

As such, the lock is another one of the properties encoded onto the `SolveParameters` struct, discussed [previously](#functional-flow). That, plus two other properties, are the salient ones for `-update`:
As such, the lock is another one of the properties encoded onto the [previously-discussed](#functional-flow) `SolveParameters` struct. That, plus two other properties, are the salient ones for `-update`:

```go
type SolveParameters struct {
Expand Down Expand Up @@ -177,13 +177,13 @@ So, barring some other conflict, `v1.2.0` is selected, resulting in the desired

Continuing with our example, it's important to note that updates with `-update` are achieved incidentally - the solver never explicitly targets a newer version. It just skips adding a hint from the lock, then selects the first version in the queue that satisfies constraints. Consequently, `-update` is only effective with certain types of constraints.

It does work with branch constraints, which we can observe by including the underlying revision. If the user has constrained on `branch = "master"`, and `Gopkg.lock` points at an older revision (say, `aabbccd`) than the canonical source's `master` branch points to (`bbccdde`), then `dep ensure` will end up contructing a queue that looks like this:
It does work with branch constraints, which we can observe by including the underlying revision. If the user has constrained on `branch = "master"`, and `Gopkg.lock` points at a topologically older revision (say, `aabbccd`) than the tip of the canonical source's `master` branch (say, `bbccdde`), then `dep ensure` will end up contructing a queue that looks like this:

```
[master@aabbccd, v1.1.0, v1.2.0, v1.1.1, v1.1.0, v1.0.0, master@bbccdde]
```

With `-update`, the hint at the head will be omitted, and `branch = "master"` will reject all of the semantic versions, finally settling on `master@bbccdde`.
With `-update`, the hint at the head will be omitted; `branch = "master"` will cause the solver to reject all of the semantic versions, and finally settle on `master@bbccdde`.

All versions in the version queue keep track of an underlying revision, which means the same is true if, for example, some upstream project force-pushes a git tag:

Expand Down
28 changes: 28 additions & 0 deletions docs/failure-modes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
title: Failure Modes
---

Like all complex, network-oriented software, dep has known failure modes. These can generally be divided into two groups: hard failures, where the dep process exits non-zero, and soft failures, where it exits zero, but maybe shouldn't have.



## Solving failures

When `dep ensure` or `dep init` exit with an error message looking something like this:

```
$ dep init
init failed: unable to solve the dependency graph: Solving failure: No versions of github.com/foo/bar met constraints:
v1.0.1: Could not introduce github.com/foo/[email protected], as its subpackage github.com/foo/bar/foo is missing. (Package is required by (root).)
v1.0.0: Could not introduce github.com/foo/[email protected], as...
v0.1.0: (another error)
master: (another error)
```

It means that the solver was unable to find a combination of versions for all dependencies that satisfy all the rules enforced by the solver. These rules are described in detail in the section on [solver invariants](the-solver.md#solving-invariants), but here's a summary:

* **`[[constraint]]` conflicts:** when projects in the dependency graph disagree on what [versions](gopkg.toml.md#version-rules) are acceptable for a project, or where to [source](gopkg.toml.md#source) it from.
* **Package validity failure:** when an imported package is quite obviously not capable of being built.
* **Import comment failure:** when the import path used to address a package differs from the [import comment](https://golang.org/cmd/go/#hdr-Import_path_checking) the package uses to specify how it should be imported.
* **Case-only import variation failure:** when two equal-except-for-case imports exist in the same build.

13 changes: 9 additions & 4 deletions docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ title: Glossary

dep uses some specialized terminology. Learn about it here!

* [Atom](#atom)
* [Constraint](#constraint)
* [Current Project](#current-project)
* [Deduction](#deduction)
Expand All @@ -23,6 +24,10 @@ dep uses some specialized terminology. Learn about it here!

---

## Atom

Atoms are a source at a particular version. In practice, this means a two-tuple of [project root](#project-root) and version, e.g. `github.com/foo/bar@master`. Atoms are primarily internal to the [solver](#solver), and the term is rarely used elsewhere.

## Constraint

Constraints have both a narrow and a looser meaning. The narrow sense refers to a [`[[constraint]]`](Gopkg.toml.md#constraint) stanza in `Gopkg.toml`. However, in some contexts, the word may be used more loosely to refer to the idea of applying rules and requirements to dependency management in general.
Expand Down Expand Up @@ -68,19 +73,19 @@ The root import path for a project. A project root is defined as:
* For the current project, the location of the `Gopkg.toml` file defines the project root
* For dependencies, the root of the network [source](#source) (VCS repository) is treated as the project root

These are generally one and the same, though not always. When using dep inside a monorepo, multiple `Gopkg.toml` files may exist at subpaths for discrete projects, defining each of those import paths as Project Roots. This works fine when working directly on those projects. If, however, any project not in the repository seeks to import the monorepo, dep will treat the monorepo's as one big Project, with the root directory being the Project Root; it will disregard any and all `Gopkg.toml` files in subdirectories.
These are generally one and the same, though not always. When using dep inside a monorepo, multiple `Gopkg.toml` files may exist at subpaths for discrete projects, designating each of those import paths as Project Roots. This works fine when working directly on those projects. If, however, any project not in the repository seeks to import the monorepo, dep will treat the monorepo's as one big Project, with the root directory being the Project Root; it will disregard any and all `Gopkg.toml` files in subdirectories.

This may also be referred to as the "import root" or "root import path."

## Solver

"The solver" is a reference to the domain-specific SAT solver contained in [gps](#gps). More detail can be found on its [reference page](solver.md).
"The solver" is a reference to the domain-specific SAT solver contained in [gps](#gps). More detail can be found on its [reference page](the-solver.md).

## Source

The remote entities that hold versioned code. Sources are specifically the entity containing the code, not any particular version of thecode itself.
The remote entities that hold versioned code. Sources are specifically the entity containing the code, not any particular version of thecode itself.

Used in lieu of "VCS" because Go package management tools will soon learn to use more than just VCS systems.
"Source" is used in lieu of "VCS" because Go package management tools will soon learn to use more than just VCS systems.

## Source Root

Expand Down
4 changes: 2 additions & 2 deletions docs/migrating.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,13 +109,13 @@ Unfortunately, `dep init` does not write out a partial `Gopkg.toml` when it fail

Soft failures are usually obvious: `dep init` exits cleanly, but subsequent builds or tests fail. dep's soft failures are usually more drastically than subtly wrong - an explosion of type errors when you try to build, because a wildly incorrect version for some dependency got selected.

If you do encounter problems like this, `dep status` is your first diagnostic step. This will tell you what versions were selected for all your dependencies. You may know which dependencies are a problem simply from your building or testing error messages; if not, compare the `dep status` list against the versions recorded by your previous tool to find the differences.
If you do encounter problems like this, `dep status` is your first diagnostic step; it will report what versions were selected for all your dependencies. It may be clear which dependencies are a problem simply from your building or testing error messages. If not, compare the `dep status` list against the versions recorded by your previous tool to find the differences.

Once you've identified the problematic dependenc(ies), the next step is exerting appropriate controls over them via `Gopkg.toml`. (Note - this advice is intentionally terse; look at [Zen of Constraints and Locks]() if you want a deeper understanding of how to optimally utilize dep's controls)

For each of the following items, assume that you should run `dep ensure` after making the suggested change. If that fails, consult [dealing with solving failures]().

* If the wrong `[[constraint]]` was inferred for one of your direct dependencies, change it. Then, file an issue against dep, please - while `dep init` may choose to omit a constraint, converting one incorrectly is considered a bug.
* If the wrong `[[constraint]]` was inferred for one of your direct dependencies, change it. Then, file an issue against dep (please!) - while `dep init` may choose to omit a constraint, converting one incorrectly is considered a bug.
* If one of your transitive dependencies is at the wrong version, define an `[[override]]` on it to force it to the version you need.
* If the version you need is a specific git commit, it's preferable to instead manually change the `revision` to the desired hash in `Gopkg.lock` for that project, then drop the `version` or `branch` fields (if any).
* If one of your direct dependencies is at the wrong version and there's no `[[constraint]]` on it in `Gopkg.toml` already, then define an appropriate one.
Expand Down
Loading

0 comments on commit b2c090f

Please sign in to comment.