Switch to modules (letsencrypt#4211)

Because the package versions in go.mod match what we use in Godeps.json,
there are no substantive code diffs. However, there are some tiny
differences resulting from how go mod vendors things differently than
godep:

go mod does not preserve executable permissions on shell scripts
Some packages have import lines like:
package ocsp // import "golang.org/x/crypto/ocsp"

godep used to remove the comment from these lines, but go mod vendor does not.

This introduces several indirect dependencies that we didn't have
before. This is because godep used to operate at a package level, but
go mod operates at a module (~= repository) level. So if we used a
given repository, but didn't use all of its packages, we wouldn't
previously care about the transitive dependencies of the packages we
weren't using. However, in the go mod world, once we care about the
repository, we care about all of that repository's transitive
dependencies. AFAICT this doesn't affect vendoring.

Fixes letsencrypt#4116

.travis.yml

@@ -40,10 +40,10 @@ env:
     - RUN="integration" BOULDER_CONFIG_DIR="test/config-next"
     - RUN="unit"
     - RUN="unit-next" BOULDER_CONFIG_DIR="test/config-next"
-    # godep-restore runs with a separate container because it needs to fetch
+    # gomod-vendor runs with a separate container because it needs to fetch
     # packages from GitHub et. al., which is incompatible with the DNS server
     # override in the boulder container (used for service discovery).
-    - RUN="godep-restore" CONTAINER="netaccess"
+    - RUN="gomod-vendor" CONTAINER="netaccess"
     - RUN="coverage" CONTAINER="netaccess"
 
 matrix:

CONTRIBUTING.md

@@ -253,44 +253,22 @@ tag).
 
 # Dependencies
 
-We vendorize all our dependencies using `godep`. Vendorizing means we copy the contents of those dependencies into our own repo. This has a few advantages:
-  - If the remote sites that host our various dependencies are unreachable, it is still possible to build Boulder solely from the contents of its repo.
-  - The versions of our dependencies can't change out from underneath us.
+We use [go modules](https://github.com/golang/go/wiki/Modules) and vendor our
+dependencies. As of Go 1.12, this may require setting the GO111MODULE=on and
+GOFLAGS=-mod=vendor environment variables. Inside the Docker containers for
+Boulder tests, these variables are set for you, but if you ever work outside
+those containers you will want to set them yourself.
 
-Note that this makes it possible to edit the local copy of our dependencies rather than the upstream copy. Occasionally we do this in great emergencies, but in general this is a bad idea because it means the next person to update that dependency will overwrite the changes.
-
-Instead, it's better to contribute a patch upstream, then pull down changes. For dependencies that we expect to update semi-regularly, we create a fork in the letsencrypt organization, and vendorize that fork. For such forked dependencies, make changes by submitting a pull request to the letsencrypt fork. Once the pull request is reviewed and merged, (a) submit the changes as an upstream pull request, and (b) run `godep` to update to the latest version in the main Boulder. There are two advantages to this approach:
-  - If upstream is slow to merge for any reason, we don't have to wait.
-  - When we make changes, our first review is from other Boulder contributors rather than upstream. That way we make sure code meets our needs first before asking someone else to spend time on it.
+To add a dependency, add the import statement to your .go file, then run
+`go build` on it. This will automatically add the dependency to go.mod. Next,
+run `go mod vendor` to save a copy in the vendor folder.
 
 When vendorizing dependencies, it's important to make sure tests pass on the version you are vendorizing. Currently we enforce this by requiring that pull requests containing a dependency update include a comment indicating that you ran the tests and that they succeeded, preferably with the command line you run them with.
 
 ## Updating Dependencies
 
-All Go dependencies are vendored under the vendor directory, to [make dependency management easier](https://golang.org/cmd/go/#hdr-Vendor_Directories).
-
-To update a dependencies:
-
-```
-# Fetch godep
-go get -u github.com/tools/godep
-# Check out the currently vendorized version of each dependency.
-godep restore
-# Update to the latest version of a dependency. Alternately you can cd to the
-# directory under GOPATH and check out a specific revision. Here's an example
-# using cfssl:
-go get -u github.com/cloudflare/cfssl/...
-# Update the Godep config to the appropriate version.
-godep update github.com/cloudflare/cfssl/...
-# Save the dependencies
-godep save ./...
-git add Godeps vendor
-git commit
-```
-
-NOTE: If you get "godep: no packages can be updated," there's a good chance you're trying to update a single package that belongs to a repo with other packages. For instance, `godep update golang.org/x/crypto/ocsp` will produce this error, because it's part of the `golang.org/x/crypto` repo, from which we also vendor other packages. Godep requires that all packages from the same repo be on the same version, so it can't update just one. See https://github.com/tools/godep/issues/164 for the issue dedicated to fixing it.
-
-Certain dependencies we rely on themselves also vendor packages that we vendor. This generally isn't an issue unless the version that is vendored by our dependency uses a version with breaking changes from the version that we vendor. In this case either we need to switch to the same version or attempt to get the dependency to do the same.
+To upgrade a dependency, [see the Go
+docs](https://github.com/golang/go/wiki/Modules#how-to-upgrade-and-downgrade-dependencies).
 
 # Go Version
 

Makefile

@@ -39,7 +39,7 @@ $(OBJDIR):
 $(CMD_BINS): build_cmds
 
 build_cmds: | $(OBJDIR)
-	GOBIN=$(OBJDIR) go install $(GO_BUILD_FLAGS) ./...
+	GOBIN=$(OBJDIR) GO111MODULE=on go install -mod=vendor $(GO_BUILD_FLAGS) ./...
 	cp $(OBJDIR)/boulder-va $(OBJDIR)/boulder-remoteva
 
 # Building an RPM requires `fpm` from https://github.com/jordansissel/fpm

docker-compose.yml

@@ -7,6 +7,8 @@ services:
             FAKE_DNS: 10.77.77.77
             PKCS11_PROXY_SOCKET: tcp://boulder-hsm:5657
             BOULDER_CONFIG_DIR: test/config
+            GO111MODULE: "on"
+            GOFLAGS: "-mod=vendor"
         volumes:
           - .:/go/src/github.com/letsencrypt/boulder
           - ./.gocache:/root/.cache/go-build
@@ -74,6 +76,9 @@ services:
             driver: none
     netaccess:
         image: letsencrypt/boulder-tools-go${TRAVIS_GO_VERSION:-1.12}:2019-04-08
+        environment:
+            GO111MODULE: "on"
+            GOFLAGS: "-mod=vendor"
         networks:
           - bluenet
         volumes:

go.mod

@@ -0,0 +1,52 @@
+module github.com/letsencrypt/boulder
+
+go 1.12
+
+require (
+	github.com/apoydence/onpar v0.0.0-20181125144932-f2f06780798d // indirect
+	github.com/asaskevich/govalidator v0.0.0-20170903095215-73945b6115bf // indirect
+	github.com/beeker1121/goque v0.0.0-20170321141813-4044bc29b280
+	github.com/beorn7/perks v0.0.0-20160229213445-3ac7bf7a47d1 // indirect
+	github.com/cloudflare/cfssl v0.0.0-20190409034051-768cd563887f
+	github.com/go-gorp/gorp v2.0.0+incompatible // indirect
+	github.com/go-sql-driver/mysql v0.0.0-20170715192408-3955978caca4
+	github.com/golang/mock v1.2.0
+	github.com/golang/protobuf v1.3.1
+	github.com/golang/snappy v0.0.0-20170215233205-553a64147049 // indirect
+	github.com/google/certificate-transparency-go v0.0.0-20181127102053-c25855a82c75
+	github.com/grpc-ecosystem/go-grpc-prometheus v0.0.0-20170826090648-0dafe0d496ea
+	github.com/jmhodges/clock v0.0.0-20160418191101-880ee4c33548
+	github.com/jmoiron/sqlx v0.0.0-20180124204410-05cef0741ade // indirect
+	github.com/kisielk/sqlstruct v0.0.0-20150923205031-648daed35d49 // indirect
+	github.com/letsencrypt/challtestsrv v1.0.2
+	github.com/letsencrypt/pkcs11key v1.0.0
+	github.com/lib/pq v1.1.0 // indirect
+	github.com/mattn/go-sqlite3 v1.10.0 // indirect
+	github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
+	github.com/miekg/dns v1.1.8
+	github.com/miekg/pkcs11 v0.0.0-20180208123754-88ac7c418f89
+	github.com/onsi/ginkgo v1.8.0 // indirect
+	github.com/onsi/gomega v1.5.0 // indirect
+	github.com/poy/onpar v0.0.0-20181125144932-f2f06780798d // indirect
+	github.com/prometheus/client_golang v0.8.0
+	github.com/prometheus/client_model v0.0.0-20150212101744-fa8ad6fec335
+	github.com/prometheus/common v0.0.0-20161002210234-85637ea67b04 // indirect
+	github.com/prometheus/procfs v0.0.0-20160411190841-abf152e5f3e9 // indirect
+	github.com/stretchr/testify v1.3.0 // indirect
+	github.com/syndtr/goleveldb v0.0.0-20180331014930-714f901b98fd // indirect
+	github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399
+	github.com/weppos/publicsuffix-go v0.4.1-0.20190430132455-7c1d5dc5cdc2
+	github.com/ziutek/mymysql v1.5.4 // indirect
+	github.com/zmap/zcrypto v0.0.0-20180601141720-cf96f6a85166
+	github.com/zmap/zlint v0.0.0-20190218154616-f38bd223a43c
+	golang.org/x/crypto v0.0.0-20190411191339-88737f569e3a
+	golang.org/x/net v0.0.0-20190415214537-1da14a5a36f2
+	golang.org/x/sys v0.0.0-20190416152802-12500544f89f // indirect
+	golang.org/x/text v0.3.0
+	google.golang.org/appengine v1.4.0 // indirect
+	google.golang.org/genproto v0.0.0-20190415143225-d1146b9035b9 // indirect
+	google.golang.org/grpc v1.20.0
+	gopkg.in/go-gorp/gorp.v2 v2.0.0-20180410155428-6032c66e0f5f
+	gopkg.in/square/go-jose.v2 v2.1.4
+	gopkg.in/yaml.v2 v2.2.2
+)