Skip to content

Commit

Permalink
Update healthcheck related examples and add head support (minio#5650)
Browse files Browse the repository at this point in the history
- Add head method for healthcheck endpoint. Some platforms/users
may use the HTTP Head method to check for health status.

- Add liveness and readiness probe examples in Kubernetes yaml
example docs. Note that readiness probe not added to StatefulSet
example due to kubernetes/kubernetes#27114
  • Loading branch information
nitisht authored Mar 15, 2018
1 parent 1c0c336 commit abffa00
Show file tree
Hide file tree
Showing 6 changed files with 63 additions and 23 deletions.
2 changes: 1 addition & 1 deletion cmd/generic-handlers.go
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@ func guessIsHealthCheckReq(req *http.Request) bool {
return false
}
aType := getRequestAuthType(req)
return req.Method == http.MethodGet && aType == authTypeAnonymous &&
return aType == authTypeAnonymous && (req.Method == http.MethodGet || req.Method == http.MethodHead) &&
(req.URL.Path == healthCheckPathPrefix+healthCheckLivenessPath ||
req.URL.Path == healthCheckPathPrefix+healthCheckReadinessPath)
}
Expand Down
2 changes: 2 additions & 0 deletions cmd/healthcheck-router.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,9 @@ func registerHealthCheckRouter(mux *router.Router) {

// Liveness handler
healthRouter.Methods(http.MethodGet).Path(healthCheckLivenessPath).HandlerFunc(LivenessCheckHandler)
healthRouter.Methods(http.MethodHead).Path(healthCheckLivenessPath).HandlerFunc(LivenessCheckHandler)

// Readiness handler
healthRouter.Methods(http.MethodGet).Path(healthCheckReadinessPath).HandlerFunc(ReadinessCheckHandler)
healthRouter.Methods(http.MethodHead).Path(healthCheckReadinessPath).HandlerFunc(ReadinessCheckHandler)
}
28 changes: 6 additions & 22 deletions docs/healthcheck/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,37 +3,21 @@
Minio server exposes two un-authenticated, healthcheck endpoints - liveness probe and readiness probe at `/minio/health/live` and `/minio/health/ready` respectively.

### Liveness probe

This probe is used to identify situations where the server is running but may not behave optimally, i.e. sluggish response or corrupt backend. Such problems can be *only* fixed by a restart.

Internally, Minio liveness probe handler does a ListBuckets call. If successful, the server returns 200 OK, otherwise 503 Service Unavailable.

When liveness probe fails, Kubernetes like platforms restart the container.

Sample configuration in a Kubernetes `yaml` file.

```yaml
livenessProbe:
httpGet:
path: /minio/health/live
port: 9000
initialDelaySeconds: 10
periodSeconds: 20
```
When liveness probe fails, Kubernetes like platforms restart the container.

### Readiness probe

This probe is used to identify situations where the server is not ready to accept requests yet. In most cases, such conditions recover in some time.

Internally, Minio readiness probe handler checks for total go-routines. If the number of go-routines is less than 1000 (threshold), the server returns 200 OK, otherwise 503 Service Unavailable.

Platforms like Kubernetes *do not* forward traffic to a pod until its readiness probe is successful.
Platforms like Kubernetes *do not* forward traffic to a pod until its readiness probe is successful.

Sample configuration in a Kubernetes `yaml` file.
### Configuration example

```yaml
livenessProbe:
httpGet:
path: /minio/health/ready
port: 9000
initialDelaySeconds: 10
periodSeconds: 20
```
Sample `liveness` and `readiness` probe configuration in a Kubernetes `yaml` file can be found [here](https://github.com/minio/minio/blob/master/docs/orchestration/kubernetes-yaml/minio-standalone-deployment.yaml).
27 changes: 27 additions & 0 deletions docs/orchestration/kubernetes-yaml/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,24 @@ spec:
value: "minio123"
ports:
- containerPort: 9000
# Readiness probe detects situations when Minio server instance
# is not ready to accept traffic. Kubernetes doesn't forward
# traffic to the pod till readiness checks fail.
readinessProbe:
httpGet:
path: /minio/health/ready
port: 9000
initialDelaySeconds: 120
periodSeconds: 20
# Liveness probe detects situations where Minio server instance
# is not working properly and needs restart. Kubernetes automatically
# restarts the pods if liveness checks fail.
livenessProbe:
httpGet:
path: /minio/health/live
port: 9000
initialDelaySeconds: 120
periodSeconds: 20
```

Create the Deployment
Expand Down Expand Up @@ -298,6 +316,15 @@ spec:
volumeMounts:
- name: data
mountPath: /data
# Liveness probe detects situations where Minio server instance
# is not working properly and needs restart. Kubernetes automatically
# restarts the pods if liveness checks fail.
livenessProbe:
httpGet:
path: /minio/health/live
port: 9000
initialDelaySeconds: 120
periodSeconds: 20
# These are converted to volume claims by the controller
# and mounted at the paths mentioned above.
volumeClaimTemplates:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,15 @@ spec:
volumeMounts:
- name: data
mountPath: /data
# Liveness probe detects situations where Minio server instance
# is not working properly and needs restart. Kubernetes automatically
# restarts the pods if liveness checks fail.
livenessProbe:
httpGet:
path: /minio/health/live
port: 9000
initialDelaySeconds: 120
periodSeconds: 20
# These are converted to volume claims by the controller
# and mounted at the paths mentioned above.
volumeClaimTemplates:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,21 @@ spec:
value: "minio123"
ports:
- containerPort: 9000
# Readiness probe detects situations when Minio server instance
# is not ready to accept traffic. Kubernetes doesn't forward
# traffic to the pod while readiness checks fail.
readinessProbe:
httpGet:
path: /minio/health/ready
port: 9000
initialDelaySeconds: 120
periodSeconds: 20
# Liveness probe detects situations where Minio server instance
# is not working properly and needs restart. Kubernetes automatically
# restarts the pods if liveness checks fail.
livenessProbe:
httpGet:
path: /minio/health/live
port: 9000
initialDelaySeconds: 120
periodSeconds: 20

0 comments on commit abffa00

Please sign in to comment.