Add Docker/k8s sample for Sentinel RLS token server and update document

Signed-off-by: Eric Zhao <[email protected]>

sentinel-cluster/sentinel-cluster-server-envoy-rls/Dockerfile

@@ -0,0 +1,10 @@
+FROM java:8-jre
+
+ENV SENTINEL_HOME /sentinel
+
+COPY target/sentinel-envoy-rls-token-server.jar $SENTINEL_HOME/
+
+WORKDIR $SENTINEL_HOME
+
+ENTRYPOINT ["sh", "-c"]
+CMD ["java -Dcsp.sentinel.log.dir=/sentinel/logs/ ${JAVA_OPTS} -jar sentinel-envoy-rls-token-server.jar"]

sentinel-cluster/sentinel-cluster-server-envoy-rls/README.md

@@ -13,3 +13,47 @@ Build the executable jar:
 ```bash
 mvn clean package -P prod
 ```
+
+## Rule configuration
+
+Currently Sentinel RLS token server supports dynamic rule configuration via the yaml file.
+The file may provide rules for one *domain* (defined in Envoy's conf file).
+In Envoy, one rate limit request might carry multiple *rate limit descriptors*
+(which will be generated from [Envoy rate limit actions](https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/route/route.proto#envoy-api-msg-route-ratelimit)).
+One rate limit descriptor may have multiple entries (key-value pair).
+We may set different threshold for each rate limit descriptors.
+
+A sample rule configuration file:
+
+```yaml
+domain: foo
+descriptors:
+  - resources:
+    - key: "destination_cluster"
+      value: "service_httpbin"
+    count: 1
+```
+
+This rule only takes effect for domain `foo`. It will limit the max QPS to 1 for
+all requests targeted to the `service_httpbin` cluster.
+
+We need to provide the path to yaml file via the `SENTINEL_RLS_RULE_FILE_PATH` env
+(or `-Dcsp.sentinel.rls.rule.file` opts). Then as soon as the content in the rule file has been changed,
+Sentinel will reload the new rules from the file to the `EnvoyRlsRuleManager`.
+
+We may check the logs in `~/logs/csp/sentinel-record.log.xxx` to see whether the rules has been loaded.
+We may also retrieve the converted `FlowRule` via the command API `localhost:8719/cluster/server/flowRules`.
+
+## Configuration items
+
+The configuration list:
+
+| Item (env) | Item (JVM property) | Description | Default Value | Required |
+|--------|--------|--------|--------|--------|
+| `SENTINEL_RLS_GRPC_PORT` | `csp.sentinel.grpc.server.port` | The RLS gRPC server port | **10240** | false |
+| `SENTINEL_RLS_RULE_FILE_PATH` | `csp.sentinel.rls.rule.file` | The path of the RLS rule yaml file | - | **true** |
+| `SENTINEL_RLS_ACCESS_LOG` | - | Whether to enable the access log (`on` for enable) | off | false |
+
+## Samples
+
+- [Kubernetes sample](./sample/k8s)

sentinel-cluster/sentinel-cluster-server-envoy-rls/sample/k8s/README.md

@@ -0,0 +1,105 @@
+# Sentinel Envoy RLS - Kubernetes sample
+
+This sample will illustrate how to use Sentinel RLS token server with Envoy in Kubernetes clusters.
+
+## Build the Docker image
+
+We could use the pre-built Docker image: `registry.cn-hangzhou.aliyuncs.com/sentinel-docker-repo/sentinel-envoy-rls-server:latest`
+
+We can also manually build the Docker image in the `sentinel-cluster-server-envoy-rls` directory:
+
+```bash
+docker build -t "sentinel/sentinel-envoy-rls-server:latest" -f ./Dockerfile .
+```
+
+## Deploy Sentinel RLS token server
+
+Next we could deploy the Sentinel RLS token server in the K8S cluster.
+We've provided a deployment template for Sentinel RLS token server in `sentinel-rls.yml`.
+It includes:
+
+- A `ConfigMap` that contains the cluster flow control rule for Envoy global rate limiting.
+  This will be mounted as a file in the target `Deployment`, so that the Sentinel RLS token server
+  could load the rules dynamically as soon as the rule in the `ConfigMap` has been updated.
+- A `Deployment` for Sentinel RLS token server. It will mount the `ConfigMap` as a volume
+  for dynamic rule configuration.
+- A `Service` that exports the Sentinel command port (8719) and the RLS gRPC port (by default 10245)
+  on a cluster-internal IP so that the Envoy pods could communicate with the RLS server.
+
+The sample rate limiting rule in the `sentinel-rule-cm`:
+
+```yaml
+domain: foo
+descriptors:
+  # For requests to the "service_httpbin" cluster, limit the max QPS to 1
+  - resources:
+    - key: "destination_cluster"
+      value: "service_httpbin"
+    count: 1
+```
+
+You may enable the access log in the Sentinel RLS token server (output to console)
+via the `SENTINEL_RLS_ACCESS_LOG` env:
+
+```yaml
+env:
+  - name: SENTINEL_RLS_ACCESS_LOG
+    value: "on"
+```
+
+You may also append JVM opts via the `JAVA_OPTS` env.
+
+After preparing the yaml template, you may deploy the Sentinel RLS token server:
+
+```bash
+kubectl apply -f sample/k8s/sentinel-rls.yml
+```
+
+## Deploy Envoy
+
+Next we could deploy the Envoy instances in the K8S cluster. If you've already had Envoy instances running,
+you could configure the address (`sentinel-rls-service`) and the port (`10245`)
+of the rate limit cluster in your Envoy configuration.
+
+We've provided a deployment template for Envoy in `envoy.yml`.
+It includes:
+
+- A `ConfigMap` that contains the configuration for Envoy.
+  This will be mounted as a file in the target `Deployment`, which will be loaded as the configuration
+  file by Envoy.
+- A `Deployment` for Envoy. It will mount the `ConfigMap` as a volume
+  for configuration.
+- A `Service` that exports the Envoy endpoint port (by default 10000) on a cluster-internal IP
+  so that it could be accessible from other pods. If you need external access, you could choose the
+  `LoadBalancer` type or add a frontend ingress.
+
+In the sample, we have two [Envoy clusters](https://www.envoyproxy.io/docs/envoy/latest/api-v2/clusters/clusters):
+
+- `service_httpbin`: HTTP proxy to `httpbin.org`
+- `rate_limit_cluster`: the cluster of the Sentinel RLS token server
+
+This route configuration tells Envoy to route incoming requests to `httpbin.org`. In the `http_filters` conf item,
+we added the `envoy.rate_limit` filter to the filter chain so that the global rate limiting is enabled.
+We set the rate limit domain as `foo`, which matches the item in the Envoy RLS rule.
+In the `route_config`, we provide the rate limit action: `{destination_cluster: {}}`, which will
+generate the rate limit descriptor containing the actual target cluster name (e.g. `service_httpbin`).
+Then we could set rate limit rules for each target clusters.
+
+After preparing the yaml template, you may deploy the Envoy instance:
+
+```bash
+kubectl apply -f sample/k8s/envoy.yml
+```
+
+## Test the rate limiting
+
+Now it's show time! We could visit the URL `envoy-service:10000/json` in K8S pods.
+Since we set the max QPS to 1, we could emit concurrent requests to the URL, and we
+could see the first request passes, while the latter requests are blocked (status 429):
+
+![image](https://user-images.githubusercontent.com/9434884/68571798-d0a46500-049e-11ea-8488-5e90f56f23a5.png)
+
+## Update the rules dynamically
+
+You could update the rules in the `sentinel-rule-cm` ConfigMap. Once the content is updated,
+Sentinel will perceive the changes and load the new rules to `EnvoyRlsRuleManager`.

sentinel-cluster/sentinel-cluster-server-envoy-rls/sample/k8s/envoy.yml

@@ -0,0 +1,132 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: envoy-cm
+data:
+  envoy-yml: |-
+    admin:
+      access_log_path: /tmp/admin_access.log
+      address:
+        socket_address:
+          protocol: TCP
+          address: 127.0.0.1
+          port_value: 9901
+    static_resources:
+      listeners:
+      - name: listener_0
+        address:
+          socket_address:
+            protocol: TCP
+            address: 0.0.0.0
+            port_value: 10000
+        filter_chains:
+        - filters:
+          - name: envoy.http_connection_manager
+            typed_config:
+              "@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
+              stat_prefix: ingress_http
+              route_config:
+                name: local_route
+                virtual_hosts:
+                - name: local_service
+                  domains: ["*"]
+                  routes:
+                  - match:
+                      prefix: "/"
+                    route:
+                      host_rewrite: httpbin.org
+                      cluster: service_httpbin
+                  rate_limits:
+                    - stage: 0
+                      actions:
+                        - {destination_cluster: {}}
+              http_filters:
+                - name: envoy.rate_limit
+                  config:
+                    domain: foo
+                    stage: 0
+                    rate_limit_service:
+                      grpc_service:
+                        envoy_grpc:
+                          cluster_name: rate_limit_cluster
+                        timeout: 0.25s
+                - name: envoy.router
+      clusters:
+      - name: service_httpbin
+        connect_timeout: 0.5s
+        type: LOGICAL_DNS
+        # Comment out the following line to test on v6 networks
+        dns_lookup_family: V4_ONLY
+        lb_policy: ROUND_ROBIN
+        load_assignment:
+          cluster_name: service_httpbin
+          endpoints:
+          - lb_endpoints:
+            - endpoint:
+                address:
+                  socket_address:
+                    address: httpbin.org
+                    port_value: 80
+      - name: rate_limit_cluster
+        type: LOGICAL_DNS
+        connect_timeout: 0.25s
+        lb_policy: ROUND_ROBIN
+        http2_protocol_options: {}
+        load_assignment:
+          cluster_name: rate_limit_cluster
+          endpoints:
+          - lb_endpoints:
+            - endpoint:
+                address:
+                  socket_address:
+                    address: sentinel-rls-service
+                    port_value: 10245
+---
+apiVersion: apps/v1beta2
+kind: Deployment
+metadata:
+  name: envoy-deployment-basic
+  labels:
+    app: envoy
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: envoy
+  template:
+    metadata:
+      labels:
+        app: envoy
+    spec:
+      containers:
+        - name: envoy
+          image: envoyproxy/envoy:v1.12.0
+          ports:
+            - containerPort: 10000
+          command: ["envoy"]
+          args: ["-c", "/tmp/envoy/envoy.yaml"]
+          volumeMounts:
+            - name: envoy-config
+              mountPath: /tmp/envoy
+      volumes:
+        - name: envoy-config
+          configMap:
+            name: envoy-cm
+            items:
+              - key: envoy-yml
+                path: envoy.yaml
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: envoy-service
+  labels:
+    name: envoy-service
+spec:
+  type: ClusterIP
+  ports:
+    - port: 10000
+      targetPort: 10000
+      protocol: TCP
+  selector:
+    app: envoy

sentinel-cluster/sentinel-cluster-server-envoy-rls/sample/k8s/sentinel-rls.yml

@@ -0,0 +1,68 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: sentinel-rule-cm
+data:
+  rule-yaml: |-
+    domain: foo
+    descriptors:
+      - resources:
+        - key: "destination_cluster"
+          value: "service_httpbin"
+        count: 1
+---
+apiVersion: apps/v1beta2
+kind: Deployment
+metadata:
+  name: sentinel-rls-server
+  labels:
+    app: sentinel
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: sentinel
+  template:
+    metadata:
+      labels:
+        app: sentinel
+    spec:
+      containers:
+        - name: sentinelserver
+          # You could replace the image with your own image here
+          image: "registry.cn-hangzhou.aliyuncs.com/sentinel-docker-repo/sentinel-envoy-rls-server:latest"
+          imagePullPolicy: Always
+          ports:
+            - containerPort: 10245
+            - containerPort: 8719
+          volumeMounts:
+            - name: sentinel-rule-config
+              mountPath: /tmp/sentinel
+          env:
+            - name: SENTINEL_RLS_RULE_FILE_PATH
+              value: "/tmp/sentinel/rule.yaml"
+      volumes:
+        - name: sentinel-rule-config
+          configMap:
+            name: sentinel-rule-cm
+            items:
+              - key: rule-yaml
+                path: rule.yaml
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: sentinel-rls-service
+  labels:
+    name: sentinel-rls-service
+spec:
+  type: ClusterIP
+  ports:
+    - port: 8719
+      targetPort: 8719
+      name: sentinel-command
+    - port: 10245
+      targetPort: 10245
+      name: sentinel-grpc
+  selector:
+    app: sentinel