Skip to content

Commit

Permalink
optimize docs for load balance (bfenetworks#508)
Browse files Browse the repository at this point in the history
  • Loading branch information
mileszhang2016 authored May 26, 2020
1 parent a639f90 commit c6290b9
Show file tree
Hide file tree
Showing 6 changed files with 123 additions and 101 deletions.
90 changes: 54 additions & 36 deletions docs/en_us/introduction/balance.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,74 +2,92 @@

## Sub cluster level load balance

- In sub cluster level, load balance rules also can be configured. The rules define traffic weight which is distributed to each sub cluster。
### Overview

- A special virtual sub cluster "BLACKHOLE" could be used to discard traffic。
- Usually one cluster has more than one sub clusters. In BFE, it is supported to
define weights for distributing traffic to each sub cluster.

- Also there is a special virtual sub cluster "BLACKHOLE" for each cluster.
- Traffic allocated to "BLACKHOLE" will be discarded.
- "BLACKHOLE" is used to prevent traffic overloading to the whole cluster.

- Load balance between sub clusters, it is a very important feature for BFE.
This feature is very useful in multi-IDCs scenario.

### Example

- Consider following configuration
- Two IDC:IDC1, IDC2
- Two BFE cluster:BFE1, BFE2
- Two backend sub cluster:sub-cluster-1, sub-cluster-2
- Consider the following scenario
- Two IDCs:IDC_1, IDC_2
- Two BFE clusters:BFE_1, BFE_2
- Two backend sub clusters:SubCluster_1, SubCluster_2

- In BFE cluster(BFE1 and BFE2):
- BFE1:{sub-cluster-1:w11,sub-cluster-2:w12, Blackhole:w1B}
- BFE2:{sub-cluster-1:w21,sub-cluster-2:w22, Blackhole:w2B}
- In BFE clusters, weights for distributing traffic to each sub cluster can be configured as follows:
- BFE_1:{SubCluster_1: W11,SubCluster_2: W12, Blackhole: W1B}
- BFE_2:{SubCluster_1: W21,SubCluster_2: W22, Blackhole: W2B}

- Based on the weight in configuration, BFE distribute traffic to backend sub cluster.
- For example,if weight configuration {w11, w12, w1B} ={45,45,10}, percent of traffic to sub cluster is sub-cluster-1, sub-cluster-2 and Blackhole is 45%, 45% and 10%.
- Based on the above configuration, BFE clusters distribute traffic to backend sub clusters.
- For example,if configuration of BFE_1 is {W11,W12, W1B} ={45,45,10}, traffic percentage to SubCluster_1, SubCluster_2 and Blackhole is 45%, 45% and 10%.

![Forwarding Table](../../images/gslb.png)

## Instance level load balance

- Usually, a sub cluster is composed of multiple instances. Within sub cluster, WRR(weighted round robin) is used to distribute request among instance。
- Instance can be assign different weight based on its capacity。
- Usually, a sub cluster is composed of multiple instances.
- Within sub cluster, several policies is provided for distributing traffic among instances. e.g.,
- WRR(Weighted Round Robin)
- WLC(Weighted Least Connection)
- Instances can be assigned with different weights based on their capacity。

## Health check
BFE do health check for each backend instance. A instance has following two states:

- NORMAL state:the instance act normally in processing request.
- CHECKING state:the instance is abnormal and can't process request. BFE do health check periodically in this state.
BFE supports health check for each backend instance.

BFE maintains a state machine for each instance, with two states:

State switch:
- NORMAL:the instance acts normally in processing request.
- CHECKING:the instance fails to process request and response to BFE. BFE starts health check for this instance, until it acts normally again.

The condition for switching between states:
- NORMAL to CHECKING, when:
- Consecutive failure, in connecting or sending request to the instance, exceed a threshhod.
- Number of consecutive failures, in connecting or sending request to the instance, exceeds a threshhod.

- CHECKING to NORMAL, when:
- BFE receive correct response from backend instance for health check request.
- BFE receives correct response for health check request from the instance.


## Retry in failure

If request routing fail, BFE support retry the request in two levels
If request forwarding fails, BFE supports retry in two ways

- Re-route request within same sub cluster.
- Re-route request to other sub cluster.
- In-Sub-Cluster Retry: Re-forward request within the same sub cluster.
- Cross-Sub-Cluster Rery: Re-forward request to other sub cluster.

## Connection pool

TCP connection between BFE and backend instance support:
Two ways are supported for TCP connection between BFE and backend instance:

- Short-Lived Connection:BFE forwards each request to backend instance with a new established TCP connection.

- short-lived connection:BFE route each request request to backend server with a new established TCP connection.
- Connection Pool:
- BFE maintains a connection pool for each backend instance.

- connection pool:
- BFE maintain connection pool to instances.
- To an incoming request:
- if there is an available connection, reuse it.
- else establish a new TCP connection.
- For a request forwarded to given backend instance:
- If there is an available idle connection in the connection pool, take it from the pool and reuse it.
- Otherwise, establish a new TCP connection.

- After finish processing a request via a connection:
- If current size of connection pool is less than configured number, the connection is added into the pool.
- Else close the connection directly.
- After processing for a request is finished, for the connection used for this request:
- If number of idle connections in the pool is less than configured capacity, the connection is added into the pool.
- Otherwise, close the connection directly.

## Session stickiness

BFE support session stickiness based on following identity of request request:
BFE supports session stickiness.

Session could be defined based on following infos of the request:
- Source IP
- Field in request header, cookie etc.

Keep session in different routing level:
- Sub cluster level: request of a session is sent to same sub cluster (may be different instance in this sub cluster).
- Instance level: request of a session is sent to same instance.
Session stickiness is supported for the following two levels:
- Sub cluster level: requests of the same session are forwarded to the same sub cluster (may be different instances in this sub cluster).
- Instance level: requests of the same session are fowarded to the same instance.

32 changes: 20 additions & 12 deletions docs/en_us/introduction/route.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,32 @@
# Request Routing

In BFE configuration, a "product" could be composed of multiple clusters. User can define how requests are routed among the clusters. Request routing is based on the content of HTTP request.
## Overview

## Rule configuration
- In BFE [forwarding model](./forward_model.md), after "product name" for one request is determined, the destionation cluster should be identified.
- A Forwarding Table is provided for each product inside BFE.
- For each request, the forwarding table for the determined product is searched, and destination cluster is identified.

- Fields in HTTP Header is used to define routing rule for distributing traffic in cluster level, for example:
- host, path, query, cookie, method, etc.
- BFE provide a "Condition" expression to define how to route request with special header. This is routing rule in cluster level load balance.
- If multiple rules are configured, BFE would match the rules in sequence. Matching procedure stop if one rule is matched.
## Forwarding Table

- Forwarding table is composed by one or more "forwarding rules".
- Each forwarding rule has two parts: condition(for matching the request), destionation cluster.
- Condition is expressed in [Condition Expression](../condition).
- For a request, multiple rules in forwarding table are searched up to down.(i.e., the order of forwarding rules is very important.) The procedure will stop if some rule is matched.
- There must be one Default Rule in the forwarding table. If no other rules is matched for a request, the destionation cluster defined in Default Rule is returned.

## Example

- A product "demo", which needs process three kinds of traffic: static content traffic, "post" traffic, other traffic, then we can define three clusters :
## Example

- A product "demo" has three clusters:
- demo-static:serve static content
- demo-post:serve post request
- demo-main: serve other traffic

- To BFE configuration, following routing rules can be added:
- Rule 1: req_path_prefix_in("/static", false) -> demo-static, which means requests with path prefixed with "/static" will be routed to cluster demo-static.
- Rule 2: req_method_in("POST")&&req_path_prefix_in("/setting",false) -> demo-post, which means that request which use method "POST" and is prefixed with "/setting" will be routed to cluster "demo-post".
- Rule 3: default -> demo-main, which means all request which doesn't match above rules will be sent to cluster "demo-main".
- The expected scenarios:
- Requests with path prefixed with "/static" will be forwarded to cluster demo-static.
- Request using method "POST" and prefixed with "/setting" will be forwarded to cluster "demo-post".
- Other request will be forwarded to cluster "demo-main".

- The corresponding forwarding table is shown as follows.

![Forwarding Table](../../images/forwarding_table.png)
Binary file added docs/images/forwarding_table.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/gslb.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
52 changes: 28 additions & 24 deletions docs/zh_cn/introduction/balance.md
Original file line number Diff line number Diff line change
@@ -1,46 +1,50 @@
# 负载均衡
# 集群内负载均衡

## 集群级别负载均衡
## 概述

### 机制说明
- 一个集群往往包含多个子集群。在BFE中,可以针对集群配置其多个子集群间的流量分配比例。

- BFE可以配置一个集群中的子集群的流量分配比例。
- 另外,对每个集群,还包含一个虚拟的子集群,BLACKHOLE(黑洞)。
- 分给黑洞子集群的流量会被主动丢弃
- 黑洞用来防止集群整体过载

- 子集群粒度的负载均衡,这是BFE的一个重要功能。这个功能在多IDC场景非常有用。

- 该配置中可以包含一个虚拟的子集群,BLACKHOLE(黑洞)。分向黑洞的流量会被丢弃。它可以用来防止子集群过载。

### 示例场景
## 示例场景

- 假设基本场景如下图所示
- 示例场景如下图所示,包含

- 共两个IDC:IDC1和IDC2
- 两个IDC:IDC_1和IDC_2

- 两个BFE集群:BFE1和BFE2
- 两个BFE集群:BFE_1和BFE_2

- 后端产品线的集群有两个子集群:子集群1和子集群2
- 后端集群有两个子集群:SubCluster_1和SubCluster_2

- BFE集群(BFE1和BFE2)可以根据子集群(外加Blackhole)分流比例的配置,分发流量。如:
- 可以针对BFE集群,设置子集群的分流比例,如:

- BFE1集群的分流配置为:{子集群1:w11,子集群2:w12, Blackhole:w1B}
- BFE_1集群的分流配置为:{SubCluster_1: W11,SubCluster_2: W12, Blackhole: W1B}

- BFE2集群的分流配置为:{子集群1:w21,子集群2:w22, Blackhole:w2B}
- BFE_2集群的分流配置为:{SubCluster_1: W21,SubCluster_2: W22, Blackhole: W2B}

- BFE实例根据上述配置做WRR调度(加权轮询),向子集群转发请求
- BFE实例根据上述配置做WRR调度(加权轮询),向子集群转发请求

- 例如,当BFE1的分流配置{w11,w12,
w1B}为{45,45,10}时,该集群在BFE1转发给子集群1、子集群2、Blackhole的流量比例依次为:45%、45%、10%。
- 例如,当BFE_1的分流配置{W11,W12, W1B}为{45,45,10}时,BFE_1转发给SubCluster_1、SubCluster_1、Blackhole的流量比例依次为:45%、45%、10%。

- 通过修改上述配置,可以将流量在不同的子集群之间切换,实现负载均衡、快速止损、过载保护等目的。

![Forwarding Table](../../images/gslb.png)

## 实例级别负载均衡
# 子集群内的负载均衡

## 负载均衡算法

### 机制说明
- 在同一子集群内的多个实例间,BFE支持使用多种负载均衡算法:
* WRR: 加权轮询算法
* WLC: 加权最小连接数
- WRR: 加权轮询算法
- WLC: 加权最小连接数
- 如实例的处理能力不同,可以分配不同的权重。

### 健康检查
## 健康检查

对于BFE下游的每一个实例,BFE会维护一个状态机,包含2个状态:

Expand All @@ -60,7 +64,7 @@
- BFE从该实例收到健康检查请求的正确响应


### 失败重试机制
## 失败重试机制

BFE在转发时,支持以下两种失败重试机制:

Expand All @@ -73,7 +77,7 @@ BFE在转发时,支持以下两种失败重试机制:
- 在原目标子集群之外、使用另外一个子集群进行重试


### 连接池
## 连接池

BFE和下游实例的连接支持两种方式:

Expand All @@ -93,7 +97,7 @@ BFE和下游实例的连接支持两种方式:
- 如果连接池中的idle连接数量小于连接池的大小,则将当前使用的连接放入连接池
- 如果连接池中的idle连接数量大于等于连接池的大小,则关闭当前使用的连接

### 会话保持
## 会话保持

BFE向下游转发请求时,支持将相同来源请求,转发至固定的业务后端(某个子集群或某个实例)

Expand Down
50 changes: 21 additions & 29 deletions docs/zh_cn/introduction/route.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,35 @@
# 集群间分流

## 机制说明
- 集群间分流是指BFE收到请求,基于收到消息内容进行路由,确定由产品线中哪一个集群来处理消息。

## 示例场景
## 概述

- 对于一个产品线的流量,可能需要定义不同的集群来处理。例如,对产品线(demo),可以定义3种服务集群,实现对不同类型的流量的处理:
- 在BFE的[接入转发流程](./forward_model.md)中,在确定产品线后,要进一步确定处理该请求的目标集群
- 在BFE内,为每个产品线维护一张“转发表”
- 对每个属于该产品线的请求,查询转发表,获得目标集群

- 静态集群(demo-static):服务静态流量
- post集群(demo-post):服务post流量
- main集群(demo-main):服务其他流量

- BFE支持根据配置的分流规则, 将流量转发到以上不同的集群。
## 转发表

## 分流的配置方式
- 转发表由多条“转发规则”组成
- 每条转发规则包括两部分:匹配条件,目标集群
- 匹配条件使用“[条件表达式(Condition Expression)](../condition)”来表述
- 多条转发规则顺序执行。只要命中任何一条转发规则,就会结束退出
- 转发表必须包含“默认规则(Default)”。在所有转发规则都没有命中的时候,执行默认规则

- BFE基于请求内容进行条件判断,来决定分流
- 可以使用的信息包括:host,path, query,cookie,method等
- BFE提供了一个称为Condition的描述机制
## 示例

- 产品线demo,包含以下3种服务集群:
+ 静态集群(demo-static):服务静态流量
+ post集群(demo-post):服务post流量
+ main集群(demo-main):服务其他流量
- 期望的转发条件如下:
+ 对于path以"/static"为前缀的,都发往demo-static集群
+ 请求方法为"POST"、且path以"/setting"为前缀的,都发往demo-post
+ 其它请求,都发往demo-main
- 对应以上要求,产品线demo的转发表如下图所示
![Forwarding Table](../../images/forwarding_table.png)

对于上面的例子,我们可以demo这个产品线创建了3条分流规则

- 第一条:对于path以"/static"作为前缀的,都发往demo-static

- 第二条:请求方法为"POST"、并且path是以"/setting"为前缀的,都发往demo-post

- 第三条:如果以上两条规则没有命中,则凡是这个产品的流量、都发往demo-main

## 配置分流规则的注意事项

在配置一个产品线的分流规则时,需要注意以下事项:

- 多条分流规则之间是有序的

- 对于一个HTTP请求,会按照从上至下的顺序,逐条去尝试匹配分流条件
- 只要命中任何一个分流规则的条件,就会结束

- 必须存在一个"default规则"

- default规则永远处于规则列表的最后一条
- 如果其他分流规则都没有命中,则请求会发往default规则所指定的集群

0 comments on commit c6290b9

Please sign in to comment.