Skip to content

Commit

Permalink
update balancer-monolith connection direction (#8)
Browse files Browse the repository at this point in the history
closes #5
  • Loading branch information
dyc3 authored Sep 14, 2023
1 parent f33efa9 commit ad6278c
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 17 deletions.
16 changes: 6 additions & 10 deletions join-process.typ
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= Join Process <Chapter::JoinProcess>

There are 2 types of nodes in this architecture. The first type is the OTT Monolith \index{monolith}, which refers to the current node.js monolithic server. The second type is the Smart Load Balancer \index{load balancer}, which needs to know which monoliths control which rooms, and how to route requests to the correct monolith.
There are 2 types of nodes in this architecture. The first type is the OTT Monolith, which refers to the current node.js monolithic server. The second type is the Smart Load Balancer, which needs to know which monoliths control which rooms, and how to route requests to the correct monolith.

For the sake of simplicity, the initial implementation of the Balancer will be a single node. This can be upgraded to a cluster of load balancers in the future.

Expand All @@ -21,7 +21,7 @@ If the client fails to provide an auth token, the Balancer must terminate the co

Balancers must be able to determine which Monolith nodes are hosting which rooms.

Monolith nodes must gossip \cite{wikigossip} to Balancer nodes to inform them of the rooms that they have loaded. This also implies that they must notify all Balancers of their existance on startup. The Balancer must maintain a hashmap of room names to Monolith nodes.
Monolith nodes must gossip to Balancer nodes to inform them of the rooms that they have loaded. This also implies that they must notify all Balancers of their existance on startup. The Balancer must maintain a hashmap of room names to Monolith nodes.

They must also maintain a hashmap of monolith nodes to a list of rooms that they are hosting to verify that only one monolith is hosting a room at a time. When the gossip is received, the Balancer must check to see if In the event that a Balancer finds that more than one Monolith is hosting a room, it must randomly select one of the Monoliths to be the authoritative node for that room, and inform the other Monoliths that they must unload the room. This method will not work as effectively if there is more than one Balancer, but it is a simple solution for the initial implementation.

Expand All @@ -32,18 +32,14 @@ Monoliths must gossip:
- when a room is unloaded
- at a maximum interval of 20 seconds (eg. if 20 seconds pass without a room being loaded or unloaded, the Monolith must gossip)


The gossip message must contain: (see @Figure::gossip-class-diag)

- a list of rooms that the Monolith is hosting
- the load of the Monolith


The Monoliths will know where to send the gossip messages by reading a configuration file. This file will contain a list of Balancer nodes, and the Monolith will send the gossip messages to each of them.

#figure(
image("figures/gossip-class-diag.png"),
caption: "Gossip class diagram."
image("figures/gossip-class-diag.png", width: 20%),
caption: "Simplified gossip class diagram. Actual implementation will vary."
) <Figure::gossip-class-diag>

== Creating or Loading Rooms
Expand Down Expand Up @@ -76,8 +72,8 @@ The Balancer must be able to select the Monolith that is most appropriate to han
- Otherwise, the Balancer must select the Monolith with the lowest number of loaded rooms. The Monolith will load the room on demand, if it exists in the database.

== Joining an Unloaded Room
When a user tries to join a permanent room in OpenTogetherTube, the request is first received by the Balancer\index{balancer}.
The Balancer forwards the request to one of the available Monoliths \index{monolith} based on the current load balancing algorithm (See @Section::MonolithNodeSelection).
When a user tries to join a permanent room in OpenTogetherTube, the request is first received by the Balancer.
The Balancer forwards the request to one of the available Monoliths based on the current load balancing algorithm (See @Section::MonolithNodeSelection).
This process is shown in @Figure::unloaded-room.

#figure(
Expand Down
8 changes: 3 additions & 5 deletions protocol.typ
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

== Messaging Protocol

The Balancer must maintain a maximum of one websocket connection to each Monolith node. This implies that the Balancer must be able to convey the user identity of messages sent to the Monolith.
The Monolith must maintain a maximum of one websocket connection to each Balancer.

The client communicates over the websocket with messages in JSON format, so the Balancer should do the same. The Balancer will send messages to the Monolith in the following format:

Expand Down Expand Up @@ -34,8 +34,6 @@ For broadcast messages, the Monolith can omit the `id` field.
}
```



=== Messages sent during Join and Leave

When a client opens a websocket connection, the client must immediately send an "auth" message. This is because the browser's websocket API does not allow for the client to send headers with the initial connection request.
Expand Down Expand Up @@ -86,6 +84,6 @@ These messages conform to the the protocol defined by the types in the #link("ht

== Messages sent during Monolith connection startup

When a Monolith starts up, and it has balancers configured, it must initiate a websocket connection to each of the balancers. The Monolith must send an "init" message to the Balancer to inform it of the port that it is listening for HTTP requests on, and an auth token to verify authenticity.
When a Monolith starts up, and load balancing is enabled, it must listen on a seperate port for incoming balancer connections. Balancers will initiate connections based on their configured Monolith discovery method. Upon accepting a new connection, the Monolith must send an "init" message to the Balancer to inform it of the port that it is listening for normal HTTP requests on, and an auth token to verify authenticity.

Once the Balancer receives this message, the connection is considered established, and the Monolith and Balancer can begin sending messages to each other.
Once the Balancer receives this message, the connection is considered fully established, and the Monolith and Balancer can begin sending messages to each other.
4 changes: 2 additions & 2 deletions solution-overview.typ
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
The Monolith's current internals is shown in @Figure::monolith-class-current.

#figure(
image("figures/monolith-class-current.png"),
image("figures/monolith-class-current.png", width: 40%),
caption: "Class diagram for the Monolith's current internals"
) <Figure::monolith-class-current>

It suffers from years of ad-hoc development, and is not designed to scale. It's riddled with technical debt from previous attempts at horizontal scaling \index{horizontal scaling}. In it's current state, it is not possible to add more monoliths without causing room synchronization issues. The solution to this problem is to use a load balancer \index{load balancer}.
It suffers from years of ad-hoc development, and is not designed to scale. It's riddled with technical debt from previous attempts at horizontal scaling. In it's current state, it is not possible to add more monoliths without causing room synchronization issues. The solution to this problem is to use a load balancer.

== A Smart Load Balancer

Expand Down

0 comments on commit ad6278c

Please sign in to comment.