Skip to content

Commit

Permalink
rename monolith discovery to service discovery (#312)
Browse files Browse the repository at this point in the history
related: #306
  • Loading branch information
dyc3 authored Feb 29, 2024
1 parent c6f0f2f commit 80d9a89
Show file tree
Hide file tree
Showing 14 changed files with 84 additions and 84 deletions.
2 changes: 1 addition & 1 deletion balancer-design.typ
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
caption: "Class diagram showing the structure and relationships between types in the Balancer.",
) <Figure::balancer-internals-class>

Shown in @Figure::balancer-internals-class is the internal structure of the Balancer. The Balancer will discover Monoiths using the `MonolithDiscoverer` (a process further described in @Chapter::MonolithDiscovery), and the `MonolithConnectionManager` will establish connections to each Monolith. `Balancer` will update `BalancerContext` according to the messages it receives from `BalancerLink`. `Balancer` will then use `BalancerContext` to route messages to the appropriate Monoliths. Clients work similarly, except that they establish connections to the Balancer via `BalancerService`. `BalancerService` handles proxying HTTP requests to the appropriate Monoliths, and also accepting and upgrading WebSocket connections.
Shown in @Figure::balancer-internals-class is the internal structure of the Balancer. The Balancer will discover Monoiths using the `MonolithDiscoverer` (a process further described in @Chapter::ServiceDiscovery), and the `MonolithConnectionManager` will establish connections to each Monolith. `Balancer` will update `BalancerContext` according to the messages it receives from `BalancerLink`. `Balancer` will then use `BalancerContext` to route messages to the appropriate Monoliths. Clients work similarly, except that they establish connections to the Balancer via `BalancerService`. `BalancerService` handles proxying HTTP requests to the appropriate Monoliths, and also accepting and upgrading WebSocket connections.


#figure(
Expand Down
44 changes: 22 additions & 22 deletions figures/discovery/discovery-class.mmd
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
classDiagram
class MonolithDiscoverer {
class ServiceDiscoverer {
<<trait>>

+discover() Result~Vec~MonolithConnectionConfig~~
+discover() Result~Vec~ConnectionConfig~~
+mode() DiscoveryMode
}
note for MonolithDiscoverer "Abstraction over whatever method \nis actually used to discover Monoliths"
note for ServiceDiscoverer "Abstraction over whatever method \nis actually used to discover services"

class DiscoveryMode {
<<enum>>
Expand All @@ -14,44 +14,44 @@ classDiagram
}

class DiscoveryTask {
discovery: dyn MonolithDiscovery
discovery: dyn ServiceDiscoverer

monoliths: HashSet~MonolithConnectionConfig~
discovery_tx: Sender~MonolithDiscoveryMsg~
monoliths: HashSet~ConnectionConfig~
discovery_tx: Sender~ServiceDiscoveryMsg~

+new(discovery: MonolithDiscovery, discovery_tx: Sender~MonolithDiscoveryMsg~)
+new(discovery: ServiceDiscoverer, discovery_tx: Sender~ServiceDiscoveryMsg~)
+do_continuous_discovery()
-do_discovery()
}

class MonolithDiscoveryMsg {
added: Vec~MonolithConnectionConfig~
removed: Vec~MonolithConnectionConfig~
class ServiceDiscoveryMsg {
added: Vec~ConnectionConfig~
removed: Vec~ConnectionConfig~
}

class MonolithConnectionManager {
discovery_rx: Receiver~MonolithDiscoveryMsg~
discovery_rx: Receiver~ServiceDiscoveryMsg~
link: BalancerLink

monoliths: HashSet~MonolithConnectionConfig~
monoliths: HashSet~ConnectionConfig~
%% There's a mermaid-cli bug that prevents us from using the line below
%%connection_tasks: HashMap~MonolithConnectionConfig, ActiveConnection~
%%connection_tasks: HashMap~ConnectionConfig, ActiveConnection~
connection_tasks: HashMap

+new(discovery_rx: Receiver~MonolithDiscoveryMsg~, link: BalancerLink) Self
+new(discovery_rx: Receiver~ServiceDiscoveryMsg~, link: BalancerLink) Self
+do_connection_job()
}

class MonolithConnectionConfig {
class ConnectionConfig {
host: HostOrIp
port: u16
}

DiscoveryTask --> MonolithDiscoveryMsg : sends
DiscoveryTask --> ServiceDiscoveryMsg : sends
DiscoveryTask --> MonolithConnectionManager : connected via one way channel
MonolithDiscoveryMsg --* MonolithConnectionConfig : collects
MonolithDiscoveryMsg --> MonolithConnectionManager : received by
MonolithDiscoverer --o MonolithConnectionConfig : collects
DiscoveryTask --> MonolithDiscoverer : calls
MonolithDiscoverer --> DiscoveryMode
MonolithConnectionManager ..> MonolithConnectionConfig : uses
ServiceDiscoveryMsg --* ConnectionConfig : collects
ServiceDiscoveryMsg --> MonolithConnectionManager : received by
ServiceDiscoverer --o ConnectionConfig : collects
DiscoveryTask --> ServiceDiscoverer : calls
ServiceDiscoverer --> DiscoveryMode
MonolithConnectionManager ..> ConnectionConfig : uses
2 changes: 1 addition & 1 deletion figures/discovery/discovery-class.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions figures/discovery/discovery-sequence.mmd
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ sequenceDiagram
participant MCM as :MonolithConnectionManager
box Discovery Module
participant DT as DiscoveryTask
participant MD as :MonolithDiscoverer
participant MD as :ServiceDiscoverer
end

DT->>+MD: Poll for Monoliths
Expand All @@ -12,5 +12,5 @@ sequenceDiagram
Note over DT: Builds a Discovery update message
DT->>MCM: Send Discovery update over channel
Note over MCM: Connects to discovered Monoliths


2 changes: 1 addition & 1 deletion figures/discovery/discovery-sequence.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 0 additions & 1 deletion figures/discovery/monolith-discoverers.svg

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,47 +1,47 @@
classDiagram
direction BT
class MonolithDiscoverer {
class ServiceDiscoverer {
<<Trait>>
+discover()
}
ManualMonolithDiscoverer --|> MonolithDiscoverer
HarnessMonolithDiscoverer --|> MonolithDiscoverer
FlyMonolithDiscoverer --|> MonolithDiscoverer
DnsMonolithDiscoverer --|> MonolithDiscoverer
ManualServiceDiscoverer --|> ServiceDiscoverer
HarnessServiceDiscoverer --|> ServiceDiscoverer
FlyServiceDiscoverer --|> ServiceDiscoverer
DnsServiceDiscoverer --|> ServiceDiscoverer

DnsDiscoveryConfig --o DnsMonolithDiscoverer
DnsDiscoveryConfig : +u16 monolith_port
DnsDiscoveryConfig --o DnsServiceDiscoverer
DnsDiscoveryConfig : +u16 service_port
DnsDiscoveryConfig : +Option~String~ dns_server
DnsDiscoveryConfig : +String query
DnsDiscoveryConfig : +Option~u64~ polling_duration
class DnsMonolithDiscoverer{
class DnsServiceDiscoverer{
+DnsDiscoveryConfig config
+new(config: DnsDiscoveryConfig)
+discover()
}

FlyDiscoveryConfig --o FlyMonolithDiscoverer
FlyDiscoveryConfig : +u16 monolith_port
FlyDiscoveryConfig --o FlyServiceDiscoverer
FlyDiscoveryConfig : +u16 service_port
FlyDiscoveryConfig : +String fly_app
FlyDiscoveryConfig : +Option~u64~ polling_duration
class FlyMonolithDiscoverer{
class FlyServiceDiscoverer{
+FlyDiscoveryConfig config
+String query
+new(config: FlyDiscoveryConfig)
+discover()
}

ManualDiscoveryConfig --o ManualMonolithDiscoverer
ManualDiscoveryConfig --o ManualServiceDiscoverer
ManualDiscoveryConfig : +Vec monoliths
class ManualMonolithDiscoverer{
class ManualServiceDiscoverer{
+ManualDiscoveryConfig config
+new(config: ManualDiscoveryConfig)
+discover()
}

HarnessDiscoveryConfig --o HarnessMonolithDiscoverer
HarnessDiscoveryConfig --o HarnessServiceDiscoverer
HarnessDiscoveryConfig : +u16 port
class HarnessMonolithDiscoverer{
class HarnessServiceDiscoverer{
+HarnessDiscoveryConfig config
+new(config: HarnessDiscoveryConfig)
+discover()
Expand Down
1 change: 1 addition & 0 deletions figures/discovery/service-discoverers.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions harness-design.typ
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ Auth tokens must be present in all the places they would normally be, but they d

== Emulated Monoliths

Emulated Monoliths are not actual Monoliths, they merely act like them. To that end, each emulated Monolith needs to listen on 2 ports each (one for normal http traffic, and one for the balancer websocket connections). This also means that there needs to be a Monolith discoverer that the harness can use to convey the correct ports to the Balancer. (@Section::HarnessMonolithDiscovery)
Emulated Monoliths are not actual Monoliths, they merely act like them. To that end, each emulated Monolith needs to listen on 2 ports each (one for normal http traffic, and one for the balancer websocket connections). This also means that there needs to be a service discoverer that the harness can use to convey the correct ports to the Balancer. (@Section::HarnessMonolithDiscovery)

== Emulated Clients

Emulated Clients are similar to emulated Monoliths, but they do not need to listen on any ports. They merely need to be able to connect to the Balancer, and send/receive messages. They should behave like a normal client, and they should optionally be able to keep track of the room state they are connected to.

== Conveying Emulated Monoliths to Balancer Discovery <Section::HarnessMonolithDiscovery>
== Conveying Emulated Monoliths to Balancer <Section::HarnessMonolithDiscovery>

When emulated Monoliths are created, they need to convey their existence to the Balancer. To that end, the `TestRunner` should also manage a `DiscoveryProvider` that connects to the Balancer and tells it about the emulated Monoliths that are running. For the harness to connect, the balancer first needs a port to listen on. Assuming the port binds correctly, the discoverer will listen until a harness connects, and then store references to all emulated monoliths given by the discovery provider. To handle connecting to the balancer, the harness uses the `DiscoveryProvider` mentioned earlier. The provider takes in a set of emulated monoliths, and connects to the same port the discoverer is listening on.
2 changes: 1 addition & 1 deletion main.typ
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Document rendered on #datetime.today().display()
#pagebreak()
#include "balancer-design.typ"
#pagebreak()
#include "monolith-discovery.typ"
#include "service-discovery.typ"
#pagebreak()
#include "join-process.typ"
#pagebreak()
Expand Down
34 changes: 0 additions & 34 deletions monolith-discovery.typ

This file was deleted.

34 changes: 34 additions & 0 deletions service-discovery.typ
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
= Service Discovery <Chapter::ServiceDiscovery>

In order to establish connections with Monoliths, the Balancer needs to know the network address of each Monolith. Similarly, the collector needs to know the network addresses of Balancers for visualization. This is done through a process called discovery. The discovery process is responsible for finding the network address of each Monolith and connecting to it. @Figure::discovery-sequence shows the sequence diagram for the discovery process.

There are 2 different discovery modes: Polling and Continuous. Polling is used for `ServiceDiscoverer`s that do not provide real time updates (Like `FlyServiceDiscoverer`), and Continuous is used for `ServiceDiscoverer`s that do provide real time updates (Like `HarnessServiceDiscoverer`). The discovery process is ultimately the same for both modes.

#figure(
image("figures/discovery/discovery-class.svg"),
caption: "Class Diagram for the Service Discovery Process."
) <Figure::discovery-class>

#figure(
image("figures/discovery/discovery-sequence.svg"),
caption: "Sequence Diagram for the Service Discovery Process."
) <Figure::discovery-sequence>

== Implementation

There are three current implementations of the `ServiceDiscoverer` trait.

#figure(
image("figures/discovery/service-discoverers.svg", width: 85%),
caption: "Class Diagram for Service Discoverers."
) <Figure::service-discoverers>

The first implementation is used when connecting to the instance of the monolith on fly.io. Before connecting the user inputs the port that the monolith should listen on for the load balancer connection and a string representing the fly app. These properties are both encapsulated by the `FlyDiscoveryConfig` struct.

To discover the monolith, the configuration and a query are passed into the discoverer struct, `FlyServiceDiscoverer`. In order to connect to the instance on fly.io, an IPv6 DNS lookup is created using the query `global.<fly app name>.internal` #cite(<fly-dns-lookups>). The discovery process uses the results of the lookup and connects the balancer.

The second implementation is used when manually connecting to any instance excluding the one on fly.io and the discovery process works slightly differently. Any number of monolith connections, represented by the `ConnectionConfig` are passed into the manual discoverer. The discoverer then clones the monoliths and connects.

`HarnessServiceDiscoverer` is the third implementation and is used for testing with the harness. The discoverer opens a port and listens for incoming websocket connections. When a connection is made, the discoverer listens for a message from the harness dictating all the monoliths that are visible to the balancer.

The fourth implementation comes in the form of `DnsServiceDiscoverer` when connecting the monolith to the docker dns server. This process follows a similar procedure to that of `FlyServiceDiscoverer` but queries IPv4 addresses instead of IPv6 addresses as Docker does not support IPv6 addresses.
4 changes: 2 additions & 2 deletions visualization-design.typ
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,9 @@ Grafana is a tool primarily meant for time series data, and no current data sour

=== Balancer Discovery

While the visualization is running: Multiple instances of the balancer can be active simultaneously, new instances can become active, and instances can go offline. The addresses of these balancers are not known at runtime, so a discovery process similiar to @Chapter::MonolithDiscovery must run to collect data from the discovered balancers.
While the visualization is running: Multiple instances of the balancer can be active simultaneously, new instances can become active, and instances can go offline. The addresses of these balancers are not known at runtime, so a discovery process similiar to @Chapter::ServiceDiscovery must run to collect data from the discovered balancers.

In order to achieve this, a new rust crate will be created to handle this discovery process. Implementation will likely be similar to @Chapter::MonolithDiscovery. A port will be opened to listen for active instances of the balancer. When a connection or connections are found, the balancer discoverer clones the balancer(s) and connects.
In order to achieve this, a new rust crate will be created to handle this discovery process. Implementation will likely be similar to @Chapter::ServiceDiscovery. A port will be opened to listen for active instances of the balancer. When a connection or connections are found, the balancer discoverer clones the balancer(s) and connects.

=== Querying Balancers

Expand Down
4 changes: 2 additions & 2 deletions weekly-reports.typ
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@

*Figures Updated*
- Added: @Figure::balancer-channels-client-monolith
- Updated: @Figure::monolith-discoverers
- Updated: @Figure::service-discoverers
- Added: @Figure::malformed-websocket-test-sequence
- Added: @Figure::monolith-id-sequence
- Added: @Figure::DOM-class-visualization
Expand Down Expand Up @@ -100,7 +100,7 @@
- Minor update to @Figure::balancer-internals-class
- Added new figure to describe internals of MonolithSelection trait: @Figure::monolith-selection-internals-class
- Ensured all members could run multiple monoliths and load balancers on their systems for manual testing
- Updated @Figure::monolith-discoverers with new implementation for docker
- Updated @Figure::service-discoverers with new implementation for docker
- Added information to explain Grafana and D3.js

*Tasks for next week*
Expand Down

0 comments on commit 80d9a89

Please sign in to comment.