Structure serialization

[LeadcodeDev]

Discussed in #225

^{Originally posted by LeadcodeDev December 25, 2024}
Hello,

During the two years of development on this new version of Mineral's core, we've experimented with several approaches to managing our data from Discord's Websocket and API.

Introduction

Today, we deliver all data accessible from a top-level data structure.
This involves retrieving additional resources when the initial structure is supposed to be able to access them; either via the cache if it has data, or via a request to the Discord API.

One example is more telling :

// Top-level structure
final member = Member();

// Second-level structure accessible by chaining from the member
final server = member.server;

// Third level structure
final channels = member.server.channels;

Problems

The problem with the way things currently work is that the Discord API does not systematically provide us with all the resources we need to model all the class depths; we have to make additional requests in the background, which are therefore imposed on you by the framework.

So even if the final developer doesn't need certain data, it's still there.

The problem as it stands is that we have to retrieve a multitude of resources for a single ‘small’ data structure.
This implies and imposes this obligation on the end developer.

Proposal

We want to experiment with a more ‘atomic’ approach aimed at reducing complexity and our dependence on multiple resources as much as possible. To this end, we propose a slightly different approach based on the ‘resource on demand’ principle.

An example is (once again) more telling :

// Top-level structure
final member = Member();

// New first-level structure
final server = await member.getServer();

// (Again) a new top-level structure
final channels = await server.channels.fetch();

The fact of forcing the end developer to request a resource explicitly implies more code to write but drastically reduces the executive delay of the relative code because the data structure to be shared with the "view" (consumer of events...) is radically minimised.

This simplifies and further reduces dependency on a cache (redis, memory, etc.) and is therefore in line with our philosophy.

With this in mind, the fetch method will query the cache first and then Discord's HTTP API to retrieve the missing data.

Examples

In the context of a server resource, a request to the Discord API provides us with neither members nor channels.
So each point that is not "pre-loaded" could be the subject of a manager which would act as a node.

client.events.server.serverCreate((Server server) async {
  final Map<Snowflake, ServerChannel> channels = await server.channels.fetch();
});

In this example, channels would be a node allowing recovery actions to be performed according to the needs of the end developer (fetch, resolve...).

Hypothetically we can apply this principle to users :

client.events.server.serverCreate((Server server) async {
  final Map<Snowflake, Member> members = await server.members.fetch();
});

What do you think ?
Do you have any ideas for improvements or alternatives ?