[API Proposal]: PinnedBlockMemoryPool metrics

[BrennanConroy]

Background and Motivation

Add metrics to the PinnedBlockMemoryPool that's used by Kestrel, IIS, and Http.Sys.

Proposed API

Microsoft.AspNetCore.MemoryPool

aspnetcore.memorypool.pooled

Name	Instrument Type	Unit	Description
aspnetcore.memorypool.pooled	UpDownCounter	By	Number of bytes currently pooled and available for reuse.

Attribute	Type	Description	Examples	Presence
owner	string	The name of the library or subsystem using the memory pool instance.	kestrel; iis	if owner is specified when the memory pool is created.

aspnetcore.memorypool.allocated

Name	Instrument Type	Unit	Description
aspnetcore.memorypool.allocated	Counter	By	Total number of bytes allocated by the memory pool. Allocation occurs when a memory rental request exceeds the available pooled memory.

Attribute	Type	Description	Examples	Presence
owner	string	The name of the library or subsystem using the memory pool instance.	kestrel; iis	if owner is specified when the memory pool is created.

aspnetcore.memorypool.evicted

Name	Instrument Type	Unit	Description
aspnetcore.memorypool.evicted	Counter	By	Total number of bytes evicted from the memory pool. Eviction occurs when idle pooled memory is reclaimed.

Attribute	Type	Description	Examples	Presence
owner	string	The name of the library or subsystem using the memory pool instance.	kestrel; iis	if owner is specified when the memory pool is created.

aspnetcore.memorypool.rented

Name	Instrument Type	Unit	Description
aspnetcore.memorypool.rented	Counter	By	Total number of rented bytes from the pool.

Attribute	Type	Description	Examples	Presence
owner	string	The name of the library or subsystem using the memory pool instance.	kestrel; iis	if owner is specified when the memory pool is created.

Microsoft.AspNetCore.Connections.Abstractions.dll

namespace Microsoft.AspNetCore.Connections;

public interface IMemoryPoolFactory
{
-    MemoryPool<byte> CreatePool();
+    MemoryPool<byte> CreatePool(MemoryPoolOptions? options = null);
}

// Maybe not sealed to allow custom factory impls to pass through custom options?
+ public sealed class MemoryPoolOptions
+ {
+     public string? Owner { get; set; }
+ }

Usage

public class MyBackgroundService : BackgroundService
{
    private readonly MemoryPool<byte> _memoryPool;

    public MyBackgroundService(IMemoryPoolFactory factory)
    {
        _memoryPool = factory.CreatePool(new MemoryPoolOptions() { Owner = "BackgroundService" });
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                await Task.Delay(20, stoppingToken);
                var rented = _memoryPool.Rent(100);
                rented.Dispose();
            }
            catch (OperationCanceledException)
            {
                return;
            }
        }
    }
}

[JamesNK]

There can be multiple instances of memory pools. They will all write to the same counters so metrics will represent all metrics. For example, Kestrel HTTP layer could have a memory pool, Kestrel sockets layer could have a memory pool, a library could have created their own memory pool (SignalR, gRPC, etc).

There should be a way to view metrics for individual memory pools. The way to do that would be to add tags. A memory pool is given a name and that name is added as a tag when recording the value.

public void IncrementTotalMemory(int bytes)
{
    _totalAllocatedMemory.Add(bytes, new KeyValuePair<string, object>("aspnetcore.memorypool.owner", _owner));
}

This should be fine because the number of memory pools and owners should be low carindality. Although it might be worth calling out in the .NET API to avoid have many names to avoid exceeding metrics dimensions limits.

Name could be specified when creating a memory pool:

public KestrelThing(IMemoryPoolFactory<T> memoryPoolFactory)
{
    _memoryPool = memoryPoolFactory.Create(new MemoryPoolOptions { Owner = "Kestrel" });
}

[BrennanConroy]

_memoryPool = memoryPoolFactory.Create(new MemoryPoolOptions { Owner = "Kestrel" });

That's an interesting idea. Adding options to the Create call. What potential future options would there be? David was wondering about options for eviction delay and various thresholds in #61554 (comment) and #61554 (comment) but those seem specific to our implementation. We'd likely want options to be more generic and apply to "all" pool implementations.

[BrennanConroy]

Updated with Owner tag and added options type to factory.

[JamesNK]

Evicted description:

Total number of bytes evicted from the memory pool. Eviction occurs when idle pooled memory is reclaimed. Evicted memory is available for garbage collection.

[halter73]

API Review Notes:

• Can you calculate the concurrently leased bytes?
  • Yes: allocated - pooled - evicted.
• Do we prefer aspnetcore.connections.memorypool over aspnetcore.memorypool?
  • No. We don't plan on adding new memory pool implementations to ASP.NET Core.

API Approved!