Feature/consumable array buffer writer pooling

[YairHalberstadt]

Initial implementation of pooling for ConsumableArrayBufferWriter.

Before:

Method	MessageSize	NumIterations	Mean	Error	StdDev	Ratio	RatioSD	Gen 0	Gen 1	Gen 2	Allocated
BASELINE: PipeReaderConsumeAllEachTime	4096	100	80.66 us	1.894 us	2.776 us	1.00	0.00	-	-	-	1 B
MessagePipeReaderConsumeAllEachTime	4096	100	117.55 us	1.714 us	1.603 us	1.47	0.05	-	-	-	425 B
MessagePipeReaderConsumeNoneEachTime	4096	100	509.39 us	10.146 us	13.193 us	6.30	0.27	285.1563	285.1563	285.1563	1048834 B

After:

Method	MessageSize	NumIterations	Mean	Error	StdDev	Ratio	RatioSD	Gen 0	Gen 1	Gen 2	Allocated
PipeReaderConsumeAllEachTime	4096	100	80.60 us	1.576 us	2.406 us	1.00	0.00	-	-	-	1 B
MessagePipeReaderConsumeAllEachTime	4096	100	118.24 us	2.357 us	2.205 us	1.47	0.05	-	-	-	145 B
MessagePipeReaderConsumeNoneEachTime	4096	100	162.39 us	2.330 us	2.180 us	2.01	0.06	-	-	-	415 B

Upper bound on benefits of pooling without any synchronization or resizing:

Method	MessageSize	NumIterations	Mean	Error	StdDev	Ratio	RatioSD	Gen 0	Gen 1	Gen 2	Allocated
MessagePipeReaderConsumeNoneEachTime	4096	100	148.69 us	2.215 us	2.072 us	1.82	0.05	-	-	-	113 B

One issue with this approach is that it requires users to call Complete once they are done parsing. I don't know if this is considered part of the contract.

If not the _backlog will not be returned to the ArrayPool. One consequence of this is we might pool it long enough for it to go on gen2 heap, but then drop it, which could lead to more GC pressure. See this comment by Stephen Toub.

[davidfowl]

@YairHalberstadt Can you squash all commits and base this on master? It seems to be based on your previous branch.

[davidfowl]

One issue with this approach is that it requires users to call Complete once they are done parsing. I don't know if this is considered part of the contract.

It is.

[davidfowl]

@YairHalberstadt also, change the ConsumableArrayBufferWriter to be byte specific.

[davidfowl]

Why did this test change so much?

[YairHalberstadt]

Since we're using the ArrayPool there's no guarantee the buffer will be clean, so I removed this check (The for loops).
For the same reason there's no guarantee that the new buffer wont happen to contain the same bytes as the old buffer. In fact it's quite likely due to the way these tests are set up, and happened by chance in one of my runs.

Assumptions about the size and capacity of the Buffer are now much weaker, so I had to change/remove them.

[davidfowl]

I'd like to see Consume return the buffer to the pool if everything has been consumed (or at least return to the original capacity ~ 256)

[mattnischan]

Just a thought and probably due to my own ignorance, but would it be wiser to use MemoryPool here instead? I realize that MemoryPool just effectively passes through to ArrayPool at this point in time, but there has been discussion on optimizing it specifically and it does seem semantically more appropriate.

[davidfowl]

Just a thought and probably due to my own ignorance, but would it be wiser to use MemoryPool here instead? I realize that MemoryPool just effectively passes through to ArrayPool at this point in time, but there has been discussion on optimizing it specifically and it does seem semantically more appropriate.

Nope. It allocates an IMemoryOwner every time you rent, we optimized this in pipelines in 3.0 to directly use the array pool directly by default.

[mattnischan]

Ah yeah, that does seem worse, in that case.

[davidfowl]

I think we want to return the buffer on consumed regardless of the size no?

[YairHalberstadt]

If it's the Default size why return it? The moment anyone wrote anything we would ask for it back, and if they've finished with it they should dispose...