sqs: Optimize for concurrency

[antoineco]

Reopens #225, which was accidentally merged (I reverted it).

Closes #222

[antoineco]

Performance on 2 cores with 2 receiver|processor|deleter per thread. No CPU throttling was observed with the current CPU limit of 1.

[antoineco]

Here is the result of another load test with dynamic concurrency settings (+1 receiver|processor|deleter every 10s), virtually unlimited buffers (size > total messages in the SQS queue), and TriggerMesh's default CPU limit (500m):

Things become relatively unstable beyond 6 receiver|processor|deleter, which is 3 per thread.

The default limit of 500m is causing a fair amount of CPU throttling:

[antoineco]

The same load test as above, but this time without CPU limit. This time the performance remains quite stable until 12 receiver|processor|deleter (6 per thread) but figures did not double compared to the previous experiment, because CloudEvents are still sent sequentially after messages are received from the queue.

Like what I observed before, the CPU usage remains below 400m:

[antoineco]

@sebgoa the decision is now about defining the default per Pod.

Do we want to

1. Assume maximum performance per Pod is always a goal and raise the default request+limit to achieve that at 6 messages processors per thread? (-> 450 msg/s)

   Trade-off: we won't be able to schedule as many replicas per node, and it might be an overkill for more moderate traffic (500 msg/s feels like a lot to me)

2. Take a more moderate stance and stick to this PR's values with 2 messages processors per thread? (-> 150 msg/s)

   Trade-off: scaling to higher rates requires horizontal auto-scaling, which we have to enable in a separate PR (this statement can apply to all cases, actually).

3. Meet half-way, e.g. raise the default to 3 messages processors per thread? (-> 250 msg/s)

   Trade-off: we won't be able to schedule as many replicas per node as in 2., but still a bit more than in 1.

I would personally vote for 3 but follow-up with auto-scaling (#230), and maybe also #227 to spawn/terminate goroutines dynamically based on the number of messages being received.