Beacon sync recalibrate blocks queue #3099
Conversation
why: Was fall back for the case that the DB table was inaccessible before `FC` module reorg.
why: Provides useful settings, e.g. for memory debugging
why: Old queue setup provided a staging area which was much too large consuming too much idle memory. Also the command-line re-calibrating for debugging was much too complicated. And the naming for the old setup was wrong: There is no max queue size. Rather there is a HWM where filling the queue stops when reached. The currently tested size allows for 1.5k blocks on the queue.
| @@ -147,8 +147,8 @@ func blocksStagedCanImportOk*(ctx: BeaconCtxRef): bool = | |||
| if ctx.pool.nBuddies == 0: | |||
| return true | |||
|
|
|||
| # Start importing if the queue is long enough. | |||
| if ctx.pool.blocksStagedQuLenMax <= ctx.blk.staged.len: | |||
| # Start importing if the queue is filled up enough | |||
There was a problem hiding this comment.
why wait here? 1 block should be enough to start importing, no?
There was a problem hiding this comment.
When importing while peers are still downloading you will most certainly run into timeouts so loosing the peers.
There was a problem hiding this comment.
how is this different when you have more than N blocks in the queue already? ie is it ok to download and process blocks in parallel when there are many blocks resting in the queue but not when there are few?
There was a problem hiding this comment.
also, when did you last test this? there were many bugs in the peer management itself as well as performance issues that have been fixed.
There was a problem hiding this comment.
It proved not ok downloading and importing at the same time. The chance to loose peers is too high, so grabbing everything while they are there seems to be the wise approach.
There was a problem hiding this comment.
I can give it a try again to do it simultaneously, importing and downloading. We'll see (as you see these are mere flags to be set.)
No description provided.