From 846066a34413509695434ed5a661280d7db4f993 Mon Sep 17 00:00:00 2001 From: Caitlin <10053862+con-cat@users.noreply.github.com> Date: Fri, 6 Aug 2021 13:35:38 +1000 Subject: [PATCH] Update docs on Redis Message Priorities The naming of priority queues in Redis doesn't currently work as it's described in the docs - the queues have a separator as well as a priority number appended to them, and the highest priority queue has no suffix. This change updates the docs to reflect this, and adds information on how to configure the separator. Relevant link: https://github.com/celery/kombu/issues/422 --- docs/userguide/routing.rst | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/docs/userguide/routing.rst b/docs/userguide/routing.rst index 300c655a12d..ab1a0d6c2c4 100644 --- a/docs/userguide/routing.rst +++ b/docs/userguide/routing.rst @@ -274,22 +274,34 @@ To start scheduling tasks based on priorities you need to configure queue_order_ The priority support is implemented by creating n lists for each queue. This means that even though there are 10 (0-9) priority levels, these are consolidated into 4 levels by default to save resources. This means that a -queue named celery will really be split into 4 queues: +queue named celery will really be split into 4 queues. + +The highest priority queue will be named celery, and the the other queues will +have a separator (by default `\x06\x16`) and their priority number appended to +the queue name. .. code-block:: python - ['celery0', 'celery3', 'celery6', 'celery9'] + ['celery', 'celery\x06\x163', 'celery\x06\x166', 'celery\x06\x169'] -If you want more priority levels you can set the priority_steps transport option: +If you want more priority levels or a different separator you can set the +priority_steps and sep transport options: .. code-block:: python app.conf.broker_transport_options = { 'priority_steps': list(range(10)), + 'sep': ':', 'queue_order_strategy': 'priority', } +The config above will give you these queue names: + +.. code-block:: python + + ['celery', 'celery:1', 'celery:2', 'celery:3', 'celery:4', 'celery:5', 'celery:6', 'celery:7', 'celery:8', 'celery:9'] + That said, note that this will never be as good as priorities implemented at the server level, and may be approximate at best. But it may still be good enough