Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reorg cleanup query timeout #272

Open
ryandotsmith opened this issue Sep 3, 2024 · 0 comments
Open

reorg cleanup query timeout #272

ryandotsmith opened this issue Sep 3, 2024 · 0 comments

Comments

@ryandotsmith
Copy link
Member

It is possible that the query used to delete data during a reorg is too slow for the default 10s database timeout. During a reorg Shovel will delete data from shovel.task_updates with this query and all the integration's tables with this query.

It is possible that one of these two queries is taking longer than 10s. Since Shovel automatically creates an index on shovel.task_updates, I'm wondering if we are missing an index on the integration tables.

The ideal fix is to make these queries fast. Perhaps with a better index.

If you are experiencing this problem please use the postgres logs to show the exact query that is timing out. Also please share the table DDL so we can look for missing indexes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant