Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make concurrent session requests more robust #902

Closed
4 tasks done
nphilipp opened this issue Aug 17, 2023 · 0 comments · Fixed by #905
Closed
4 tasks done

Make concurrent session requests more robust #902

nphilipp opened this issue Aug 17, 2023 · 0 comments · Fixed by #905
Assignees
@nphilipp
Labels
API bug database concerning database models, migration, … maintenance

Comments

@nphilipp
Copy link
Collaborator

nphilipp commented Aug 17, 2023 •
edited
Loading

Stories

As a user of Duffy,
I want that requesting sessions/nodes is successful,
so I can conduct my tests.

As an operator of Duffy,
I want that processing nodes already assigned to a session (“contextualizing”) works reliably,
so that there no unused left over nodes which just cut into our cloud budget.

Acceptance Criteria

When a user requests a session…:

  • Operations in the database which can potentially conflict are attempted several times in the case of conflicts, if they still fail, an appropriate HTTP status is returned.
  • Operations which can’t possibly conflict (say, resources are exclusive to a query) aren’t made more complicated with retrial mechanisms.
  • Other encountered issues are fixed.
  • Comprehensive concurrency tests exist and reliably succeed even with multiple repetitions of the test.
@nphilipp nphilipp added bug database concerning database models, migration, … API maintenance labels Aug 17, 2023
@nphilipp nphilipp self-assigned this Aug 17, 2023
nphilipp added a commit to nphilipp/duffy that referenced this issue Aug 24, 2023
@nphilipp
Revamp creating sessions
214082e 
- Return HTTP code 422 (unprocessable entity) if nodes are ultimately
  failed to be allocated for the new session.
- Don’t retry subsequent database operations, as all involved objects
  (database rows) should now be exclusive to the current query. This
  makes the code much more readable. Instead, return appropriate HTTP
  codes on failures.
- After the node allocation transaction is committed, consider all
  objects as stale to the session and reload when needed.

Fixes: CentOS#902

Signed-off-by: Nils Philippsen <[email protected]>
@nphilipp nphilipp mentioned this issue Aug 24, 2023
Merged
nphilipp added a commit to nphilipp/duffy that referenced this issue Aug 24, 2023
@nphilipp
Revamp creating sessions
d65d068 
- Return HTTP code 422 (unprocessable entity) if nodes are ultimately
  failed to be allocated for the new session.
- Don’t retry subsequent database operations, as all involved objects
  (database rows) should now be exclusive to the current query. This
  makes the code much more readable. Instead, return appropriate HTTP
  codes on failures.
- After the node allocation transaction is committed, consider all
  objects as stale to the session and reload when needed.

Fixes: CentOS#902

Signed-off-by: Nils Philippsen <[email protected]>
nphilipp added a commit to nphilipp/duffy that referenced this issue Sep 6, 2023
@nphilipp
Revamp creating sessions
233e7e2 
- Return HTTP code 422 (unprocessable entity) if nodes are ultimately
  failed to be allocated for the new session.
- Use (only) two separate transactions which get retried on DB
  conflicts. It is necessary to retry even the second transaction: even
  though the nodes used there are exclusive (allocated to the session in
  question), the query used to allocate nodes elsewhere can get their
  rows read, which can lead to conflicts.
- Separate these transactions better by reloading all necessary objects
  from the database.
- Return appropriate HTTP codes on failures.
- Really unallocate nodes/delete the session on contextualization failures.

Fixes: CentOS#902

Signed-off-by: Nils Philippsen <[email protected]>
nphilipp added a commit to nphilipp/duffy that referenced this issue Sep 6, 2023
@nphilipp
DB: Reduce risk of read/write conflicts
ae8a4ed 
This adds indexes to a couple often-queried columns and advises the
database to prefer index-based lookups instead of sequential scans.

Related: CentOS#902

Signed-off-by: Nils Philippsen <[email protected]>
@nphilipp nphilipp linked a pull request Sep 6, 2023 that will close this issue
Revamp creating sessions and related #905
Merged
nphilipp added a commit to nphilipp/duffy that referenced this issue Sep 6, 2023
@nphilipp
Revamp creating sessions
4ff636f 
- Return HTTP code 422 (unprocessable entity) if nodes are ultimately
  failed to be allocated for the new session.
- Use (only) two separate transactions which get retried on DB
  conflicts. It is necessary to retry even the second transaction: even
  though the nodes used there are exclusive (allocated to the session in
  question), the query used to allocate nodes elsewhere can get their
  rows read, which can lead to conflicts.
- Separate these transactions better by reloading all necessary objects
  from the database.
- Return appropriate HTTP codes on failures.
- Really unallocate nodes/delete the session on contextualization failures.

Fixes: CentOS#902

Signed-off-by: Nils Philippsen <[email protected]>
nphilipp added a commit to nphilipp/duffy that referenced this issue Sep 6, 2023
@nphilipp
DB: Reduce risk of read/write conflicts
3e9e43a 
This adds indexes to a couple often-queried columns and advises the
database to prefer index-based lookups instead of sequential scans.

Related: CentOS#902

Signed-off-by: Nils Philippsen <[email protected]>
nphilipp added a commit that referenced this issue Sep 6, 2023
@nphilipp
Revamp creating sessions
48c11ec 
- Return HTTP code 422 (unprocessable entity) if nodes are ultimately
  failed to be allocated for the new session.
- Use (only) two separate transactions which get retried on DB
  conflicts. It is necessary to retry even the second transaction: even
  though the nodes used there are exclusive (allocated to the session in
  question), the query used to allocate nodes elsewhere can get their
  rows read, which can lead to conflicts.
- Separate these transactions better by reloading all necessary objects
  from the database.
- Return appropriate HTTP codes on failures.
- Really unallocate nodes/delete the session on contextualization failures.

Fixes: #902

Signed-off-by: Nils Philippsen <[email protected]>
nphilipp added a commit that referenced this issue Sep 6, 2023
@nphilipp
DB: Reduce risk of read/write conflicts
9f8304d 
This adds indexes to a couple often-queried columns and advises the
database to prefer index-based lookups instead of sequential scans.

Related: #902

Signed-off-by: Nils Philippsen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nphilipp nphilipp

Labels
API bug database concerning database models, migration, … maintenance
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Revamp creating sessions and related nphilipp/duffy
1 participant
@nphilipp