Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In-memory order updater #5872

Draft
wants to merge 32 commits into
base: main
Choose a base branch
from

Conversation

benjaminwil
Copy link
Contributor

@benjaminwil benjaminwil commented Oct 11, 2024

Summary

This pull request proposes a replacement to the default Spree::OrderUpdater that has new and improved functionality:

  • Increased performance due to the updater making fewer write calls to the database.
  • A built-in update simulator, so that changes to an order can be previewed before persisting them.

There may be some beneficial side-effects that come out of this new order updater implementation:

  • Significantly faster order factories, and thus a significantly faster test suite for Solidus gems and Solidus applications.

We don't expect this to be the default order updater implementation in the next minor version of Solidus, but we would like to propose it as the default for the next major version of Solidus.

Note: The commits on this pull request have a long list of co-authors, as the Super Good team is approaching this as a collaborative mob programming exercise.

Milestones

For this order updater, we intend to achieve the following during updates:

  1. Don't perform writes to the database.
  2. Preload associated records to eliminate reads required.

We appreciate that there is a lot of complexity to achieving these goals (dealing with active promotions, for example).

Notes

  • We should provide more context about performance gains in this PR description. It would be insightful to include the actual number of writes the current Spree::OrderUpdater makes on a typical recalculate.
  • We should further explain why a production store may want to use an order update simulator so it's clear why this feature is worth having. For now, I'll just say that we have worked with stores who could benefit from this feature. Sometimes admins must make significant changes to completed orders, and it would be valuable for them to preview a set of changes before submitting them and causing updates on many order-associated tables.

Checklist

Check out our PR guidelines for more details.

The following are mandatory for all PRs:

The following are not always needed:

  • 📖 I have updated the README to account for my changes.
  • 📑 I have documented new code with YARD.
  • 🛣️ I have opened a PR to update the guides.
  • ✅ I have added automated tests to cover my changes.
  • 📸 I have attached screenshots to demo visual changes.

@github-actions github-actions bot added the changelog:solidus_core Changes to the solidus_core gem label Oct 11, 2024
@benjaminwil benjaminwil changed the title In memory order updater In-memory order updater Oct 11, 2024
@benjaminwil benjaminwil force-pushed the in-memory-order-updater branch 2 times, most recently from 27f19a8 to 340032c Compare October 11, 2024 22:19
@forkata forkata force-pushed the in-memory-order-updater branch from d22210b to d244646 Compare November 8, 2024 22:34
@jarednorman
Copy link
Member

Just making a note that we are waiting on Alistair to rebase #6026 against this.

@AlistairNorman could you just make sure that gets done at some point before the 20th? That way we can have it for our next session.

@AlistairNorman AlistairNorman force-pushed the in-memory-order-updater branch from a3bb1fc to 06e3a2a Compare December 19, 2024 18:27
@stewart stewart force-pushed the in-memory-order-updater branch from 9fd5c8d to 27e4988 Compare December 20, 2024 21:30
benjaminwil and others added 22 commits January 17, 2025 14:01
https://github.com/sds/db-query-matchers

Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Andrew Stewart <[email protected]>
Co-authored-by: Harmony Evangelina <[email protected]>
Co-authored-by: Jared Norman <[email protected]>
Co-authored-by: Nick Van Doorn <[email protected]>
Co-authored-by: Noah Silvera <[email protected]>
Co-authored-by: Senem Soy <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
Co-authored-by: Tom Van Manen <[email protected]>
In subsequent commits we'll ensure that this can update orders in
memory, without persisting changes using manipulative DB queries.

Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Andrew Stewart <[email protected]>
Co-authored-by: Harmony Evangelina <[email protected]>
Co-authored-by: Jared Norman <[email protected]>
Co-authored-by: Nick Van Doorn <[email protected]>
Co-authored-by: Noah Silvera <[email protected]>
Co-authored-by: Senem Soy <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
Co-authored-by: Tom Van Manen <[email protected]>
We want our new in-memory order updater to be able to persist or not
persist changes to the order record.

WORK IN PROGRESS

This is a first step in ensuring we don't need to write to the database
using the order updater. Clearly we have more work to do to ensure this
functions like the existing updater.

Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Andrew Stewart <[email protected]>
Co-authored-by: Harmony Evangelina <[email protected]>
Co-authored-by: Jared Norman <[email protected]>
Co-authored-by: Nick Van Doorn <[email protected]>
Co-authored-by: Noah Silvera <[email protected]>
Co-authored-by: Senem Soy <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
Co-authored-by: Tom Van Manen <[email protected]>
This is in service of supporting the InMemoryOrderUpdater's goal to not do database writes.
…atabase writes

We have prevented write calls to update the cost and `updated_at` of a
shipment, as well as allowed us to conditionally persist item totals, by
passing down the `persist` argument to that method.

Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Andrew Stewart <[email protected]>
Co-authored-by: Harmony Evangelina <[email protected]>
Co-authored-by: Kendra Riga <[email protected]>
Co-authored-by: Jared Norman <[email protected]>
Co-authored-by: Tom Van Manen <[email protected]>
Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
Co-authored-by: Senem Soy <[email protected]>
Co-authored-by: Benjamin Willems <[email protected]>
Update implies that we are persisting the change in Rails, which this
method does not do.

Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Senem Soy <[email protected]>
Co-authored-by: Andrew Stewart <[email protected]>
Co-authored-by: Kendra Riga <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
Co-authored-by: Benjamin Willems <[email protected]>
Update implies that we are persisting the change in Rails, which this
method does not do.

Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Andrew Stewart <[email protected]>
Co-authored-by: Benjamin Willems <[email protected]>
Co-authored-by: Senem Soy <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
Co-authored-by: Kendra Riga <[email protected]>
These methods don't persist so it's more accurate to say that they
recalculate the total instead of saying that they update it.

Co-Authored-By: Adam Mueller <[email protected]>
Co-Authored-By: Benjamin Willems <[email protected]>
Co-Authored-By: Andrew Stewart <[email protected]>
Co-Authored-By: Harmony Bouvier <[email protected]>
Co-Authored-By: Jared Norman <[email protected]>
Co-Authored-By: Kendra Riga <[email protected]>
Co-Authored-By: Sofia Besenski <[email protected]>
Co-Authored-By: Chris Todorov <[email protected]>
Co-Authored-By: Tom Van Manen <[email protected]>
Co-Authored-By: Noah Silvera <[email protected]>
We want all the methods that might persist data to be called update_
instead of recalculate to be clear that they hit the database.

Co-Authored-By: Adam Mueller <[email protected]>
Co-Authored-By: Benjamin Willems <[email protected]>
Co-Authored-By: Andrew Stewart <[email protected]>
Co-Authored-By: Harmony Bouvier <[email protected]>
Co-Authored-By: Jared Norman <[email protected]>
Co-Authored-By: Kendra Riga <[email protected]>
Co-Authored-By: Sofia Besenski <[email protected]>
Co-Authored-By: Chris Todorov <[email protected]>
Co-Authored-By: Tom Van Manen <[email protected]>
Co-Authored-By: Noah Silvera <[email protected]>
This is calling the recalculate method not update_adjustments.

Co-Authored-By: Adam Mueller <[email protected]>
Co-Authored-By: Benjamin Willems <[email protected]>
Co-Authored-By: Andrew Stewart <[email protected]>
Co-Authored-By: Harmony Bouvier <[email protected]>
Co-Authored-By: Jared Norman <[email protected]>
Co-Authored-By: Kendra Riga <[email protected]>
Co-Authored-By: Sofia Besenski <[email protected]>
Co-Authored-By: Chris Todorov <[email protected]>
Co-Authored-By: Tom Van Manen <[email protected]>
Co-Authored-By: Noah Silvera <[email protected]>
This puts all the update and recalculate methods together.

Co-Authored-By: Adam Mueller <[email protected]>
Co-Authored-By: Benjamin Willems <[email protected]>
Co-Authored-By: Andrew Stewart <[email protected]>
Co-Authored-By: Harmony Bouvier <[email protected]>
Co-Authored-By: Jared Norman <[email protected]>
Co-Authored-By: Kendra Riga <[email protected]>
Co-Authored-By: Sofia Besenski <[email protected]>
Co-Authored-By: Chris Todorov <[email protected]>
Co-Authored-By: Tom Van Manen <[email protected]>
Co-Authored-By: Noah Silvera <[email protected]>
We want to start breaking out some of the complex logic of the in memory
updater into smaller more focused classes.

Co-Authored-By: Adam Mueller <[email protected]>
Co-Authored-By: Benjamin Willems <[email protected]>
Co-Authored-By: Andrew Stewart <[email protected]>
Co-Authored-By: Harmony Bouvier <[email protected]>
Co-Authored-By: Jared Norman <[email protected]>
Co-Authored-By: Kendra Riga <[email protected]>
Co-Authored-By: Sofia Besenski <[email protected]>
Co-Authored-By: Chris Todorov <[email protected]>
Co-Authored-By: Tom Van Manen <[email protected]>
Co-Authored-By: Noah Silvera <[email protected]>
This is just a stub for now, but we want to eventually introduce a class
to handle running the promotion adjustments in memory.

Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Andrew Stewart <[email protected]>
Co-authored-by: Harmony Evangelina <[email protected]>
Co-authored-by: Kendra Riga <[email protected]>
Co-authored-by: Jared Norman <[email protected]>
Co-authored-by: Tom Van Manen <[email protected]>
Co-authored-by: Senem Soy <[email protected]>
Co-authored-by: Benjamin Willems <[email protected]>
Co-Authored-By: Jared Norman <[email protected]>
Co-Authored-By: benjamin wil <[email protected]>
Co-Authored-By: Kendra Riga <[email protected]>
Co-Authored-By: Sofia Besenski <[email protected]>
Co-Authored-By: Harmony Evangelina <[email protected]>
Co-Authored-By: Jared Norman <[email protected]>
Co-Authored-By: Kendra Riga <[email protected]>
Co-Authored-By: Sofia Besenski <[email protected]>
Co-Authored-By: benjamin wil <[email protected]>
This change more clearly splits up the logic for recalculating shipment/item
adjustment totals from the logic for persisting updated values to the database.

Continuing to colocating logic for recalculating item totals in one PORO makes
it easier to reason about and helps to simplify the higher-level order updater.

Co-Authored-By: Harmony Evangelina <[email protected]>
Co-Authored-By: Jared Norman <[email protected]>
Co-Authored-By: Kendra Riga <[email protected]>
Co-Authored-By: Sofia Besenski <[email protected]>
Co-Authored-By: benjamin wil <[email protected]>
Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Andrew Stewart <[email protected]>
Co-authored-by: Jared Norman <[email protected]>
Co-authored-by: Noah Silvera <[email protected]>
Co-authored-by: Benjamin Willems <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
To support the in-memory order updater, we want adjustments to be
modified without writing to the database.
Adding autosave: true in the association on line_items ensures that
the adjustment amount will be saved with the line_item, or by
extension the order.
The tests for the OrderTaxation directly have been updated to use
`size` instead of `count` to ensure an in-memory reported count/size.

Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Andrew Stewart <[email protected]>
Co-authored-by: Jared Norman <[email protected]>
Co-authored-by: Noah Silvera <[email protected]>
Co-authored-by: Benjamin Willems <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
harmonymjb and others added 9 commits January 17, 2025 14:03
Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Andrew Stewart <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: Benjamin Willems <[email protected]>
Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: benjamin wil <[email protected]>
Co-authored-by: Harmony Bouvier <[email protected]>
Co-authored-by: Kendra Riga <[email protected]>
Co-authored-by: Senem <[email protected]>
Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: benjamin wil <[email protected]>
Co-authored-by: Harmony Bouvier <[email protected]>
Co-authored-by: Kendra Riga <[email protected]>
Co-authored-by: Senem <[email protected]>
Instead of actually destroying adjustments in the OrderTaxation class, we can just mark them for removal. This will allow them to be removed when the order is persisted, instead of being removed during the recalculate process.

This does have the side effect of needing to check for tax adjustments that are marked for destruction when calculating tax totals elsewhere in the OrderUpdaters

Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: benjamin wil <[email protected]>
Co-authored-by: Harmony Bouvier <[email protected]>
Co-authored-by: Kendra Riga <[email protected]>
Co-authored-by: Senem <[email protected]>
Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: benjamin wil <[email protected]>
Co-authored-by: Harmony Bouvier <[email protected]>
Co-authored-by: Kendra Riga <[email protected]>
Co-authored-by: Senem <[email protected]>
Ensure that when setting line item totals, we only consider the
adjustments that haven't been marked for deletion. We still need to make
the equivalent change to the regular OrderUpdater class.

Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: benjamin wil <[email protected]>
Co-authored-by: Harmony Bouvier <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
Ensure that when setting line item totals, we only consider the
adjustments that haven't been marked for deletion.

Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: benjamin wil <[email protected]>
Co-authored-by: Harmony Bouvier <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: benjamin wil <[email protected]>
Co-authored-by: Harmony Bouvier <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
This updates Spree::OrderUpdater to use the new ItemTotal class,
avoiding duplicating all of that logic.

Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: benjamin wil <[email protected]>
Co-authored-by: Harmony Bouvier <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
@jarednorman jarednorman force-pushed the in-memory-order-updater branch from 4a0f623 to 90f5420 Compare January 17, 2025 22:03
jarednorman added a commit to SuperGoodSoft/solidus that referenced this pull request Jan 17, 2025
While working on the in-memory updater in solidusio#5872, we found the need to
change how item totals were being calculated, so that we could mark
adjustments for destruction without actually destroying them, while
still keeping tax adjustments intact. This change is completely
backwards-compatible with the current OrderUpdater, so to reduce the
scope of our PR, we wanted to make this change separately.

Since the OrderUpdater is already very large, this helps reduce its
responsibilities and makes it easier to test this behaviour. We don't
see it as necessary to make this a configurable class, but this change
leaves that option open in the future.

Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Harmony Bouvier <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
Co-authored-by: benjamin wil <[email protected]>
jarednorman added a commit to SuperGoodSoft/solidus that referenced this pull request Jan 17, 2025
While working on the in-memory updater in solidusio#5872, we found the need to
change how item totals were being calculated, so that we could mark
adjustments for destruction without actually destroying them, while
still keeping tax adjustments intact. This change is completely
backwards-compatible with the current OrderUpdater, so to reduce the
scope of our PR, we wanted to make this change separately.

Since the OrderUpdater is already very large, this helps reduce its
responsibilities and makes it easier to test this behaviour. We don't
see it as necessary to make this a configurable class, but this change
leaves that option open in the future.

Co-authored-by: Adam Mueller <[email protected]>
Co-authored-by: Alistair Norman <[email protected]>
Co-authored-by: Chris Todorov <[email protected]>
Co-authored-by: Harmony Bouvier <[email protected]>
Co-authored-by: Sofia Besenski <[email protected]>
Co-authored-by: benjamin wil <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog:solidus_core Changes to the solidus_core gem
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants