Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CORE-349: request estimated costs from Cromwell for all workflows #3201

Draft
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

davidangb
Copy link
Contributor

Ticket: CORE-349

All workflows now request their estimated cost from Cromwell and display that cost in the UI. Before this PR, we only requested estimated cost for workflows if a per-workflow cost threshold was set for the submission.


PR checklist

  • Include the JIRA issue number in the PR description and title
  • Make sure Swagger is updated if API changes
    • ...and Orchestration's Swagger too!
  • If you changed anything in model/, then you should publish a new official rawls-model and perform the corresponding dependency updates as specified in the README:
    • in the automation subdirectory
    • in workbench-libs
    • in firecloud-orchestration
  • Get two thumbsworth of PR review
  • Verify all tests go green, including CI tests
  • Squash commits and merge to develop (branches are automatically deleted after merging)
  • Inform other teams of any substantial changes via Slack and/or email

Future.successful(WorkflowCostBreakdown(id, workflowCost, "USD", workflowStatus, Seq.empty))
if (abortedMap.keySet.contains(id))
Future(WorkflowCostBreakdown(id, workflowCost, "USD", WorkflowStatuses.Aborted.toString, Seq.empty))
else Future(WorkflowCostBreakdown(id, workflowCost, "USD", workflowStatus, Seq.empty))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the submission monitor now calls getCost() for all workflows instead of calling status() for non-capped workflows and getCost() for capped workflows. Thus, the abort-related logic in this mock's status() needed to be copied into this mock's getCost().

@@ -100,7 +100,7 @@ class SubmissionMonitorSpec(_system: ActorSystem)
workflowsRecs.map { workflowRec =>
scala.util.Success(
Option(
(workflowRec.copy(status = WorkflowStatuses.Succeeded.toString),
(workflowRec.copy(status = WorkflowStatuses.Succeeded.toString, cost = Option(BigDecimal.valueOf(0))),
Some(ExecutionServiceOutputs(workflowRec.externalId.get, Map("o1" -> Left(AttributeString("foo")))))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all workflows now return an estimated cost, so test expectations needed updating

(workflowRec.copy(cost = Option(BigDecimal.valueOf(0))), None)
)
)
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

previously, the monitor would return None because the workflow's status hadn't changed. Now, since the monitor compares both status and cost, and the cost has changed, it returns Some instead of None.

@@ -26,6 +26,7 @@ trait ExecutionServiceCluster extends ErrorReportable {
userInfo: UserInfo
): Future[(ExecutionServiceId, Seq[Either[ExecutionServiceStatus, ExecutionServiceFailure]])]

// currently unused
def status(workflowRec: WorkflowRecord, userInfo: UserInfo): Future[ExecutionServiceStatus]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I chose not to delete the now-unused status() method, but I could be convinced it needs cleaning up

executionServiceCluster.status(workflowRec, petUser).map { newStatus =>
if (newStatus.status != workflowRec.status) Option(workflowRec.copy(status = newStatus.status))
else None
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

collapsing these two cases into one is the core of this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant