Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It's not possible to consume this action's output if it contains enormous amount of data #458

Open
stanislav-zaprudskiy opened this issue Dec 7, 2021 · 0 comments
Labels
kind/enhancement Improvements or new features

Comments

@stanislav-zaprudskiy
Copy link

In my case I have numbers of CRDs and resources going to be be deployed into Kubernetes (you can take kube-prometheus-stack as example), and preview command with diff: true. As the result, pulumi generates 25k+ of diff output. My workflow then assumes that I post that diff into the corresponding GitHub release body, but prior to doing that I'm pre-formatting that release' body:

    - uses: pulumi/actions@v3
      id: pulumi-preview
      with:
        command: preview
        stack-name: prod
        diff: true

    - name: Build release body
      run: |
        cat << HEREDOC > release-body.md
        ${{ steps.changelog.outputs.changelog }}

        <details>
          <summary>pulumi preview</summary>

          ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}

          \`\`\`
        $(echo "${{ steps.pulumi-preview.outputs.output }}" | base64 -w0 | cut -c -55536| base64 -d | sed -r -e 's/(.*)/  \1/')
          \`\`\`
        </details>
        HEREDOC

    # updating release contents further, by consuming the file generated
    # ...

Here I attempted to cut the output to 55536 characters, so that overall release body doesn't not exceed 65536. However, it doesn't have much effect, as everything seems to be failing even before the task is executed. I receive the following error for "Build release body" step:

Error: The template is not valid. System.InvalidOperationException: Maximum object size exceeded
   at GitHub.DistributedTask.ObjectTemplating.TemplateMemory.AddBytes(Int32 bytes)
   at GitHub.DistributedTask.ObjectTemplating.TemplateMemory.AddBytes(LiteralToken literal)
   at GitHub.DistributedTask.ObjectTemplating.TemplateUnraveler.LiteralState..ctor(ReaderState parent, LiteralToken literal, TemplateContext context, Int32 removeBytes)
   at GitHub.DistributedTask.ObjectTemplating.TemplateUnraveler.ReaderState.CreateState(ReaderState parent, TemplateToken value, TemplateContext context, Int32 removeBytes)
   at GitHub.DistributedTask.ObjectTemplating.TemplateUnraveler.BasicExpressionState.Next(TemplateToken value, Int32 removeBytes)
   at GitHub.DistributedTask.ObjectTemplating.TemplateUnraveler.MappingValueBasicExpression()
   at GitHub.DistributedTask.ObjectTemplating.TemplateUnraveler.Unravel(Boolean expand)
   at GitHub.DistributedTask.ObjectTemplating.TemplateUnraveler.AllowScalar(Boolean expand, ScalarToken& scalar)
   at GitHub.DistributedTask.ObjectTemplating.TemplateEvaluator.Evaluate(DefinitionInfo definition)
   at GitHub.DistributedTask.ObjectTemplating.TemplateEvaluator.HandleMappingWithAllLooseProperties(DefinitionInfo mappingDefinition, DefinitionInfo keyDefinition, DefinitionInfo valueDefinition, MappingToken mapping)
   at GitHub.DistributedTask.ObjectTemplating.TemplateEvaluator.Evaluate(DefinitionInfo definition)
   at GitHub.DistributedTask.ObjectTemplating.TemplateEvaluator.Evaluate(TemplateContext context, String type, TemplateToken template, Int32 removeBytes, Nullable`1 fileId, Boolean omitHeader)

It fails with same error even if I do a simple:

    - name: Build release body
      run: echo "${{ steps.pulumi-preview.outputs.output }}"

or

    - name: Build release body
      env:
        FOO: "${{ steps.pulumi-preview.outputs.output }}"
      run: echo "noop"

like if the task definition could not be rendered, because it contains too much text.

What works, however, is if I do some expression against the output, e.g.:

    - name: Build release body
      run: echo "${{ contains(steps.pulumi-preview.outputs.output, 'Previewing') }}"

but that doesn't help to extract the value by any means.

Just to mention, things work fine with smaller outputs. And in my case it's all on GitHub Enterprise v3.0.

Proposal

What if the action would also optionally write the output to a file? Such file could then be further processed, or just consumed as artifact.

@stanislav-zaprudskiy stanislav-zaprudskiy added the kind/bug Some behavior is incorrect or out of spec label Dec 7, 2021
@leezen leezen added kind/enhancement Improvements or new features and removed kind/bug Some behavior is incorrect or out of spec labels Dec 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement Improvements or new features
Projects
None yet
Development

No branches or pull requests

2 participants