Introduce Atmos YAML functions !include and !env
#943
Conversation
# Conflicts: # go.mod
📝 WalkthroughWalkthroughThis pull request introduces several updates to the Atmos project, primarily focusing on enhancing YAML function capabilities, improving file handling utilities, and updating dependency versions. Key changes include an increment in the Atmos tool version from 1.149.0 to 1.153.0, along with the addition of new functions for environment variable retrieval and file inclusion. The updates also involve modifications to multiple internal execution files and enhancements to the documentation for new YAML functions. Changes
Possibly related PRs
Suggested ReviewersThank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
♻️ Duplicate comments (2)
internal/exec/yaml_func_include.go (1)
27-43:⚠️ Potential issueLet's handle filenames with spaces when splitting arguments.
Splitting
strby spaces may not properly handle filenames that contain spaces. This issue has been noted before. Let's update the parsing logic to support filenames with spaces, possibly by recognizing quoted strings.Apply this diff to improve argument parsing:
- parts, err := u.SplitStringByDelimiter(str, ' ') + parts, err := u.ParseArguments(str)Where
ParseArgumentsis a function that correctly parses arguments, respecting quotes.pkg/utils/yaml_utils.go (1)
128-143:⚠️ Potential issueLet's handle filenames with spaces when splitting arguments in
getValueWithTag.Splitting
valby spaces may not properly handle filenames that contain spaces. This issue has been noted before. Let's update the parsing logic to support filenames with spaces, possibly by recognizing quoted strings.
🧹 Nitpick comments (18)
internal/exec/yaml_func_include.go (1)
15-15: Consider removing the unused parametercurrentStack.The parameter
currentStackis not used withinprocessTagInclude. Removing it could improve code clarity.pkg/utils/yaml_utils.go (1)
112-118: Consider sanitizing file paths and arguments.When constructing strings with
fandq, let's ensure they are properly sanitized or escaped to prevent potential injection or parsing issues, especially if they can contain special characters.Also applies to: 173-177
internal/exec/go_getter_utils.go (1)
21-54: Consider usingurl.Parsefor robust URI validation.The
ValidateURIfunction manually parses URIs using string operations. Let's consider usingurl.Parseto properly parse and validate URIs, improving robustness and maintainability.pkg/utils/string_utils.go (1)
23-34: Addition ofSplitStringByDelimiterenhances utility functions.The new
SplitStringByDelimiterfunction provides a robust way to split strings by a delimiter while respecting quoted substrings. Consider adding unit tests to validate its behavior with various input scenarios.internal/exec/yaml_func_env.go (1)
13-17: Add function documentationThe function lacks documentation explaining its purpose, parameters, and return value.
+// processTagEnv processes the !env YAML tag which retrieves values from environment variables. +// It supports both simple types (returned as strings) and complex types (JSON-encoded strings +// that are automatically decoded). +// +// Parameters: +// - atmosConfig: The Atmos configuration +// - input: The raw tag input string (e.g. "!env MY_VAR") +// - currentStack: The name of the current stack being processed +// +// Returns: +// - For simple values: the string value of the environment variable +// - For JSON-encoded values: the decoded value as appropriate Go type func processTagEnv( atmosConfig schema.AtmosConfiguration, input string, currentStack string, ) any {pkg/utils/yq_utils.go (1)
34-34: Strong work on the pointer optimization, warrior!The change to use a pointer receiver for
atmosConfigis a worthy optimization that prevents unnecessary copying of the configuration object.Consider adding a nil check at the start of the function to make it more defensive:
func EvaluateYqExpression(atmosConfig *schema.AtmosConfiguration, data any, yq string) (any, error) { + if atmosConfig == nil { + return nil, fmt.Errorf("EvaluateYqExpression: atmosConfig cannot be nil") + }pkg/utils/json_utils.go (1)
137-145: Consider adding debug logging for validation failures.The implementation is solid, but adding debug-level logging when JSON validation fails could help with troubleshooting configuration issues.
func IsJSON(data string) bool { if strings.TrimSpace(data) == "" { return false } var js json.RawMessage - return json.Unmarshal([]byte(data), &js) == nil + err := json.Unmarshal([]byte(data), &js) + if err != nil { + LogDebug(atmosConfig, "JSON validation failed: %v", err) + return false + } + return true }pkg/utils/hcl_utils.go (1)
145-153: Maintain consistency with JSON validation logging.The implementation is good, but should follow the same error logging pattern suggested for the
IsJSONfunction.func IsHCL(data string) bool { if strings.TrimSpace(data) == "" { return false } var hclData any - return hcl.Unmarshal([]byte(data), &hclData) == nil + err := hcl.Unmarshal([]byte(data), &hclData) + if err != nil { + LogDebug(atmosConfig, "HCL validation failed: %v", err) + return false + } + return true }internal/exec/vendor_model_component.go (2)
130-133: Consider making the timeout configurable.The 10-minute timeout is hardcoded, which might be insufficient for large packages or slow networks. Consider:
- Making it configurable through atmosConfig
- Implementing a retry mechanism for transient network issues
183-184: Standardize timeout handling across functions.The timeout handling is inconsistent between
installComponentandinstallMixin. Consider:
- Creating a shared constant for the timeout value
- Standardizing the error message format
+const defaultGetterTimeout = 10 * time.Minutepkg/utils/file_utils.go (2)
251-281: Consider adding path validation.While the path resolution is well-implemented, consider adding:
- Validation against path traversal attacks
- Check for absolute paths in the input
- Validation of the resolved path against allowed directories
283-316: Enhance file parsing robustness.Consider adding:
- Maximum file size check to prevent memory exhaustion
- More specific error messages for each failure case
- Content validation before parsing
+const maxFileSize = 10 * 1024 * 1024 // 10MB limit func DetectFormatAndParseFile(filename string) (any, error) { + info, err := os.Stat(filename) + if err != nil { + return nil, fmt.Errorf("failed to stat file: %w", err) + } + if info.Size() > maxFileSize { + return nil, fmt.Errorf("file size %d exceeds maximum allowed size %d", info.Size(), maxFileSize) + }website/docs/core-concepts/stacks/yaml-functions/env.mdx (2)
28-28: Add Oxford comma for better readability.Add a comma after "map" in the list of complex types.
-If the value is a complex type (list, map or object), it must be returned as a JSON-encoded string. +If the value is a complex type (list, map, or object), it must be returned as a JSON-encoded string.🧰 Tools
🪛 LanguageTool
[style] ~28-~28: The serial comma (Oxford comma, Harvard comma) is missing.
Context: ... If the value is a complex type (list, map or object), it must be returned as a JSON-...(SERIAL_COMMA_ON)
17-23: Add more comprehensive examples.The current examples only show simple string values. Consider adding examples of complex types using JSON-encoded strings.
vars: api_key: !env API_KEY + # Complex type example + config: !env APP_CONFIG # APP_CONFIG='{"timeout": 30, "retries": 3}' settings: provisioned_by_user: !env USER + # List example + allowed_regions: !env REGIONS # REGIONS='["us-east-1", "us-west-2"]'internal/exec/file_utils.go (1)
89-96: Consider using a predefined set for invalid characters.The switch statement could be replaced with a map lookup for better maintainability.
+var invalidWindowsChars = map[rune]bool{ + '\\': true, '/': true, ':': true, '*': true, + '?': true, '"': true, '<': true, '>': true, '|': true, +} base = strings.Map(func(r rune) rune { - switch r { - case '\\', '/', ':', '*', '?', '"', '<', '>', '|': + if invalidWindowsChars[r] { return '_' - default: - return r } + return r }, base)website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (2)
46-54: Enhance documentation with practical examples.The documentation clearly introduces the new functions. Consider adding quick inline examples for each function to demonstrate their basic usage without requiring users to navigate to the detailed documentation.
🧰 Tools
🪛 LanguageTool
[style] ~47-~47: A comma is missing here.
Context: ...oncepts/projects/configuration/stores) (e.g. SSM Parameter Store, Artifactory, etc.)...(EG_NO_COMMA)
[grammar] ~48-~48: Please add a punctuation mark at the end of paragraph.
Context: ... Artifactory, etc.) into Atmos stack manifests - The [!env](/core-concepts/sta...(PUNCTUATION_PARAGRAPH_END)
[style] ~53-~53: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 3167 characters long)
Context: ...ctions in Atmos stack manifests - The [!include](/core-concepts/stacks/yaml-f...(EN_EXCESSIVE_EXCLAMATION)
119-127: Add examples for advanced use cases.Consider adding examples that demonstrate:
- Combining multiple YAML functions
- Error handling scenarios
- Complex data structures
website/docs/core-concepts/stacks/yaml-functions/include.mdx (1)
129-152: Expand YQ expression examples.Consider adding examples for:
- Complex filtering operations
- Multiple value selection
- Array manipulation
- Conditional queries
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (36)
examples/quick-start-advanced/Dockerfile(1 hunks)go.mod(3 hunks)internal/exec/describe_affected.go(1 hunks)internal/exec/describe_component.go(1 hunks)internal/exec/describe_config.go(1 hunks)internal/exec/describe_dependents.go(1 hunks)internal/exec/describe_stacks.go(1 hunks)internal/exec/describe_workflows.go(1 hunks)internal/exec/file_utils.go(2 hunks)internal/exec/go_getter_utils.go(1 hunks)internal/exec/stack_processor_utils.go(3 hunks)internal/exec/terraform_outputs.go(2 hunks)internal/exec/validate_stacks.go(1 hunks)internal/exec/vendor_component_utils.go(8 hunks)internal/exec/vendor_model.go(5 hunks)internal/exec/vendor_model_component.go(6 hunks)internal/exec/vendor_utils.go(2 hunks)internal/exec/yaml_func_env.go(1 hunks)internal/exec/yaml_func_include.go(1 hunks)internal/exec/yaml_func_store.go(1 hunks)internal/exec/yaml_func_utils.go(1 hunks)pkg/config/const.go(1 hunks)pkg/utils/file_utils.go(2 hunks)pkg/utils/hcl_utils.go(2 hunks)pkg/utils/json_utils.go(1 hunks)pkg/utils/string_utils.go(2 hunks)pkg/utils/version_utils.go(1 hunks)pkg/utils/yaml_utils.go(6 hunks)pkg/utils/yq_test.go(1 hunks)pkg/utils/yq_utils.go(1 hunks)tests/fixtures/scenarios/complete/stacks/catalog/terraform/template-functions-test2/defaults.yaml(1 hunks)website/docs/core-concepts/stacks/yaml-functions/env.mdx(1 hunks)website/docs/core-concepts/stacks/yaml-functions/exec.mdx(0 hunks)website/docs/core-concepts/stacks/yaml-functions/include.mdx(1 hunks)website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx(2 hunks)website/docs/integrations/atlantis.mdx(1 hunks)
💤 Files with no reviewable changes (1)
- website/docs/core-concepts/stacks/yaml-functions/exec.mdx
✅ Files skipped from review due to trivial changes (2)
- website/docs/integrations/atlantis.mdx
- go.mod
🧰 Additional context used
📓 Learnings (7)
examples/quick-start-advanced/Dockerfile (2)
Learnt from: aknysh
PR: cloudposse/atmos#775
File: examples/quick-start-advanced/Dockerfile:9-9
Timestamp: 2024-11-12T05:52:05.088Z
Learning: It is acceptable to set `ARG ATMOS_VERSION` to a future version like `1.105.0` in `examples/quick-start-advanced/Dockerfile` if that will be the next release.
Learnt from: osterman
PR: cloudposse/atmos#801
File: examples/quick-start-advanced/Dockerfile:9-9
Timestamp: 2024-11-23T00:13:22.004Z
Learning: When updating the `ATMOS_VERSION` in Dockerfiles, the team prefers to pin to the next future version when the PR merges, even if the version is not yet released.
internal/exec/validate_stacks.go (2)
Learnt from: haitham911
PR: cloudposse/atmos#731
File: internal/exec/validate_stacks.go:0-0
Timestamp: 2024-11-12T05:52:05.088Z
Learning: In `internal/exec/validate_stacks.go`, when downloading the Atmos JSON Schema file to the temp directory, the temporary file is overwritten each time, so explicit removal is not necessary.
Learnt from: haitham911
PR: cloudposse/atmos#731
File: internal/exec/validate_stacks.go:93-98
Timestamp: 2024-11-12T05:52:05.088Z
Learning: When downloading schema files in `internal/exec/validate_stacks.go`, use a consistent temporary file name to overwrite the file each time and avoid creating multiple temporary files.
website/docs/core-concepts/stacks/yaml-functions/env.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!env` is used to retrieve environment variables and assign them to sections in stack manifests. It supports both simple types (string, number, boolean) and complex types (JSON-encoded lists, maps, objects).
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!include` allows downloading local or remote files from different sources and assigning their contents to sections in stack manifests. It supports various protocols (file, http, git, s3, etc.) and can filter content using YQ expressions.
website/docs/core-concepts/stacks/yaml-functions/include.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!include` allows downloading local or remote files from different sources and assigning their contents to sections in stack manifests. It supports various protocols (file, http, git, s3, etc.) and can filter content using YQ expressions.
internal/exec/describe_stacks.go (1)
Learnt from: Cerebrovinny
PR: cloudposse/atmos#764
File: internal/exec/describe_stacks.go:289-295
Timestamp: 2024-11-13T21:37:07.852Z
Learning: In the `internal/exec/describe_stacks.go` file of the `atmos` project written in Go, avoid extracting the stack name handling logic into a helper function within the `ExecuteDescribeStacks` method, even if the logic appears duplicated.
internal/exec/stack_processor_utils.go (1)
Learnt from: osterman
PR: cloudposse/atmos#795
File: internal/exec/stack_processor_utils.go:378-386
Timestamp: 2024-11-19T23:00:45.899Z
Learning: In the `ProcessYAMLConfigFile` function within `internal/exec/stack_processor_utils.go`, directory traversal in stack imports is acceptable and should not be restricted.
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/env.mdx
[style] ~28-~28: The serial comma (Oxford comma, Harvard comma) is missing.
Context: ... If the value is a complex type (list, map or object), it must be returned as a JSON-...
(SERIAL_COMMA_ON)
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx
[style] ~47-~47: A comma is missing here.
Context: ...oncepts/projects/configuration/stores) (e.g. SSM Parameter Store, Artifactory, etc.)...
(EG_NO_COMMA)
[grammar] ~48-~48: Please add a punctuation mark at the end of paragraph.
Context: ... Artifactory, etc.) into Atmos stack manifests - The [!env](/core-concepts/sta...
(PUNCTUATION_PARAGRAPH_END)
[style] ~53-~53: Using many exclamation marks might seem excessive (in this case: 7 exclamation marks for a text that’s 3167 characters long)
Context: ...ctions in Atmos stack manifests - The [!include](/core-concepts/stacks/yaml-f...
(EN_EXCESSIVE_EXCLAMATION)
website/docs/core-concepts/stacks/yaml-functions/include.mdx
[grammar] ~99-~99: Please add a punctuation mark at the end of paragraph.
Context: ...wnguide.org/) files as strings in Atmos manifests ## Supported File Protocols The `!inc...
(PUNCTUATION_PARAGRAPH_END)
[style] ~104-~104: A comma is missing here.
Context: ...g local file paths: - absolute paths (e.g. /Users/me/Documents/values.yaml) - ...
(EG_NO_COMMA)
[style] ~105-~105: A comma is missing here.
Context: ...re the !include function is executed (e.g. ./values.yaml, `../config/values.yaml...
(EG_NO_COMMA)
[style] ~106-~106: A comma is missing here.
Context: ...efined in atmos.yaml CLI config file (e.g. stacks/catalog/vpc/defaults.yaml) To...
(EG_NO_COMMA)
[typographical] ~111-~111: To join two clauses or introduce examples, consider using an em dash.
Context: ...pports the following protocols: - tar - Tar files, potentially compressed (tar.g...
(DASH_RULE)
[typographical] ~112-~112: To join two clauses or introduce examples, consider using an em dash.
Context: ...mpressed (tar.gz, tar.bz2, etc.) - zip - Zip files - http - HTTP URLs - https...
(DASH_RULE)
[typographical] ~113-~113: To join two clauses or introduce examples, consider using an em dash.
Context: ....bz2, etc.) - zip - Zip files - http - HTTP URLs - https - HTTPS URLs - git...
(DASH_RULE)
[typographical] ~114-~114: To join two clauses or introduce examples, consider using an em dash.
Context: ...Zip files - http - HTTP URLs - https - HTTPS URLs - git - Git repositories, w...
(DASH_RULE)
[typographical] ~115-~115: To join two clauses or introduce examples, consider using an em dash.
Context: ...HTTP URLs - https - HTTPS URLs - git - Git repositories, which can be accessed ...
(DASH_RULE)
[typographical] ~116-~116: To join two clauses or introduce examples, consider using an em dash.
Context: ... can be accessed via HTTPS or SSH - hg - Mercurial repositories, accessed via HTT...
(DASH_RULE)
[typographical] ~117-~117: To join two clauses or introduce examples, consider using an em dash.
Context: ...ories, accessed via HTTP/S or SSH - s3 - Amazon S3 bucket URLs - gcs - Google C...
(DASH_RULE)
[typographical] ~118-~118: To join two clauses or introduce examples, consider using an em dash.
Context: ...H - s3 - Amazon S3 bucket URLs - gcs - Google Cloud Storage URLs - oci - Open...
(DASH_RULE)
[typographical] ~119-~119: To join two clauses or introduce examples, consider using an em dash.
Context: ...gcs- Google Cloud Storage URLs -oci` - Open Container Initiative (OCI) images -...
(DASH_RULE)
[typographical] ~120-~120: To join two clauses or introduce examples, consider using an em dash.
Context: ...ontainer Initiative (OCI) images - scp - Secure Copy Protocol for SSH-based trans...
(DASH_RULE)
[typographical] ~121-~121: To join two clauses or introduce examples, consider using an em dash.
Context: ...rotocol for SSH-based transfers - sftp - SSH File Transfer Protocol :::tip You c...
(DASH_RULE)
[style] ~125-~125: Using many exclamation marks might seem excessive (in this case: 8 exclamation marks for a text that’s 3710 characters long)
Context: ... templates first, and then executes the !include function, allowing you to provi...
(EN_EXCESSIVE_EXCLAMATION)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Summary
🔇 Additional comments (28)
internal/exec/vendor_utils.go (2)
364-364: EnsureValidateURIis properly accessible.At line 364, the function
ValidateURIis called. Confirm thatValidateURIis correctly imported from its new location,internal/exec/go_getter_utils.go, and that it's accessible in this context.
537-537: ConfirmSanitizeFileNamefunction usage.The call to
SanitizeFileName(uri)replaces the previoussanitizeFileNamefunction. Make sure thatSanitizeFileNameis properly imported frominternal/exec/file_utils.goand that it behaves as expected.internal/exec/stack_processor_utils.go (2)
221-221: Update toUnmarshalYAMLFromFileenhances error context.The change to use
u.UnmarshalYAMLFromFileat line 221 improves error handling by includingatmosConfigandfilePathin the unmarshalling process, providing better context in case of errors.
1811-1811: Correct handling of relative paths in imports.The use of
u.ResolveRelativePathat lines 1811 and 1825 ensures that relative paths in the import section are correctly resolved, allowing directory traversal in stack imports as per prior learnings.Also applies to: 1825-1825
pkg/utils/version_utils.go (1)
8-9: LGTM! Enhanced version displayThe addition of themed styling improves the visibility of version information for users.
internal/exec/describe_workflows.go (1)
72-72: LGTM! Consistent pointer usageThe change to pass atmosConfig by reference maintains consistency with other command execution functions.
internal/exec/yaml_func_store.go (1)
8-10: 🛠️ Refactor suggestionMaintain consistency in battle formation, warrior!
While the schema import is correctly added, we should maintain consistency with other functions that now use pointer receivers.
Update the function signature to use a pointer receiver:
-func processTagStore(atmosConfig schema.AtmosConfiguration, input string, currentStack string) any { +func processTagStore(atmosConfig *schema.AtmosConfiguration, input string, currentStack string) any {Likely invalid or redundant comment.
internal/exec/yaml_func_utils.go (1)
73-78: Well-structured battle plan, warrior!The new YAML function handlers are cleanly integrated into the existing switch statement, and the comment update provides better clarity about YAML explicit tags.
Also applies to: 80-82
internal/exec/describe_component.go (1)
66-66: Clean and consistent code adaptation, warrior!The update to use
&atmosConfigaligns perfectly with the new pointer receiver pattern implemented across the codebase.pkg/config/const.go (1)
82-84: LGTM! The new constant follows the established pattern.The addition of
AtmosYamlFuncEnvconstant aligns well with the PR objectives and maintains consistency with other YAML function constants.pkg/utils/yq_test.go (1)
46-46: LGTM! Consistent with pointer usage pattern.The change to use a pointer for
atmosConfigaligns with the broader codebase updates and maintains consistency across the project.internal/exec/vendor_model_component.go (1)
Line range hint
109-125: Well-structured temporary directory handling!The code demonstrates good security practices with restricted permissions (0700) and proper cleanup using defer. The comments effectively explain the rationale behind using a temporary directory.
internal/exec/describe_affected.go (1)
282-282: Good optimization using pointer receiver!Passing
atmosConfigby reference prevents unnecessary copying of the configuration structure.internal/exec/vendor_model.go (2)
116-124: Excellent UX considerations!The code provides multiple ways to quit (ctrl+c, esc, q) and sensibly limits the width to 120 characters for better readability.
271-271: Same timeout configuration concern as in vendor_model_component.gointernal/exec/describe_dependents.go (1)
70-70: LGTM! Memory optimization through pointer usage.The change to pass
atmosConfigby reference is a good optimization that aligns with the codebase-wide pattern.internal/exec/terraform_outputs.go (1)
270-270: LGTM! Consistent pointer usage.The changes maintain consistency with the codebase-wide pattern of passing
atmosConfigby reference.Also applies to: 296-296
internal/exec/vendor_component_utils.go (4)
18-21: LGTM! Clean import organization.The import statements are now better organized for improved readability.
186-186: LGTM! Improved file name sanitization.Using
SanitizeFileNameensures better handling of invalid characters in file names, particularly important for cross-platform compatibility.
Line range hint
228-252: Robust URI and file system handling added.The enhanced URI scheme and local file system checks improve the reliability of component vendoring. The code now properly handles:
- OCI scheme detection
- Local file path resolution
- Source type determination
Line range hint
275-348: Enhanced mixin processing with better validation.The mixin processing logic has been improved with:
- Required field validation for
uriandfilename- Proper URI template handling
- Consistent scheme detection
internal/exec/validate_stacks.go (2)
381-385: LGTM! Added URL scheme validation.The explicit check for HTTP/HTTPS schemes improves security and provides clearer error messages.
392-397: LGTM! Improved schema file download handling.The changes align with the learnings about temporary file handling and introduce better download management through
GoGetterGet.Note: As per previous learnings, the temporary file is automatically overwritten each time, which is the desired behavior.
examples/quick-start-advanced/Dockerfile (1)
9-9: LGTM! Version update follows team conventions.The update to
ATMOS_VERSION=1.153.0aligns with the team's practice of pinning to future versions.internal/exec/file_utils.go (1)
71-99: Consider adding unit tests for edge cases.The
SanitizeFileNamefunction handles various scenarios, but should be thoroughly tested with:
- URIs with query parameters
- Paths with multiple invalid characters
- Unicode characters
- Empty strings
- Malformed URIs
tests/fixtures/scenarios/complete/stacks/catalog/terraform/template-functions-test2/defaults.yaml (3)
65-67: Add error handling test cases for remote file inclusion.The test cases should include scenarios for:
- Network timeouts
- Invalid URLs
- Rate limiting (especially for GitHub API)
- SSL certificate validation
Would you like me to generate additional test cases covering these scenarios?
64-64: Uncomment and implement the environment variable test case.The commented test case for
!envshould be implemented to ensure proper functionality of environment variable retrieval.
54-63: Consider adding path validation for local file inclusion.While the test cases are comprehensive, consider implementing path validation to prevent directory traversal attacks when including local files. This is especially important for files referenced with relative paths.
.../fixtures/scenarios/complete/stacks/catalog/terraform/template-functions-test2/defaults.yaml
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
website/docs/core-concepts/stacks/yaml-functions/include.mdx (1)
89-99: Add period at the end of the paragraph.The features list is comprehensive, but the last paragraph is missing a period.
All other files are returned unchanged, allowing you, for example, to include text and - [Markdown](https://www.markdownguide.org/) files as strings in Atmos manifests + [Markdown](https://www.markdownguide.org/) files as strings in Atmos manifests.🧰 Tools
🪛 LanguageTool
[grammar] ~99-~99: Please add a punctuation mark at the end of paragraph.
Context: ...wnguide.org/) files as strings in Atmos manifests ## Supported File Protocols The `!inc...(PUNCTUATION_PARAGRAPH_END)
website/docs/core-concepts/stacks/yaml-functions/env.mdx (3)
19-27: Enhance usage documentation with type support and behavior details.The usage section would benefit from additional clarity about:
- Supported value types (simple scalars vs complex types)
- Explicit behavior when environment variables are not set
# Get the value of an environment variable. # If the environment variable is not set, `null` will be assigned + # Supports simple scalar values (string, number, boolean) !env <env-var-name> # Get the value of an environment variable. # If the environment variable is not set, the `default-value` will be assigned + # The default value must be a simple scalar !env <env-var-name> <default-value>
43-50: Add more comprehensive examples.Consider adding examples that demonstrate:
- Different value types (string, number, boolean)
- Behavior when env var is not set (returns null)
vars: api_key: !env API_KEY is_test: !env IS_TEST false + # Examples with different types + port: !env PORT 8080 # number + debug: !env DEBUG true # boolean + unset_var: !env UNSET_VAR # returns null settings: provisioned_by_user: !env USER
62-66: Clarify template processing order.The tip about template processing would benefit from a concrete example to illustrate the order of operations.
:::tip You can use [Atmos Stack Manifest Templating](/core-concepts/stacks/templates) in the environment variable names and default values when calling the `!env` YAML function. Atmos processes the templates first, and then executes the `!env` function, allowing you to provide the parameters to the function dynamically. + + For example: + ```yaml + region: !env '${component.region}_ENDPOINT' # Template is processed first + ``` :::🧰 Tools
🪛 LanguageTool
[style] ~64-~64: Using many exclamation marks might seem excessive (in this case: 3 exclamation marks for a text that’s 705 characters long)
Context: ... templates first, and then executes the!envfunction, allowing you to provide t...(EN_EXCESSIVE_EXCLAMATION)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
internal/exec/yaml_func_env.go(1 hunks)website/docs/core-concepts/stacks/yaml-functions/env.mdx(1 hunks)website/docs/core-concepts/stacks/yaml-functions/include.mdx(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- internal/exec/yaml_func_env.go
🧰 Additional context used
📓 Learnings (2)
website/docs/core-concepts/stacks/yaml-functions/include.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!include` allows downloading local or remote files from different sources and assigning their contents to sections in stack manifests. It supports various protocols (file, http, git, s3, etc.) and can filter content using YQ expressions.
website/docs/core-concepts/stacks/yaml-functions/env.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!env` is used to retrieve environment variables and assign them to sections in stack manifests. It supports both simple types (string, number, boolean) and complex types (JSON-encoded lists, maps, objects).
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/include.mdx
[grammar] ~99-~99: Please add a punctuation mark at the end of paragraph.
Context: ...wnguide.org/) files as strings in Atmos manifests ## Supported File Protocols The `!inc...
(PUNCTUATION_PARAGRAPH_END)
[style] ~104-~104: A comma is missing here.
Context: ...g local file paths: - absolute paths (e.g. /Users/me/Documents/values.yaml) - ...
(EG_NO_COMMA)
[style] ~105-~105: A comma is missing here.
Context: ...re the !include function is executed (e.g. ./values.yaml, `../config/values.yaml...
(EG_NO_COMMA)
[style] ~106-~106: A comma is missing here.
Context: ...efined in atmos.yaml CLI config file (e.g. stacks/catalog/vpc/defaults.yaml) To...
(EG_NO_COMMA)
[typographical] ~111-~111: To join two clauses or introduce examples, consider using an em dash.
Context: ...pports the following protocols: - tar - Tar files, potentially compressed (tar.g...
(DASH_RULE)
[typographical] ~112-~112: To join two clauses or introduce examples, consider using an em dash.
Context: ...mpressed (tar.gz, tar.bz2, etc.) - zip - Zip files - http - HTTP URLs - https...
(DASH_RULE)
[typographical] ~113-~113: To join two clauses or introduce examples, consider using an em dash.
Context: ....bz2, etc.) - zip - Zip files - http - HTTP URLs - https - HTTPS URLs - git...
(DASH_RULE)
[typographical] ~114-~114: To join two clauses or introduce examples, consider using an em dash.
Context: ...Zip files - http - HTTP URLs - https - HTTPS URLs - git - Git repositories, w...
(DASH_RULE)
[typographical] ~115-~115: To join two clauses or introduce examples, consider using an em dash.
Context: ...HTTP URLs - https - HTTPS URLs - git - Git repositories, which can be accessed ...
(DASH_RULE)
[typographical] ~116-~116: To join two clauses or introduce examples, consider using an em dash.
Context: ... can be accessed via HTTPS or SSH - hg - Mercurial repositories, accessed via HTT...
(DASH_RULE)
[typographical] ~117-~117: To join two clauses or introduce examples, consider using an em dash.
Context: ...ories, accessed via HTTP/S or SSH - s3 - Amazon S3 bucket URLs - gcs - Google C...
(DASH_RULE)
[typographical] ~118-~118: To join two clauses or introduce examples, consider using an em dash.
Context: ...H - s3 - Amazon S3 bucket URLs - gcs - Google Cloud Storage URLs - oci - Open...
(DASH_RULE)
[typographical] ~119-~119: To join two clauses or introduce examples, consider using an em dash.
Context: ...gcs- Google Cloud Storage URLs -oci` - Open Container Initiative (OCI) images -...
(DASH_RULE)
[typographical] ~120-~120: To join two clauses or introduce examples, consider using an em dash.
Context: ...ontainer Initiative (OCI) images - scp - Secure Copy Protocol for SSH-based trans...
(DASH_RULE)
[typographical] ~121-~121: To join two clauses or introduce examples, consider using an em dash.
Context: ...rotocol for SSH-based transfers - sftp - SSH File Transfer Protocol :::tip You c...
(DASH_RULE)
[style] ~125-~125: Using many exclamation marks might seem excessive (in this case: 8 exclamation marks for a text that’s 3718 characters long)
Context: ... templates first, and then executes the !include function, allowing you to provi...
(EN_EXCESSIVE_EXCLAMATION)
website/docs/core-concepts/stacks/yaml-functions/env.mdx
[style] ~64-~64: Using many exclamation marks might seem excessive (in this case: 3 exclamation marks for a text that’s 705 characters long)
Context: ... templates first, and then executes the !env function, allowing you to provide t...
(EN_EXCESSIVE_EXCLAMATION)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (6)
website/docs/core-concepts/stacks/yaml-functions/include.mdx (5)
12-14: LGTM! Clear and concise introduction.The introduction effectively communicates the purpose and capabilities of the
!includefunction.
16-27: Add error handling documentation.While the usage section is clear, it would be beneficial to document how errors are handled and displayed.
Based on the past review comments where @osterman asked about error handling and @aknysh confirmed errors are shown in the console, let's add this information to the documentation.
# and assign the result to the variable !include <file-path> <yq-expression> + +:::note +All errors that occur during file inclusion (e.g., file not found, permission denied, invalid YQ expression) +are displayed in the console with detailed error messages. +:::
49-75: LGTM! Comprehensive examples covering various use cases.The examples section effectively demonstrates:
- Local and remote file inclusion
- Different file formats (YAML, HCL, JSON, Markdown, text)
- Path handling with spaces
- YQ expressions for value extraction
111-121: Consider adding security considerations for each protocol.While the protocols are well-documented, adding security considerations would be valuable.
Based on the past review comments where this was suggested and @aknysh mentioned it could be added in a separate PR, I'll skip proposing the changes here.
🧰 Tools
🪛 LanguageTool
[typographical] ~111-~111: To join two clauses or introduce examples, consider using an em dash.
Context: ...pports the following protocols: -tar- Tar files, potentially compressed (tar.g...(DASH_RULE)
[typographical] ~112-~112: To join two clauses or introduce examples, consider using an em dash.
Context: ...mpressed (tar.gz, tar.bz2, etc.) -zip- Zip files -http- HTTP URLs -https...(DASH_RULE)
[typographical] ~113-~113: To join two clauses or introduce examples, consider using an em dash.
Context: ....bz2, etc.) -zip- Zip files -http- HTTP URLs -https- HTTPS URLs -git...(DASH_RULE)
[typographical] ~114-~114: To join two clauses or introduce examples, consider using an em dash.
Context: ...Zip files -http- HTTP URLs -https- HTTPS URLs -git- Git repositories, w...(DASH_RULE)
[typographical] ~115-~115: To join two clauses or introduce examples, consider using an em dash.
Context: ...HTTP URLs -https- HTTPS URLs -git- Git repositories, which can be accessed ...(DASH_RULE)
[typographical] ~116-~116: To join two clauses or introduce examples, consider using an em dash.
Context: ... can be accessed via HTTPS or SSH -hg- Mercurial repositories, accessed via HTT...(DASH_RULE)
[typographical] ~117-~117: To join two clauses or introduce examples, consider using an em dash.
Context: ...ories, accessed via HTTP/S or SSH -s3- Amazon S3 bucket URLs -gcs- Google C...(DASH_RULE)
[typographical] ~118-~118: To join two clauses or introduce examples, consider using an em dash.
Context: ...H -s3- Amazon S3 bucket URLs -gcs- Google Cloud Storage URLs -oci- Open...(DASH_RULE)
[typographical] ~119-~119: To join two clauses or introduce examples, consider using an em dash.
Context: ...gcs- Google Cloud Storage URLs -oci` - Open Container Initiative (OCI) images -...(DASH_RULE)
[typographical] ~120-~120: To join two clauses or introduce examples, consider using an em dash.
Context: ...ontainer Initiative (OCI) images -scp- Secure Copy Protocol for SSH-based trans...(DASH_RULE)
[typographical] ~121-~121: To join two clauses or introduce examples, consider using an em dash.
Context: ...rotocol for SSH-based transfers -sftp- SSH File Transfer Protocol :::tip You c...(DASH_RULE)
154-172: LGTM! Clear instructions for handling special cases.The section on handling file paths and YQ expressions with spaces is well-documented with clear examples for both Windows and Unix-like systems.
website/docs/core-concepts/stacks/yaml-functions/env.mdx (1)
1-6: Well-structured frontmatter!The metadata is clear and comprehensive, providing good context for the documentation.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
website/docs/core-concepts/stacks/yaml-functions/env.mdx (1)
57-57: Consider using a more universal environment variable example.The
USERenvironment variable might not be available in all environments (e.g., containerized environments). Consider using a more universal example likeHOSTNAMEor a custom application-specific variable likeATMOS_USER.website/docs/core-concepts/stacks/yaml-functions/include.mdx (2)
99-99: Add missing punctuation at the end of the paragraph.Please add a period at the end of the paragraph on line 99 to enhance readability.
🧰 Tools
🪛 LanguageTool
[grammar] ~99-~99: Please add a punctuation mark at the end of paragraph.
Context: ...wnguide.org/) files as strings in Atmos manifests ## Supported File Protocols The `!inc...(PUNCTUATION_PARAGRAPH_END)
104-106: Include commas after 'e.g.' in examples.Adding commas after 'e.g.' improves clarity in your examples.
Apply this diff:
- - absolute paths (e.g. `/Users/me/Documents/values.yaml`) + - absolute paths (e.g., `/Users/me/Documents/values.yaml`) - - paths relative to the current Atmos manifest where the `!include` function is executed (e.g. `./values.yaml`, `../config/values.yaml`) + - paths relative to the current Atmos manifest where the `!include` function is executed (e.g., `./values.yaml`, `../config/values.yaml`) - - paths relative to the [`base_path`](/cli/configuration/#base-path) defined in `atmos.yaml` CLI config file (e.g. `stacks/catalog/vpc/defaults.yaml`) + - paths relative to the [`base_path`](/cli/configuration/#base-path) defined in `atmos.yaml` CLI config file (e.g., `stacks/catalog/vpc/defaults.yaml`)🧰 Tools
🪛 LanguageTool
[style] ~104-~104: A comma is missing here.
Context: ...g local file paths: - absolute paths (e.g./Users/me/Documents/values.yaml) - ...(EG_NO_COMMA)
[style] ~105-~105: A comma is missing here.
Context: ...re the!includefunction is executed (e.g../values.yaml, `../config/values.yaml...(EG_NO_COMMA)
[style] ~106-~106: A comma is missing here.
Context: ...efined inatmos.yamlCLI config file (e.g.stacks/catalog/vpc/defaults.yaml) To...(EG_NO_COMMA)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
internal/exec/yaml_func_env.go(1 hunks)internal/exec/yaml_func_include.go(1 hunks)website/docs/core-concepts/stacks/yaml-functions/env.mdx(1 hunks)website/docs/core-concepts/stacks/yaml-functions/include.mdx(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- internal/exec/yaml_func_env.go
🧰 Additional context used
📓 Learnings (2)
website/docs/core-concepts/stacks/yaml-functions/env.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!env` is used to retrieve environment variables and assign them to sections in stack manifests. It supports both simple types (string, number, boolean) and complex types (JSON-encoded lists, maps, objects).
website/docs/core-concepts/stacks/yaml-functions/include.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!include` allows downloading local or remote files from different sources and assigning their contents to sections in stack manifests. It supports various protocols (file, http, git, s3, etc.) and can filter content using YQ expressions.
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/env.mdx
[style] ~76-~76: Using many exclamation marks might seem excessive (in this case: 3 exclamation marks for a text that’s 1153 characters long)
Context: ... templates first, and then executes the !env function, allowing you to provide t...
(EN_EXCESSIVE_EXCLAMATION)
website/docs/core-concepts/stacks/yaml-functions/include.mdx
[grammar] ~99-~99: Please add a punctuation mark at the end of paragraph.
Context: ...wnguide.org/) files as strings in Atmos manifests ## Supported File Protocols The `!inc...
(PUNCTUATION_PARAGRAPH_END)
[style] ~104-~104: A comma is missing here.
Context: ...g local file paths: - absolute paths (e.g. /Users/me/Documents/values.yaml) - ...
(EG_NO_COMMA)
[style] ~105-~105: A comma is missing here.
Context: ...re the !include function is executed (e.g. ./values.yaml, `../config/values.yaml...
(EG_NO_COMMA)
[style] ~106-~106: A comma is missing here.
Context: ...efined in atmos.yaml CLI config file (e.g. stacks/catalog/vpc/defaults.yaml) To...
(EG_NO_COMMA)
[typographical] ~111-~111: To join two clauses or introduce examples, consider using an em dash.
Context: ...pports the following protocols: - tar - Tar files, potentially compressed (tar.g...
(DASH_RULE)
[typographical] ~112-~112: To join two clauses or introduce examples, consider using an em dash.
Context: ...mpressed (tar.gz, tar.bz2, etc.) - zip - Zip files - http - HTTP URLs - https...
(DASH_RULE)
[typographical] ~113-~113: To join two clauses or introduce examples, consider using an em dash.
Context: ....bz2, etc.) - zip - Zip files - http - HTTP URLs - https - HTTPS URLs - git...
(DASH_RULE)
[typographical] ~114-~114: To join two clauses or introduce examples, consider using an em dash.
Context: ...Zip files - http - HTTP URLs - https - HTTPS URLs - git - Git repositories, w...
(DASH_RULE)
[typographical] ~115-~115: To join two clauses or introduce examples, consider using an em dash.
Context: ...HTTP URLs - https - HTTPS URLs - git - Git repositories, which can be accessed ...
(DASH_RULE)
[typographical] ~116-~116: To join two clauses or introduce examples, consider using an em dash.
Context: ... can be accessed via HTTPS or SSH - hg - Mercurial repositories, accessed via HTT...
(DASH_RULE)
[typographical] ~117-~117: To join two clauses or introduce examples, consider using an em dash.
Context: ...ories, accessed via HTTP/S or SSH - s3 - Amazon S3 bucket URLs - gcs - Google C...
(DASH_RULE)
[typographical] ~118-~118: To join two clauses or introduce examples, consider using an em dash.
Context: ...H - s3 - Amazon S3 bucket URLs - gcs - Google Cloud Storage URLs - oci - Open...
(DASH_RULE)
[typographical] ~119-~119: To join two clauses or introduce examples, consider using an em dash.
Context: ...gcs- Google Cloud Storage URLs -oci` - Open Container Initiative (OCI) images -...
(DASH_RULE)
[typographical] ~120-~120: To join two clauses or introduce examples, consider using an em dash.
Context: ...ontainer Initiative (OCI) images - scp - Secure Copy Protocol for SSH-based trans...
(DASH_RULE)
[typographical] ~121-~121: To join two clauses or introduce examples, consider using an em dash.
Context: ...rotocol for SSH-based transfers - sftp - SSH File Transfer Protocol :::tip You c...
(DASH_RULE)
[style] ~125-~125: Using many exclamation marks might seem excessive (in this case: 8 exclamation marks for a text that’s 3716 characters long)
Context: ... templates first, and then executes the !include function, allowing you to provi...
(EN_EXCESSIVE_EXCLAMATION)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Acceptance Tests (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (6)
website/docs/core-concepts/stacks/yaml-functions/env.mdx (4)
1-13: Well-structured frontmatter and introduction!The metadata and introduction clearly communicate the purpose of the
!envfunction.
15-46: Clear documentation of function behavior!The usage and arguments sections effectively communicate:
- Function parameters and their purpose
- Behavior when environment variables are not set
- Default value handling
60-72: Clear explanation of quoting requirements!The documentation effectively demonstrates how to handle default values containing spaces using appropriate quoting.
74-78: Helpful tip about template processing order!The tip effectively explains how the
!envfunction integrates with Atmos Stack Manifest Templating, clarifying that templates are processed before the function executes.🧰 Tools
🪛 LanguageTool
[style] ~76-~76: Using many exclamation marks might seem excessive (in this case: 3 exclamation marks for a text that’s 1153 characters long)
Context: ... templates first, and then executes the!envfunction, allowing you to provide t...(EN_EXCESSIVE_EXCLAMATION)
internal/exec/yaml_func_include.go (1)
1-66: Code looks great!The
processTagIncludefunction is well-structured with proper error handling and logging. It aligns well with the project's standards.website/docs/core-concepts/stacks/yaml-functions/include.mdx (1)
111-121: Use em dashes instead of hyphens in the protocol list.Replacing hyphens with em dashes enhances readability in your list of supported protocols.
Apply this diff:
- - `tar` - Tar files, potentially compressed (tar.gz, tar.bz2, etc.) + - `tar` — Tar files, potentially compressed (tar.gz, tar.bz2, etc.)Repeat for lines 112-121.
🧰 Tools
🪛 LanguageTool
[typographical] ~111-~111: To join two clauses or introduce examples, consider using an em dash.
Context: ...pports the following protocols: -tar- Tar files, potentially compressed (tar.g...(DASH_RULE)
[typographical] ~112-~112: To join two clauses or introduce examples, consider using an em dash.
Context: ...mpressed (tar.gz, tar.bz2, etc.) -zip- Zip files -http- HTTP URLs -https...(DASH_RULE)
[typographical] ~113-~113: To join two clauses or introduce examples, consider using an em dash.
Context: ....bz2, etc.) -zip- Zip files -http- HTTP URLs -https- HTTPS URLs -git...(DASH_RULE)
[typographical] ~114-~114: To join two clauses or introduce examples, consider using an em dash.
Context: ...Zip files -http- HTTP URLs -https- HTTPS URLs -git- Git repositories, w...(DASH_RULE)
[typographical] ~115-~115: To join two clauses or introduce examples, consider using an em dash.
Context: ...HTTP URLs -https- HTTPS URLs -git- Git repositories, which can be accessed ...(DASH_RULE)
[typographical] ~116-~116: To join two clauses or introduce examples, consider using an em dash.
Context: ... can be accessed via HTTPS or SSH -hg- Mercurial repositories, accessed via HTT...(DASH_RULE)
[typographical] ~117-~117: To join two clauses or introduce examples, consider using an em dash.
Context: ...ories, accessed via HTTP/S or SSH -s3- Amazon S3 bucket URLs -gcs- Google C...(DASH_RULE)
[typographical] ~118-~118: To join two clauses or introduce examples, consider using an em dash.
Context: ...H -s3- Amazon S3 bucket URLs -gcs- Google Cloud Storage URLs -oci- Open...(DASH_RULE)
[typographical] ~119-~119: To join two clauses or introduce examples, consider using an em dash.
Context: ...gcs- Google Cloud Storage URLs -oci` - Open Container Initiative (OCI) images -...(DASH_RULE)
[typographical] ~120-~120: To join two clauses or introduce examples, consider using an em dash.
Context: ...ontainer Initiative (OCI) images -scp- Secure Copy Protocol for SSH-based trans...(DASH_RULE)
[typographical] ~121-~121: To join two clauses or introduce examples, consider using an em dash.
Context: ...rotocol for SSH-based transfers -sftp- SSH File Transfer Protocol :::tip You c...(DASH_RULE)
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
website/docs/core-concepts/stacks/yaml-functions/env.mdx (1)
17-27: Add error handling documentation to the usage section.The usage section would benefit from documenting how errors are handled and displayed to users. Consider adding examples of common error scenarios and their corresponding error messages.
website/docs/core-concepts/stacks/yaml-functions/include.mdx (1)
18-27: Add security considerations to the usage section.Consider adding security best practices for handling sensitive files and using remote protocols. Include recommendations for:
- File permissions
- URL validation
- Authentication handling
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
website/docs/core-concepts/stacks/yaml-functions/env.mdx(1 hunks)website/docs/core-concepts/stacks/yaml-functions/include.mdx(1 hunks)
🧰 Additional context used
📓 Learnings (2)
website/docs/core-concepts/stacks/yaml-functions/env.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!env` is used to retrieve environment variables and assign them to sections in stack manifests. It supports both simple types (string, number, boolean) and complex types (JSON-encoded lists, maps, objects).
website/docs/core-concepts/stacks/yaml-functions/include.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#0
File: :0-0
Timestamp: 2025-01-19T22:30:27.600Z
Learning: The Atmos YAML function `!include` allows downloading local or remote files from different sources and assigning their contents to sections in stack manifests. It supports various protocols (file, http, git, s3, etc.) and can filter content using YQ expressions.
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/env.mdx
[style] ~77-~77: Using many exclamation marks might seem excessive (in this case: 3 exclamation marks for a text that’s 1154 characters long)
Context: ... templates first, and then executes the !env function, allowing you to provide t...
(EN_EXCESSIVE_EXCLAMATION)
website/docs/core-concepts/stacks/yaml-functions/include.mdx
[typographical] ~111-~111: To join two clauses or introduce examples, consider using an em dash.
Context: ...pports the following protocols: - tar - Tar files, potentially compressed (tar.g...
(DASH_RULE)
[typographical] ~112-~112: To join two clauses or introduce examples, consider using an em dash.
Context: ...mpressed (tar.gz, tar.bz2, etc.) - zip - Zip files - http - HTTP URLs - https...
(DASH_RULE)
[typographical] ~113-~113: To join two clauses or introduce examples, consider using an em dash.
Context: ....bz2, etc.) - zip - Zip files - http - HTTP URLs - https - HTTPS URLs - git...
(DASH_RULE)
[typographical] ~114-~114: To join two clauses or introduce examples, consider using an em dash.
Context: ...Zip files - http - HTTP URLs - https - HTTPS URLs - git - Git repositories, w...
(DASH_RULE)
[typographical] ~115-~115: To join two clauses or introduce examples, consider using an em dash.
Context: ...HTTP URLs - https - HTTPS URLs - git - Git repositories, which can be accessed ...
(DASH_RULE)
[typographical] ~116-~116: To join two clauses or introduce examples, consider using an em dash.
Context: ... can be accessed via HTTPS or SSH - hg - Mercurial repositories, accessed via HTT...
(DASH_RULE)
[typographical] ~117-~117: To join two clauses or introduce examples, consider using an em dash.
Context: ...ories, accessed via HTTP/S or SSH - s3 - Amazon S3 bucket URLs - gcs - Google C...
(DASH_RULE)
[typographical] ~118-~118: To join two clauses or introduce examples, consider using an em dash.
Context: ...H - s3 - Amazon S3 bucket URLs - gcs - Google Cloud Storage URLs - oci - Open...
(DASH_RULE)
[typographical] ~119-~119: To join two clauses or introduce examples, consider using an em dash.
Context: ...gcs- Google Cloud Storage URLs -oci` - Open Container Initiative (OCI) images -...
(DASH_RULE)
[typographical] ~120-~120: To join two clauses or introduce examples, consider using an em dash.
Context: ...ontainer Initiative (OCI) images - scp - Secure Copy Protocol for SSH-based trans...
(DASH_RULE)
[typographical] ~121-~121: To join two clauses or introduce examples, consider using an em dash.
Context: ...rotocol for SSH-based transfers - sftp - SSH File Transfer Protocol :::tip You c...
(DASH_RULE)
[style] ~125-~125: Using many exclamation marks might seem excessive (in this case: 8 exclamation marks for a text that’s 3723 characters long)
Context: ... templates first, and then executes the !include function, allowing you to provi...
(EN_EXCESSIVE_EXCLAMATION)
⏰ Context from checks skipped due to timeout of 90000ms (5)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Build (ubuntu-latest, linux)
- GitHub Check: website-deploy-preview
- GitHub Check: Analyze (go)
- GitHub Check: Summary
🔇 Additional comments (7)
website/docs/core-concepts/stacks/yaml-functions/env.mdx (2)
31-39: Well-structured arguments section!The arguments section clearly defines both required and optional parameters with appropriate descriptions.
49-59: Excellent examples with security-conscious implementation!The examples effectively demonstrate the function's usage while maintaining security by avoiding environment variable evaluation.
website/docs/core-concepts/stacks/yaml-functions/include.mdx (5)
43-76: Comprehensive examples covering all major use cases!The examples effectively demonstrate:
- Local and remote file inclusion
- Various file formats (YAML, JSON, HCL, Markdown)
- YQ expressions for filtering
78-100: Excellent technical explanation!The description effectively explains the function's purpose, limitations it addresses, and technical capabilities.
111-121: Add protocol-specific security documentation.For each supported protocol, consider documenting:
- Authentication requirements
- SSL/TLS configurations
- Network security implications
- Rate limiting considerations
🧰 Tools
🪛 LanguageTool
[typographical] ~111-~111: To join two clauses or introduce examples, consider using an em dash.
Context: ...pports the following protocols: -tar- Tar files, potentially compressed (tar.g...(DASH_RULE)
[typographical] ~112-~112: To join two clauses or introduce examples, consider using an em dash.
Context: ...mpressed (tar.gz, tar.bz2, etc.) -zip- Zip files -http- HTTP URLs -https...(DASH_RULE)
[typographical] ~113-~113: To join two clauses or introduce examples, consider using an em dash.
Context: ....bz2, etc.) -zip- Zip files -http- HTTP URLs -https- HTTPS URLs -git...(DASH_RULE)
[typographical] ~114-~114: To join two clauses or introduce examples, consider using an em dash.
Context: ...Zip files -http- HTTP URLs -https- HTTPS URLs -git- Git repositories, w...(DASH_RULE)
[typographical] ~115-~115: To join two clauses or introduce examples, consider using an em dash.
Context: ...HTTP URLs -https- HTTPS URLs -git- Git repositories, which can be accessed ...(DASH_RULE)
[typographical] ~116-~116: To join two clauses or introduce examples, consider using an em dash.
Context: ... can be accessed via HTTPS or SSH -hg- Mercurial repositories, accessed via HTT...(DASH_RULE)
[typographical] ~117-~117: To join two clauses or introduce examples, consider using an em dash.
Context: ...ories, accessed via HTTP/S or SSH -s3- Amazon S3 bucket URLs -gcs- Google C...(DASH_RULE)
[typographical] ~118-~118: To join two clauses or introduce examples, consider using an em dash.
Context: ...H -s3- Amazon S3 bucket URLs -gcs- Google Cloud Storage URLs -oci- Open...(DASH_RULE)
[typographical] ~119-~119: To join two clauses or introduce examples, consider using an em dash.
Context: ...gcs- Google Cloud Storage URLs -oci` - Open Container Initiative (OCI) images -...(DASH_RULE)
[typographical] ~120-~120: To join two clauses or introduce examples, consider using an em dash.
Context: ...ontainer Initiative (OCI) images -scp- Secure Copy Protocol for SSH-based trans...(DASH_RULE)
[typographical] ~121-~121: To join two clauses or introduce examples, consider using an em dash.
Context: ...rotocol for SSH-based transfers -sftp- SSH File Transfer Protocol :::tip You c...(DASH_RULE)
129-152: Well-documented YQ expressions with practical examples!The section effectively demonstrates YQ usage with clear examples and helpful references.
153-172: Excellent cross-platform documentation!The section effectively covers file path handling across different operating systems with clear examples.
|
These changes were released in v1.153.0. |
what
!includeand!envwhy
The
!envYAML function is used to retrieve environment variables and assign them to the sections in Atmos stack manifestsThe
!includeYAML function allows downloading local or remote files from different sources, and assigning the file contents or individual values to the sections in Atmos stack manifestsdescription
!envAtmos YAML functionThe
!envAtmos YAML function is used to retrieve environment variables and assign them to the sections in Atmos stack manifests.If the function is called with one argument (the name of the environment variable), and the environment variable is
not present,
nullwill be assigned to the corresponding section in the Atmos manifest.If the function is called with two arguments (the name of the environment variable and the default value), and the
environment variable is not present, the default value will be assigned to the corresponding section in the Atmos manifest.
Examples:
Handling default values with spaces:
If you need to provide default values with spaces, enclose them in double quotes and use single quotes around the whole expression.
For example:
!includeAtmos YAML functionThe
!includeAtmos YAML function allows downloading local or remote files from different sources, and assigning the file contents or individual values to the sections in Atmos stack manifests.The
!includefunction can be called with either one or two parameters:Examples:
Description:
The YAML standard provides anchors and aliases, that allow you
to reuse and reference pieces of your YAML file, making it more efficient and reducing duplication.
Atmos supports YAML anchors and aliases, but the biggest limitation is that they are only available within the file in
which they are defined. You cannot reuse anchors across different files.
The
!includeAtmos YAML function overcomes this limitation by allowing you to include the content or individual valuesfrom different local and remote sources. The
!includefunction also provides the following features:Supports local files with absolute and relative paths
Supports the remote protocols provided by the
go-getterlibraryAllows you to use YQ expressions to query and filter the content of the files to retrieve individual values
Automatically detects the format of the files regardless of the file extensions. It supports files in JSON, YAML and HCL (
tfvars) formats, and automatically converts them into correct YAML structures (simple and complex types like maps and lists are supported). All other files are returned unchanged, allowing you, for example, to include text and Markdown files as strings in Atmos manifestsSupported File Protocols:
The
!includefunction supports the following local file paths:/Users/me/Documents/values.yaml)!includefunction is executed (e.g../values.yaml,../config/values.yaml)base_pathdefined inatmos.yamlCLI config file (e.g.stacks/catalog/vpc/defaults.yaml)To download remote files from different sources, the
!includefunction usesgo-getter(used by Terraform for downloading modules) and supports the following protocols:
tar- Tar files, potentially compressed (tar.gz, tar.bz2, etc.)zip- Zip fileshttp- HTTP URLshttps- HTTPS URLsgit- Git repositories, which can be accessed via HTTPS or SSHhg- Mercurial repositories, accessed via HTTP/S or SSHs3- Amazon S3 bucket URLsgcs- Google Cloud Storage URLsoci- Open Container Initiative (OCI) imagesscp- Secure Copy Protocol for SSH-based transferssftp- SSH File Transfer ProtocolUsing YQ Expressions to retrieve individual values from files:
To retrieve individual values from complex types such as maps and lists, or do any kind of filtering or querying,
you can utilize YQ expressions.
For example:
For more details, review the following docs:
Handling file paths and YQ expressions with spaces:
If you have spaces in the file names or the YQ expressions, enclose the file path and YQ expression in double quotes and
the whole expression in single quotes.
For example, on Windows:
On macOS and Linux:
Summary by CodeRabbit
Atmos v1.153.0 Release Notes
New Features
!envYAML function to retrieve environment variables in stack manifests.!includeYAML function to download and include local/remote files in stack manifests.processTagEnvandprocessTagIncludefunctions for enhanced YAML processing.!storeYAML function for reading values from a remote store in Atmos stack manifests.Improvements
Bug Fixes
Documentation