Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove template_variables from cirrus workflows #92

Merged
merged 1 commit into from
Feb 5, 2025

Conversation

cvangerpen
Copy link
Contributor

@cvangerpen cvangerpen commented Feb 4, 2025

This commit reduces the cirrus workflow module's complexity by relying on the user to simply provide Cirrus task output attribute lookups directly in their workflow state machine definition rather than using arbitrary variable names with associated entries under a workflow config 'template_variables' attribute.

The rendered JSON is now parsed via regexes to determine which resource ARNs the workflow IAM role needs permissions to execute. This also removes the need for the optional 'non_cirrus_lambda_arns' workflow config setting.

The workflow IAM role's lambda and batch policy documents were merged into one with conditional statements based on whether lambda or batch resources are used by the workflow's tasks.

Related issue(s)

  • N/A

Proposed Changes

  1. Remove template_variables and non_cirrus_lambda_arns from cirrus workflow config objects
  2. Rename template_filepath to state_machine_filepath in cirrus workflow config objects (for clarity)
  3. Remove the overhead that was necessary for handling template_variables being passed around between modules. Instead, the workflow state machine JSON is templated and then parsed for any lambda functions, batch jobs, or batch queues that the workflow IAM role will need to execute.
  4. Only create one workflow IAM role policy with conditional lambda- and batch-related permission statement blocks.

Testing

This change was validated by the following observations:

  1. Deployed this feature to an existing FilmDrop deployment. After updating the workflow configuration objects and the state machine JSON templates, an attempted Terraform deployment will produce the same workflows (albeit with only one workflow IAM role policy now instead of one or two).

Checklist

  • I have deployed and validated this change
  • Changelog
    • I have added my changes to the changelog
    • No changelog entry is necessary
  • README migration
    • I have added any migration steps to the Readme
    • No migration is necessary

This commit reduces the cirrus workflow module's complexity by relying
on the user to simply provide Cirrus task output attribute lookups
directly in their workflow state machine definition rather than using
arbitrary variable names with associated entries under a workflow config
'template_variables' attribute.

The rendered JSON is now parsed via regexes to determine which resource
ARNs the workflow IAM role needs permissions to execute. This also
removes the need for the optional 'non_cirrus_lambda_arns' workflow
config setting.

The workflow IAM role's lambda and batch policy documents were merged
into one with conditional statements based on whether lambda or batch
resources are used by the workflow's tasks.
@cvangerpen cvangerpen changed the title Removed template_variables from cirrus workflows Remove template_variables from cirrus workflows Feb 5, 2025
@cvangerpen cvangerpen marked this pull request as ready for review February 5, 2025 19:45
@cvangerpen cvangerpen merged commit 587504f into main Feb 5, 2025
4 checks passed
@cvangerpen cvangerpen deleted the cvg/reduce-cirrus-workflow-config-complexity branch February 5, 2025 21:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants