Aws secretsmanager additions #6381

mpechner-akasa · 2024-11-29T18:28:49Z

This adds the ability to specify a secretKey in the awsSecretManager TriggerAuthentication. This will allow parsing of secrets that contain Key/Value pairs (returned in JSON format).

Resubmission of #6031

Provide a description of what has been changed

Checklist

When introducing a new scaler, I agree with the scaling governance policy
I have verified that my change is according to the deprecations & breaking changes policy
Tests have been added
Changelog has been updated and is aligned with our changelog requirements
A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
A PR is opened to update the documentation on (repo) (if applicable) Documented adding secretKey parameter to access aws secret manager keda-docs#1508
Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #5940

Relates to #

Signed-off-by: Nick Richardson <[email protected]>

Signed-off-by: michael pechner <[email protected]>

zroubalik

Looking good, thanks. We should also update docs

Doc for issue: kedacore/keda#5940 PR: kedacore/keda#6381 Signed-off-by: michael pechner <[email protected]>

mpechner-akasa · 2024-12-05T23:50:53Z

#5940

docs. kedacore/keda-docs#1508

JorTurFer

Really nice job! some comments inline

tests/secret-providers/aws_secretmanager/aws_secretmanager_test.go

tests/secret-providers/aws_secretmanager_pod_identity/aws_secretmanager_pod_identity_test.go

Signed-off-by: michael pechner <[email protected]>

…-akasa/keda into aws-secretsmanager-additions

…ect. Will remove REMOVETestAwsSecretManagerJSONFormat and change aws_secret_manager_pod_identity.go once I have changed this file as expected. Signed-off-by: michael pechner <[email protected]>

// Local imports . "github.com/kedacore/keda/v2/tests/helper" Signed-off-by: michael pechner <[email protected]>

Signed-off-by: michael pechner <[email protected]>

…rue and false Signed-off-by: michael pechner <[email protected]>

Signed-off-by: michael pechner <[email protected]>

igoland linter keeps removng it. Signed-off-by: michael pechner <[email protected]>

Signed-off-by: michael pechner <[email protected]>

JorTurFer

Nice job! Thanks a lot ❤️

JorTurFer · 2025-01-17T12:52:31Z

/run-e2e aws
Update: You can check the progress here

JorTurFer · 2025-01-17T18:01:21Z

e2e tests are failing because the secret has the same name in both cases. This makes that one case passes and the other one fails because the e2e test is trying to create a secret with the same name and the secret is in deletion process

You can't create this secret because a secret with this name is already scheduled for deletion

You need to update the value of AwsCredentialsSecretName to include a different value per test case (as well as the random number to not conflict between tests). e.g: You can move the value calculation to the function getTemplateData and append a suffix with the test case

…e the secret. Instead of tryinfg to further randomize the secretname, just let the code do what it should. Added a poll to wait on the secret to be removed. Hoping 2 minniutes is more than enough. Signed-off-by: michael pechner <[email protected]>

…e the secret. Instead of trying to further randomize the secretname, just let the code do what it should. Added a poll to wait on the secret to be removed. Should happen within a few seconds. But we are talking AWS. 5 minutes really should be more then enough. Signed-off-by: michael pechner <[email protected]>

…-akasa/keda into aws-secretsmanager-additions

Signed-off-by: michael pechner <[email protected]>

tests/secret-providers/aws_secretmanager_pod_identity/aws_secretmanager_pod_identity_test.go

made sure we set a new secret name for each run Signed-off-by: michael pechner <[email protected]>

Signed-off-by: Michael D Pechner - Akasa <[email protected]>

JorTurFer

Really great job! 2 small comments inline and we are almost there!

JorTurFer · 2025-02-03T21:57:12Z

tests/secret-providers/aws_secretmanager/aws_secretmanager_test.go

@@ -70,6 +76,7 @@ type templateData struct {
 	SecretManagerSecretName          string
 	AwsAccessKeyID                   string
 	AwsSecretAccessKey               string
+	useJSONSecretFormat              bool


I think that this isn't used anymore

JorTurFer · 2025-02-03T21:57:42Z

tests/secret-providers/aws_secretmanager_pod_identity/aws_secretmanager_pod_identity_test.go

@@ -70,6 +76,7 @@ type templateData struct {
 	SecretManagerSecretName          string
 	AwsAccessKeyID                   string
 	AwsSecretAccessKey               string
+	useJSONSecretFormat              bool


I think that this isn't used anymore

JorTurFer · 2025-02-03T22:00:08Z

/run-e2e aws
Update: You can check the progress here

Signed-off-by: michael pechner <[email protected]>

JorTurFer · 2025-02-03T22:23:51Z

/run-e2e aws
Update: You can check the progress here

JorTurFer · 2025-02-03T23:06:13Z

tests/secret-providers/aws_secretmanager/aws_secretmanager_test.go

@@ -315,6 +375,7 @@ var data = templateData{
 	AwsSecretAccessKey:               base64.StdEncoding.EncodeToString([]byte(awsSecretAccessKey)),
 	AwsRegion:                        awsRegion,
 	AwsCredentialsSecretName:         awsCredentialsSecretName,
+	useJSONSecretFormat:              false,


We need to delete the assignment from here too

JorTurFer · 2025-02-03T23:06:19Z

tests/secret-providers/aws_secretmanager_pod_identity/aws_secretmanager_pod_identity_test.go

@@ -305,6 +364,7 @@ var data = templateData{
 	AwsSecretAccessKey:               base64.StdEncoding.EncodeToString([]byte(awsSecretAccessKey)),
 	AwsRegion:                        awsRegion,
 	AwsCredentialsSecretName:         awsCredentialsSecretName,
+	useJSONSecretFormat:              false,


We need to delete the assignment from here too

Signed-off-by: michael pechner <[email protected]>

…-akasa/keda into aws-secretsmanager-additions

nrichardson-akasa and others added 8 commits August 2, 2024 12:52

Add SecretKey to AWS SecretsManager TriggerAuthentication spec

655471b

Signed-off-by: Nick Richardson <[email protected]>

Update CHANGELOG

8025fd3

Signed-off-by: Nick Richardson <[email protected]>

Fix linting issue

fd88dd7

Signed-off-by: Nick Richardson <[email protected]>

Update E2E tests

124e48a

Signed-off-by: Nick Richardson <[email protected]>

Update E2E tests

b9951d1

Signed-off-by: Nick Richardson <[email protected]>

Merge branch 'main' into aws-secretsmanager-additions

b1421f3

Update E2E tests

dabf94c

Signed-off-by: Nick Richardson <[email protected]>

Merge branch 'main' into aws-secretsmanager-additions

23175d8

mpechner-akasa requested a review from a team as a code owner November 29, 2024 18:28

mpechner-akasa force-pushed the aws-secretsmanager-additions branch from a4701ef to 23175d8 Compare November 29, 2024 20:09

mpechner-akasa added 5 commits November 29, 2024 12:11

fixed mixedxe tab/space issue

f1717da

Signed-off-by: michael pechner <[email protected]>

reverted back to exportloopref

f113aac

Signed-off-by: michael pechner <[email protected]>

Merge branch 'main' into aws-secretsmanager-additions

2a7211b

Merge branch 'main' into aws-secretsmanager-additions

7f81639

Merge branch 'main' into aws-secretsmanager-additions

b3272b9

zroubalik reviewed Dec 4, 2024

View reviewed changes

mpechner-akasa added a commit to mpechner-akasa/keda-docs that referenced this pull request Dec 5, 2024

Documented adding secretKey parameter to access aws secret manager

2c33b1b

Doc for issue: kedacore/keda#5940 PR: kedacore/keda#6381 Signed-off-by: michael pechner <[email protected]>

mpechner-akasa mentioned this pull request Dec 5, 2024

Documented adding secretKey parameter to access aws secret manager kedacore/keda-docs#1508

Open

1 task

Merge branch 'main' into aws-secretsmanager-additions

a6b4067

JorTurFer reviewed Dec 15, 2024

View reviewed changes

mpechner-akasa force-pushed the aws-secretsmanager-additions branch 2 times, most recently from 36f2dcc to a6b4067 Compare December 16, 2024 22:38

mpechner-akasa added 7 commits December 16, 2024 14:39

remoace tab with spaces

1f34e8f

Signed-off-by: michael pechner <[email protected]>

Merge branch 'main' into aws-secretsmanager-additions

decfda0

Needed the hlper added in here

3e92f5c

Signed-off-by: michael pechner <[email protected]>

Merge branch 'aws-secretsmanager-additions' of github.com:nrichardson…

45e365c

…-akasa/keda into aws-secretsmanager-additions

only changed awssecreetmanager_test.go to see if my approach is corr…

e6ec7c4

…ect. Will remove REMOVETestAwsSecretManagerJSONFormat and change aws_secret_manager_pod_identity.go once I have changed this file as expected. Signed-off-by: michael pechner <[email protected]>

nd keeps removing

a0a839a

// Local imports . "github.com/kedacore/keda/v2/tests/helper" Signed-off-by: michael pechner <[email protected]>

Not sure what the correct way to import the local helper is.

f294e18

Signed-off-by: michael pechner <[email protected]>

mpechner-akasa added 6 commits January 12, 2025 18:36

removed duplicated function. Tests not run with useJSONSecretFormat t…

9c780ef

…rue and false Signed-off-by: michael pechner <[email protected]>

Merge branch 'main' into aws-secretsmanager-additions

256e26f

no excuse

d0b9d05

Signed-off-by: michael pechner <[email protected]>

fixed . "github.com/kedacore/keda/v2/tests/helper"

2b7c7f9

igoland linter keeps removng it. Signed-off-by: michael pechner <[email protected]>

go fmt needs for 1 empty line

784bdc2

Signed-off-by: michael pechner <[email protected]>

Merge branch 'main' into aws-secretsmanager-additions

8f2419a

JorTurFer approved these changes Jan 17, 2025

View reviewed changes

Merge branch 'main' into aws-secretsmanager-additions

f43c40c

mpechner-akasa added 9 commits January 17, 2025 16:31

Merge branch 'aws-secretsmanager-additions' of github.com:nrichardson…

210fe63

…-akasa/keda into aws-secretsmanager-additions

forgot the import

5d0cf03

Signed-off-by: michael pechner <[email protected]>

format

982c17f

Signed-off-by: michael pechner <[email protected]>

Sorry. did not realize the needed imcludes for the wait code.

1b9c9d9

Signed-off-by: michael pechner <[email protected]>

compiles clean. found types.ResourceNotFoundException

f78158e

Signed-off-by: michael pechner <[email protected]>

swapped wait.PollImmediate for wait.PollUntilContextTimeout

ff4e466

Signed-off-by: michael pechner <[email protected]>

forgot to fmt 1

89335bb

Signed-off-by: michael pechner <[email protected]>

JorTurFer reviewed Jan 18, 2025

View reviewed changes

tests/secret-providers/aws_secretmanager_pod_identity/aws_secretmanager_pod_identity_test.go Outdated Show resolved Hide resolved

mpechner-akasa added 2 commits January 30, 2025 17:03

as requested. removed wait on secret delete.

f069c35

made sure we set a new secret name for each run Signed-off-by: michael pechner <[email protected]>

Merge branch 'main' into aws-secretsmanager-additions

b35c611

Signed-off-by: Michael D Pechner - Akasa <[email protected]>

JorTurFer reviewed Feb 3, 2025

View reviewed changes

removed useJSONSecretFormat from test struct.

3f8b6a1

Signed-off-by: michael pechner <[email protected]>

JorTurFer reviewed Feb 3, 2025

View reviewed changes

mpechner-akasa added 3 commits February 3, 2025 15:58

Merge branch 'main' into aws-secretsmanager-additions

b45a2cd

missed a spot where useJSONSecretFormat was set

d5446a4

Signed-off-by: michael pechner <[email protected]>

Merge branch 'aws-secretsmanager-additions' of github.com:nrichardson…

df47d31

…-akasa/keda into aws-secretsmanager-additions

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Aws secretsmanager additions #6381

Aws secretsmanager additions #6381

mpechner-akasa commented Nov 29, 2024 •

edited by JorTurFer

Loading

zroubalik left a comment

mpechner-akasa commented Dec 5, 2024

JorTurFer left a comment

JorTurFer left a comment

JorTurFer commented Jan 17, 2025 •

edited by github-actions bot

Loading

JorTurFer commented Jan 17, 2025 •

edited

Loading

JorTurFer left a comment

JorTurFer Feb 3, 2025

JorTurFer Feb 3, 2025

JorTurFer commented Feb 3, 2025 •

edited by github-actions bot

Loading

JorTurFer commented Feb 3, 2025 •

edited by github-actions bot

Loading

JorTurFer Feb 3, 2025

JorTurFer Feb 3, 2025

Aws secretsmanager additions #6381

Are you sure you want to change the base?

Aws secretsmanager additions #6381

Conversation

mpechner-akasa commented Nov 29, 2024 • edited by JorTurFer Loading

Checklist

zroubalik left a comment

Choose a reason for hiding this comment

mpechner-akasa commented Dec 5, 2024

JorTurFer left a comment

Choose a reason for hiding this comment

JorTurFer left a comment

Choose a reason for hiding this comment

JorTurFer commented Jan 17, 2025 • edited by github-actions bot Loading

JorTurFer commented Jan 17, 2025 • edited Loading

JorTurFer left a comment

Choose a reason for hiding this comment

JorTurFer Feb 3, 2025

Choose a reason for hiding this comment

JorTurFer Feb 3, 2025

Choose a reason for hiding this comment

JorTurFer commented Feb 3, 2025 • edited by github-actions bot Loading

JorTurFer commented Feb 3, 2025 • edited by github-actions bot Loading

JorTurFer Feb 3, 2025

Choose a reason for hiding this comment

JorTurFer Feb 3, 2025

Choose a reason for hiding this comment

mpechner-akasa commented Nov 29, 2024 •

edited by JorTurFer

Loading

JorTurFer commented Jan 17, 2025 •

edited by github-actions bot

Loading

JorTurFer commented Jan 17, 2025 •

edited

Loading

JorTurFer commented Feb 3, 2025 •

edited by github-actions bot

Loading

JorTurFer commented Feb 3, 2025 •

edited by github-actions bot

Loading