Log Analytics Pipeline: Unable to deploy cross account lambda pipelines

[megasarl]

Describe the bug

All attemtps to setup a Log Pipeline between two AWS accounts in EU-West-1 fails the cloud formation creation with message "(AccessDeniedException) when calling the PutSubscriptionFilter operation".

Expected Behavior

The log pipeline should setup all resources including the permissions and reach a success state in the Centralized Logging with OpenSearch UI

Current Behavior

When using the UI to deploy a cross account Lambda collection the stack ends up in a rollback state with the error being:

Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (AccessDeniedException) when calling the PutSubscriptionFilter operation: User with accountId: #memberAccountID# is not authorized to perform PutSubscriptionFilter on resources arn:aws:logs:eu-west-1:#AccountID#:destination:CL-SvcPipe-a6308ec8-CL-SvcPipe-a6308ec8-CWtoFirehosetoS3StackDeliverySt-s9cKxt4GDgIW. Logs: /aws/lambda/CL-SvcPipe-a6308ec8-CWtoFirehosetoS3StackcwSubFilt-Y6nTxnoIOTsb at invokeUserFunction (/var/task/framework.js:2:6) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async onEvent (/var/task/framework.js:1:369) at async Runtime.handler (/var/task/cfn-response.js:1:1573) (RequestId: 853b8fcd-641f-41c2-8ce0-14b8c8f26474)

Following the troubleshooting guide for an earlier version for the exact error message indicates the correct values
https://docs.aws.amazon.com/solutions/latest/centralized-logging-on-aws/troubleshooting.html

aws logs describe-destinations --region eu-west-1 { "destinations": [ { "destinationName": "CL-SvcPipe-a6308ec8-CL-SvcPipe-a6308ec8-CWtoFirehosetoS3StackDeliverySt-s9cKxt4GDgIW", "targetArn": "arn:aws:firehose:eu-west-1:#AccountID#:deliverystream/CL-SvcPipe-a6308ec8-CWtoFirehosetoS3StackDeliverySt-s9cKxt4GDgIW", "roleArn": "arn:aws:iam::#AccountID#:role/CL-SvcPipe-a6308ec8-CWtoFirehosetoS3StackCWDestinat-Zf9fjlcsykyR", "accessPolicy": "{\"Version\": \"2012-10-17\", \"Statement\": [{\"Sid\": \"\", \"Effect\": \"Allow\", \"Principal\": {\"AWS\": \"#memberAccountID# \"}, \"Action\": \"logs:PutSubscriptionFilter\", \"Resource\": \"arn:aws:logs:eu-west-1:#AccountID#:destination:CL-SvcPipe-a6308ec8-CL-SvcPipe-a6308ec8-CWtoFirehosetoS3StackDeliverySt-s9cKxt4GDgIW\"}]}", "arn": "arn:aws:logs:eu-west-1:#AccountID#:destination:CL-SvcPipe-a6308ec8-CL-SvcPipe-a6308ec8-CWtoFirehosetoS3StackDeliverySt-s9cKxt4GDgIW", "creationTime": 1701945178823 } ] }
(redacted the account IDs in the logs above with #memberAccountID# , #AccountID#)

Reproduction Steps

Setup "Centralized Logging with OpenSearch" in one account togeather with Opensearch and connect a member account via the UI.
Try to create a aws-service-log pipeline for a lambda in the remote account and it should result in an error

Possible Solution

No response

Additional Information/Context

a similar earlier setup has been deployed on two other account with 2.0.0 without the issue

Solution Version

2.1.1

AWS Region. e.g., us-east-1

eu-west-1

Other information

No response

[wchaws]

A quick workaround:

1. Go to member account
2. Find a role name which looks like CrossAccountRoleFACE29D1
3. Add an extra resource id ("arn:aws:logs:*:<parent-account-id>:destination:*")as follow:
   

It seems that AWS services get updated. The above policy resource is not needed before.

[megasarl]

Workaround confirmed, seams to work as expected when adding this resource to the IAM Role

[StanForever]

Can confirm. The workaround is working.

[debashish1976]

Is there any update to this issue? I faced this issue multiple times and had resolve it with help of AWS Support. If we can update the CloudFormation template to be run on member account, then it will resolve it once and for all.

[JoeShi]

Is there any update to this issue? I faced this issue multiple times and had resolve it with help of AWS Support. If we can update the CloudFormation template to be run on member account, then it will resolve it once and for all.

OK. We will release a patch version as soon as possbile.