-
Notifications
You must be signed in to change notification settings - Fork 4.3k
Description
Describe the bug
When deploying a stack with a CloudWatch LogGroup subscription filter to a Kinesis Stream, the subscription will test that the subscription works and roll back if it doesn't.
https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-logs/lib/subscription-filter.ts
https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-logs-destinations/lib/kinesis.ts
The two files above together create a CfnSubscriptionFilter
that uses an iam.Role
with a policy (containing statements for this.stream.grantWrite(role)
and role.grantPassRole(role)
). However, the subscription filter does not have a dependency on the policy. While it depends on the Role and must wait for that to be deployed, it does not depend on the policy and so the role's policy and the subscription filter can be deployed out of order.
https://docs.aws.amazon.com/cdk/v2/guide/permissions.html#permissions_grants
Another case is when a service verifies that the role you pass to it has the right policies applied (a number of AWS services do this to make sure you didn't forget to set the policies). In those cases, the deployment may fail if the permissions are applied too late.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-dependson.html
When you have a single template that defines a service role, the role's policy (by using the AWS::IAM::Policy or AWS::IAM::ManagedPolicy resource), and a resource that uses the role, add a dependency so that the resource depends on the role's policy.
Expected Behavior
LogGroup, Role, Policy and SubscriptionFilter are deployed successfully.
Current Behavior
SubscriptionFilter is deployed before the Policy and fails with the error message
Could not deliver test message to specified Kinesis stream. Check if the given kinesis stream is in ACTIVE state.
The Kinesis Stream has already been created in another stack and is ACTIVE, so that's not the issue.
Reproduction Steps
Might not be consistently reproduceable. It depends on the order CloudFormation choses to deploy the resources, and the timing of the Policy deployment vs the SubscriptionFilter deployment. It works fine in all many of our stacks, but suddenly failed to work in a newly added (much smaller) stack with only an API Gateway API, an access control LogGroup and a SubscriptionFilter to a KinesisDestination.
Possible Solution
Quick and dirty workaround that doesn't use the CDK constructs:
const role = new Role(this, 'CloudWatchLogsCanPutRecords', {
assumedBy: new ServicePrincipal('logs.amazonaws.com'),
});
const writeGrant = someOtherStack.kinesisStream.grantWrite(role);
const passGrant = role.grantPassRole(role);
const subscriptionFilter = new CfnSubscriptionFilter(this, 'Resource', {
logGroupName: someLogGroup.logGroupName,
destinationArn: someOtherStack.kinesisStream.streamArn,
roleArn: role.roleArn,
filterPattern: FilterPattern.allEvents().logPatternString,
});
// the fix (will emit a single DependsOn for SubscriptionFilter in CF template):
writeGrant.applyBefore(subscriptionFilter);
passGrant.applyBefore(subscriptionFilter);
Additional Information/Context
No response
CDK CLI Version
1.158.0 (build ab28878)
Framework Version
No response
Node.js Version
v16.14.0
OS
Windows 10
Language
Typescript
Language Version
No response
Other information
No response