Skip to content

aws-logs-destinations: missing dependency to Policy created by KinesisDestination #21827

@HansFalkenberg-Visma

Description

@HansFalkenberg-Visma

Describe the bug

When deploying a stack with a CloudWatch LogGroup subscription filter to a Kinesis Stream, the subscription will test that the subscription works and roll back if it doesn't.

https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-logs/lib/subscription-filter.ts
https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-logs-destinations/lib/kinesis.ts

The two files above together create a CfnSubscriptionFilter that uses an iam.Role with a policy (containing statements for this.stream.grantWrite(role) and role.grantPassRole(role)). However, the subscription filter does not have a dependency on the policy. While it depends on the Role and must wait for that to be deployed, it does not depend on the policy and so the role's policy and the subscription filter can be deployed out of order.

https://docs.aws.amazon.com/cdk/v2/guide/permissions.html#permissions_grants

Another case is when a service verifies that the role you pass to it has the right policies applied (a number of AWS services do this to make sure you didn't forget to set the policies). In those cases, the deployment may fail if the permissions are applied too late.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-dependson.html

When you have a single template that defines a service role, the role's policy (by using the AWS::IAM::Policy or AWS::IAM::ManagedPolicy resource), and a resource that uses the role, add a dependency so that the resource depends on the role's policy.

Expected Behavior

LogGroup, Role, Policy and SubscriptionFilter are deployed successfully.

Current Behavior

SubscriptionFilter is deployed before the Policy and fails with the error message

Could not deliver test message to specified Kinesis stream. Check if the given kinesis stream is in ACTIVE state.

The Kinesis Stream has already been created in another stack and is ACTIVE, so that's not the issue.

Reproduction Steps

Might not be consistently reproduceable. It depends on the order CloudFormation choses to deploy the resources, and the timing of the Policy deployment vs the SubscriptionFilter deployment. It works fine in all many of our stacks, but suddenly failed to work in a newly added (much smaller) stack with only an API Gateway API, an access control LogGroup and a SubscriptionFilter to a KinesisDestination.

Possible Solution

Quick and dirty workaround that doesn't use the CDK constructs:

const role = new Role(this, 'CloudWatchLogsCanPutRecords', {
  assumedBy: new ServicePrincipal('logs.amazonaws.com'),
});
const writeGrant = someOtherStack.kinesisStream.grantWrite(role);
const passGrant = role.grantPassRole(role);
const subscriptionFilter = new CfnSubscriptionFilter(this, 'Resource', {
  logGroupName: someLogGroup.logGroupName,
  destinationArn: someOtherStack.kinesisStream.streamArn,
  roleArn: role.roleArn,
  filterPattern: FilterPattern.allEvents().logPatternString,
});
// the fix (will emit a single DependsOn for SubscriptionFilter in CF template):
writeGrant.applyBefore(subscriptionFilter);
passGrant.applyBefore(subscriptionFilter);

Additional Information/Context

No response

CDK CLI Version

1.158.0 (build ab28878)

Framework Version

No response

Node.js Version

v16.14.0

OS

Windows 10

Language

Typescript

Language Version

No response

Other information

No response

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions