feat: Default dimensions from powertools instead of emf library (#317)

* feat: disable default emf lib dimensions and allow to override default dimensions

* docs: clarify docs about default dimensions

* feat: remove minimum 1 dimension validation

since EMF supports no dimension as well

* docs: improve installation steps for sqs large message

* docs: clarify java11 aspect plugin issue for each module

Co-authored-by: Pankaj Agrawal <[email protected]>

Author: pankajagrawal16

docs/core/metrics.md

@@ -108,7 +108,6 @@ if no metrics are provided no exception will be raised. If metrics are provided,
 not met, `ValidationException` exception will be raised.
 
 !!! tip "Metric validation"
-    * Minimum of 1 dimension
     * Maximum of 9 dimensions
 
 If you want to ensure that at least one metric is emitted, you can pass `raiseOnEmptyMetrics = true` to the **@Metrics** annotation:
@@ -183,6 +182,35 @@ You can use `putMetadata` for advanced use cases, where you want to metadata as
 
 This will be available in CloudWatch Logs to ease operations on high cardinal data.
 
+## Overriding default dimension set
+
+By default, all metrics emitted via module captures `Service` as one of the default dimension. This is either specified via
+`POWERTOOLS_SERVICE_NAME` environment variable or via `service` attribute on `Metrics` annotation. If you wish to override the default 
+Dimension, it can be done via `#!java MetricsUtils.defaultDimensionSet()`.
+
+=== "App.java"
+
+    ```java hl_lines="8 9 10"
+    import software.amazon.lambda.powertools.metrics.Metrics;
+    import static software.amazon.lambda.powertools.metrics.MetricsUtils;
+    
+    public class App implements RequestHandler<Object, Object> {
+    
+        MetricsLogger metricsLogger = MetricsUtils.metricsLogger();
+        
+        static {
+            MetricsUtils.defaultDimensionSet(DimensionSet.of("CustomDimension", "booking"));
+        }
+    
+        @Override
+        @Metrics(namespace = "ExampleApplication", service = "booking")
+        public Object handleRequest(Object input, Context context) {
+            ...
+            MetricsUtils.withSingleMetric("Metric2", 1, Unit.COUNT, log -> {});
+        }
+    }
+    ```
+
 ## Creating a metric with a different dimension
 
 CloudWatch EMF uses the same dimensions across all your metrics. Use `withSingleMetric` if you have a metric that should have different dimensions.

docs/index.md

@@ -36,6 +36,9 @@ You can use [SAM](https://aws.amazon.com/serverless/sam/) to quickly setup a ser
 
 For more information about the project and available options refer to this [repository](https://github.com/aws-samples/cookiecutter-aws-sam-powertools-java/blob/main/README.md)
 
+!!! note "Using Java 9 or later?"
+    If you are working with lambda function on runtime **Java 9 or later**, please refer **[issue](https://github.com/awslabs/aws-lambda-powertools-java/issues/50)** for a workaround.
+
 === "Maven"
 
     ```xml hl_lines="3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55" 
@@ -98,7 +101,6 @@ For more information about the project and available options refer to this [repo
         </plugins>
     </build>
     ```
-    **Note:** If you are working with lambda function on runtime post java8, please refer [issue](https://github.com/awslabs/aws-lambda-powertools-java/issues/50) for workaround.
 
 === "Gradle"
 

docs/utilities/batch.md

@@ -25,6 +25,9 @@ are returned to the queue.
 
 To install this utility, add the following dependency to your project.
 
+!!! note "Using Java 9 or later?"
+    If you are working with lambda function on runtime **Java 9 or later**, please refer **[issue](https://github.com/awslabs/aws-lambda-powertools-java/issues/50)** for a workaround.
+
 === "Maven"
     ```xml hl_lines="3 4 5 6 7 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36"
     <dependencies>

docs/utilities/sqs_large_message_handling.md

@@ -33,23 +33,21 @@ This utility is compatible with versions *[1.1.0+](https://github.com/awslabs/am
 
 To install this utility, add the following dependency to your project.
 
-=== "Maven"
-    ```xml
-    <dependency>
-        <groupId>software.amazon.lambda</groupId>
-        <artifactId>powertools-sqs</artifactId>
-        <version>1.3.0</version>
-    </dependency>
-    ```
+!!! note "Using Java 9 or later?"
+If you are working with lambda function on runtime **Java 9 or later**, please refer **[issue](https://github.com/awslabs/aws-lambda-powertools-java/issues/50)** for a workaround.
 
-=== "Maven Configuration"
-
-    Configure the aspectj-maven-plugin to compile-time weave (CTW) the
-    aws-lambda-powertools-java aspects into your project. You may already have this
-    plugin in your pom. In that case add the dependency to the `aspectLibraries`
-    section.
-
-    ```xml hl_lines="13 14 15 16"
+=== "Maven"
+    ```xml hl_lines="3 4 5 6 7 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36"
+    <dependencies>
+        ...
+        <dependency>
+            <groupId>software.amazon.lambda</groupId>
+            <artifactId>powertools-sqs</artifactId>
+            <version>1.3.0</version>
+        </dependency>
+        ...
+    </dependencies>
+    <!-- configure the aspectj-maven-plugin to compile-time weave (CTW) the aws-lambda-powertools-java aspects into your project -->
     <build>
         <plugins>
             ...

docs/utilities/validation.md

@@ -15,23 +15,21 @@ This utility provides JSON Schema validation for payloads held within events and
 
 To install this utility, add the following dependency to your project.
 
+!!! note "Using Java 9 or later?"
+    If you are working with lambda function on runtime **Java 9 or later**, please refer **[issue](https://github.com/awslabs/aws-lambda-powertools-java/issues/50)** for a workaround.
+
 === "Maven"
-    ```xml
+    ```xml hl_lines="3 4 5 6 7 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36"
+    <dependencies>
+    ...
     <dependency>
         <groupId>com.amazonaws</groupId>
         <artifactId>powertools-validation</artifactId>
         <version>1.3.0</version>
     </dependency>
-    ```
-
-=== "Maven Configuration"
-
-    Configure the aspectj-maven-plugin to compile-time weave (CTW) the
-    aws-lambda-powertools-java aspects into your project. You may already have this
-    plugin in your pom. In that case add the dependency to the `aspectLibraries`
-    section.
-
-    ```xml hl_lines="13 14 15 16"
+    ...
+    </dependencies>
+    <!-- configure the aspectj-maven-plugin to compile-time weave (CTW) the aws-lambda-powertools-java aspects into your project -->
     <build>
         <plugins>
             ...

powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/MetricsUtils.java

@@ -4,11 +4,14 @@
 import java.util.function.Consumer;
 
 import software.amazon.cloudwatchlogs.emf.config.SystemWrapper;
+import software.amazon.cloudwatchlogs.emf.environment.EnvironmentProvider;
 import software.amazon.cloudwatchlogs.emf.logger.MetricsLogger;
+import software.amazon.cloudwatchlogs.emf.model.DimensionSet;
 import software.amazon.cloudwatchlogs.emf.model.MetricsContext;
 import software.amazon.cloudwatchlogs.emf.model.MetricsLoggerHelper;
 import software.amazon.cloudwatchlogs.emf.model.Unit;
 
+import static java.util.Objects.requireNonNull;
 import static java.util.Optional.ofNullable;
 import static software.amazon.lambda.powertools.core.internal.LambdaHandlerProcessor.getXrayTraceId;
 import static software.amazon.lambda.powertools.metrics.internal.LambdaMetricsAspect.REQUEST_ID_PROPERTY;
@@ -22,6 +25,7 @@
  */
 public final class MetricsUtils {
     private static final MetricsLogger metricsLogger = new MetricsLogger();
+    private static DimensionSet defaultDimensionSet;
 
     private MetricsUtils() {
     }
@@ -35,11 +39,22 @@ public static MetricsLogger metricsLogger() {
         return metricsLogger;
     }
 
+    /**
+     * Configure default dimension to be used by logger.
+     * By default, @{@link Metrics} annotation captures configured service as a dimension <i>Service</i>
+     * @param dimensionSet Default value of dimension set for logger
+     */
+    public static void defaultDimensionSet(final DimensionSet dimensionSet) {
+        requireNonNull(dimensionSet, "Null dimension set not allowed");
+        MetricsUtils.defaultDimensionSet = dimensionSet;
+    }
+
+
     /**
      * Add and immediately flush a single metric. It will use the default namespace
      * specified either on {@link Metrics} annotation or via POWERTOOLS_METRICS_NAMESPACE env var.
-     * It by default captures AwsRequestId as property if used together with {@link Metrics} annotation. It will also 
-     * capture XrayTraceId as property if tracing is enabled.
+     * It by default captures function_request_id as property if used together with {@link Metrics} annotation. It will also
+     * capture xray_trace_id as property if tracing is enabled.
      *
      * @param name   the name of the metric
      * @param value  the value of the metric
@@ -50,7 +65,8 @@ public static void withSingleMetric(final String name,
                                         final double value,
                                         final Unit unit,
                                         final Consumer<MetricsLogger> logger) {
-        MetricsLogger metricsLogger = new MetricsLogger();
+        MetricsLogger metricsLogger = logger();
+
         try {
             metricsLogger.setNamespace(defaultNameSpace());
             metricsLogger.putMetric(name, value, unit);
@@ -63,8 +79,8 @@ public static void withSingleMetric(final String name,
 
     /**
      * Add and immediately flush a single metric.
-     * It by default captures AwsRequestId as property if used together with {@link Metrics} annotation. It will also 
-     * capture XrayTraceId as property if tracing is enabled.
+     * It by default captures function_request_id as property if used together with {@link Metrics} annotation. It will also
+     * capture xray_trace_id as property if tracing is enabled.
      *
      * @param name the name of the metric
      * @param value the value of the metric
@@ -77,7 +93,8 @@ public static void withSingleMetric(final String name,
                                         final Unit unit,
                                         final String namespace,
                                         final Consumer<MetricsLogger> logger) {
-        MetricsLogger metricsLogger = new MetricsLogger();
+        MetricsLogger metricsLogger = logger();
+
         try {
             metricsLogger.setNamespace(namespace);
             metricsLogger.putMetric(name, value, unit);
@@ -88,6 +105,14 @@ public static void withSingleMetric(final String name,
         }
     }
 
+    public static DimensionSet defaultDimensionSet() {
+        return defaultDimensionSet;
+    }
+
+    public static boolean hasDefaultDimension() {
+        return defaultDimensionSet.getDimensionKeys().size() > 0;
+    }
+
     private static void captureRequestAndTraceId(MetricsLogger metricsLogger) {
         awsRequestId().
                 ifPresent(requestId -> metricsLogger.putProperty(REQUEST_ID_PROPERTY, requestId));
@@ -107,4 +132,14 @@ private static Optional<String> awsRequestId() {
         return ofNullable(context.getProperty(REQUEST_ID_PROPERTY))
                 .map(Object::toString);
     }
+
+    private static MetricsLogger logger() {
+        MetricsContext metricsContext = new MetricsContext();
+
+        if (hasDefaultDimension()) {
+            metricsContext.setDefaultDimensions(defaultDimensionSet());
+        }
+
+        return new MetricsLogger(new EnvironmentProvider(), metricsContext);
+    }
 }

powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/internal/LambdaMetricsAspect.java

@@ -23,6 +23,8 @@
 import static software.amazon.lambda.powertools.core.internal.LambdaHandlerProcessor.placedOnRequestHandler;
 import static software.amazon.lambda.powertools.core.internal.LambdaHandlerProcessor.placedOnStreamHandler;
 import static software.amazon.lambda.powertools.core.internal.LambdaHandlerProcessor.serviceName;
+import static software.amazon.lambda.powertools.metrics.MetricsUtils.defaultDimensionSet;
+import static software.amazon.lambda.powertools.metrics.MetricsUtils.hasDefaultDimension;
 import static software.amazon.lambda.powertools.metrics.MetricsUtils.metricsLogger;
 
 @Aspect
@@ -47,8 +49,9 @@ public Object around(ProceedingJoinPoint pjp,
 
             MetricsLogger logger = metricsLogger();
 
-            logger.setNamespace(namespace(metrics))
-                    .putDimensions(DimensionSet.of("Service", service(metrics)));
+            refreshMetricsContext(metrics);
+
+            logger.setNamespace(namespace(metrics));
 
             extractContext(pjp).ifPresent((context) -> {
                 coldStartSingleMetricIfApplicable(context.getAwsRequestId(), context.getFunctionName(), metrics);
@@ -65,7 +68,7 @@ public Object around(ProceedingJoinPoint pjp,
                 coldStartDone();
                 validateMetricsAndRefreshOnFailure(metrics);
                 logger.flush();
-                refreshMetricsContext();
+                refreshMetricsContext(metrics);
             }
         }
 
@@ -92,8 +95,8 @@ private void validateBeforeFlushingMetrics(Metrics metrics) {
             throw new ValidationException("No metrics captured, at least one metrics must be emitted");
         }
 
-        if (dimensionsCount() == 0 || dimensionsCount() > 9) {
-            throw new ValidationException(String.format("Number of Dimensions must be in range of 1-9." +
+        if (dimensionsCount() > 9) {
+            throw new ValidationException(String.format("Number of Dimensions must be in range of 0-9." +
                     " Actual size: %d.", dimensionsCount()));
         }
     }
@@ -102,25 +105,32 @@ private String namespace(Metrics metrics) {
         return !"".equals(metrics.namespace()) ? metrics.namespace() : NAMESPACE;
     }
 
-    private String service(Metrics metrics) {
+    private static String service(Metrics metrics) {
         return !"".equals(metrics.service()) ? metrics.service() : serviceName();
     }
 
     private void validateMetricsAndRefreshOnFailure(Metrics metrics) {
         try {
             validateBeforeFlushingMetrics(metrics);
         } catch (ValidationException e){
-            refreshMetricsContext();
+            refreshMetricsContext(metrics);
             throw e;
         }
     }
 
     // This can be simplified after this issues https://github.com/awslabs/aws-embedded-metrics-java/issues/35 is fixed
-    private static void refreshMetricsContext() {
+    public static void refreshMetricsContext(Metrics metrics) {
         try {
             Field f = metricsLogger().getClass().getDeclaredField("context");
             f.setAccessible(true);
-            f.set(metricsLogger(), new MetricsContext());
+            MetricsContext context = new MetricsContext();
+
+            DimensionSet defaultDimensionSet = hasDefaultDimension() ? defaultDimensionSet()
+                    : DimensionSet.of("Service", service(metrics));
+
+            context.setDefaultDimensions(defaultDimensionSet);
+
+            f.set(metricsLogger(), context);
         } catch (NoSuchFieldException | IllegalAccessException e) {
             throw new RuntimeException(e);
         }

powertools-metrics/src/test/java/software/amazon/lambda/powertools/metrics/MetricsLoggerTest.java

@@ -6,6 +6,7 @@
 
 import com.fasterxml.jackson.core.JsonProcessingException;
 import com.fasterxml.jackson.databind.ObjectMapper;
+import org.assertj.core.api.Assertions;
 import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.BeforeAll;
 import org.junit.jupiter.api.BeforeEach;
@@ -16,6 +17,7 @@
 import software.amazon.cloudwatchlogs.emf.model.Unit;
 
 import static java.util.Collections.*;
+import static org.assertj.core.api.Assertions.*;
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.mockito.Mockito.mockStatic;
 import static software.amazon.lambda.powertools.core.internal.SystemWrapper.getenv;
@@ -43,6 +45,31 @@ static void beforeAll() {
         }
     }
 
+    @Test
+    void singleMetricsCaptureUtilityWithDefaultDimension() {
+        try (MockedStatic<SystemWrapper> mocked = mockStatic(SystemWrapper.class);
+             MockedStatic<software.amazon.lambda.powertools.core.internal.SystemWrapper> internalWrapper = mockStatic(software.amazon.lambda.powertools.core.internal.SystemWrapper.class)) {
+            mocked.when(() -> SystemWrapper.getenv("AWS_EMF_ENVIRONMENT")).thenReturn("Lambda");
+            internalWrapper.when(() -> getenv("_X_AMZN_TRACE_ID")).thenReturn("Root=1-5759e988-bd862e3fe1be46a994272793;Parent=53995c3f42cd8ad8;Sampled=1\"");
+
+            MetricsUtils.defaultDimensionSet(DimensionSet.of("Service", "Booking"));
+
+            MetricsUtils.withSingleMetric("Metric1", 1, Unit.COUNT, "test",
+                    metricsLogger -> {});
+
+            assertThat(out.toString())
+                    .satisfies(s -> {
+                        Map<String, Object> logAsJson = readAsJson(s);
+
+                        assertThat(logAsJson)
+                                .containsEntry("Metric1", 1.0)
+                                .containsEntry("Service", "Booking")
+                                .containsKey("_aws")
+                                .containsEntry("xray_trace_id", "1-5759e988-bd862e3fe1be46a994272793");
+                    });
+        }
+    }
+
     @Test
     void singleMetricsCaptureUtility() {
         try (MockedStatic<SystemWrapper> mocked = mockStatic(SystemWrapper.class);
@@ -96,6 +123,13 @@ void singleMetricsCaptureUtilityWithDefaultNameSpace() {
         }
     }
 
+    @Test
+    void shouldThrowExceptionWhenDefaultDimensionIsNull() {
+        assertThatNullPointerException()
+                .isThrownBy(() -> MetricsUtils.defaultDimensionSet(null))
+                .withMessage("Null dimension set not allowed");
+    }
+
     private Map<String, Object> readAsJson(String s) {
         try {
             return mapper.readValue(s, Map.class);