Feature request: Remove Python 3.6 support

[michaelbrewer]

Use case

Now that Python 3.6 has been deprecated, moving up to Python 3.7 will come with some user experience improvements.

Also most of the library dependencies don't support Python 3.6 anymore in their Github Actions:

NOTE: JMESPath has already caused issues in 1.0.0, and had to be removed. Some utilities use JMESPath and not
Boto3, and so we can not account JMESPath will always be bundled at runtime like when vendored version requests was removed

Runtime dependencies:

• https://github.com/horejsek/python-fastjsonschema (only supports 3.7+)
• https://github.com/jmespath/jmespath.py (only supports 3.7+)
• https://github.com/samuelcolvin/pydantic (only supports 3.7+)
• https://github.com/boto/boto3 (On 2022-05-30, we will be dropping support for Python 3.6.)

And same for dev tooling:

• https://github.com/nedbat/coveragepy (3.7+)
• https://github.com/pytest-dev/pytest (3.7+)

Solution/User Experience

Drop Python 3.6 from the supported list for the next couple released. And then start targeting Python 3.7+ features like enhanced typing. Add back JMESPath as a dependency

And embrace some of the cool new features in Python 3.7 like:

• @dataclass decorator could remove alot of the boilerplate code.
• __getattr__ could make reflection easier
• Typing Enhancements will mean not more "ClassName"

Solution as a diff to develop batch at the time of righting:

diff --git a/.github/workflows/python_build.yml b/.github/workflows/python_build.yml
index f37f6c70..306e08b4 100644
--- a/.github/workflows/python_build.yml
+++ b/.github/workflows/python_build.yml
@@ -16,7 +16,7 @@ jobs:
     strategy:
       max-parallel: 4
       matrix:
-        python-version: [3.6, 3.7, 3.8, 3.9]
+        python-version: ["3.7", "3.8", "3.9"]
     env:
       OS: ${{ matrix.os }}
       PYTHON: ${{ matrix.python-version }}
diff --git a/pyproject.toml b/pyproject.toml
index 7263ff4b..11f84273 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -20,7 +20,7 @@ keywords = ["aws_lambda_powertools", "aws", "tracing", "logging", "lambda", "pow
 license = "MIT-0"
 
 [tool.poetry.dependencies]
-python = "^3.6.2"
+python = "^3.7.10"
 aws-xray-sdk = "^2.8.0"
 fastjsonschema = "^2.14.5"
 boto3 = "^1.18"
diff --git a/tests/functional/idempotency/test_idempotency.py b/tests/functional/idempotency/test_idempotency.py
index 40cee10e..2862b0fb 100644
--- a/tests/functional/idempotency/test_idempotency.py
+++ b/tests/functional/idempotency/test_idempotency.py
@@ -1,4 +1,5 @@
 import copy
+import dataclasses
 import sys
 from hashlib import md5
 from unittest.mock import MagicMock
@@ -32,13 +33,6 @@ from tests.functional.utils import json_serialize, load_event
 TABLE_NAME = "TEST_TABLE"
 
 
-def get_dataclasses_lib():
-    """Python 3.6 doesn't support dataclasses natively"""
-    import dataclasses
-
-    return dataclasses
-
-
 # Using parametrize to run test twice, with two separate instances of persistence store. One instance with caching
 # enabled, and one without.
 @pytest.mark.parametrize("idempotency_config", [{"use_local_cache": False}, {"use_local_cache": True}], indirect=True)
@@ -1128,11 +1122,8 @@ def test_invalid_dynamodb_persistence_layer():
     assert str(ve.value) == "key_attr [id] and sort_key_attr [id] cannot be the same!"
 
 
-@pytest.mark.skipif(sys.version_info < (3, 7), reason="requires python3.7 or higher for dataclasses")
 def test_idempotent_function_dataclasses():
     # Scenario _prepare_data should convert a python dataclasses to a dict
-    dataclasses = get_dataclasses_lib()
-
     @dataclasses.dataclass
     class Foo:
         name: str
@@ -1162,10 +1153,8 @@ def test_idempotent_function_other(data):
     assert _prepare_data(data) == data
 
 
-@pytest.mark.skipif(sys.version_info < (3, 7), reason="requires python3.7 or higher for dataclasses")
 def test_idempotent_function_dataclass_with_jmespath():
     # GIVEN
-    dataclasses = get_dataclasses_lib()
     config = IdempotencyConfig(event_key_jmespath="transaction_id", use_local_cache=True)
     mock_event = {"customer_id": "fake", "transaction_id": "fake-id"}
     idempotency_key = "test-func.collect_payment#" + hash_idempotency_key(mock_event["transaction_id"])

Alternative solutions

Ultimately, there is no other solutions that make sense like Python 3.6 support is completely removed from AWS Lambda.

Acknowledgment

• This feature request meets Lambda Powertools Tenets
• Should this be considered in other Lambda Powertools languages? i.e. Java, TypeScript

[sthulb]

Whilst Python 3.6 has hit EOL, we'll continue to support until at least the end dates provided by the announcement email:

As described in the Lambda runtime support policy [2], end of support for language runtimes in Lambda happens in two stages. Starting July 18, 2022, Lambda will no longer apply security patches and other updates to the Python 3.6 runtime used by Lambda functions, and functions using Python 3.6 will no longer be eligible for technical support. In addition, you will no longer be able to create new Lambda functions using the Python 3.6 runtime. Starting August 17, 2022, you will no longer be able to update existing functions using the Python 3.6 runtime.

We'll revisit this in late August, as that's when customers will no longer be able to update existing functions.

[michaelbrewer]

It's when your downstream dependencies drop support for python 3.6 and their are security issues or bug fixes that don't make it to your current pinned version. This is when things get problematic.

[sthulb]

If this becomes a concern for the project later, we might choose to revisit this earlier, but we’re not expecting anything that changes the timeframe mentioned above.

[michaelbrewer]

Either way, you will need to juggle supporting both versions this PR is an example, it is doable just extra overhead.

Also note, there is no integration tests verifying python 3.6 actually works on a deployed lambda.

• fix(deps): update jmespath marker to support 1.0 and py3.6 #1139

[michaelbrewer]

Maybe to keep track of tasks like this https://github.com/awslabs/aws-lambda-powertools-python/blob/6917d90444fb9e43e7317010faaa2abb9f183464/pyproject.toml#L31 we should at least have a ticket for when Python 3.6 is removed

[michaelbrewer]

Added boto3 deprecation notice which is from 2022-05-30