Storage tests: kvstore-general_tests_phase_1 fails for K64F

[0xc0170]

Description of defect

| K64F-ARMC6 | K64F  | features-storage-tests-kvstore-general_tests_phase_1                         | TDB_set_key_init_deinit    | 0      | 2      | FAIL   | 3.33    

The PR #11531 contained similar failures but not always. As @maclobdell reported couple of hours ago:

I have been running kvstore tests on 3 different FRDM-K64F on the master branch, before and after this PR was merged. I see different intermittent failures across different boards, but I see the same failures before and after this PR.

mostly I see these tests failing, but it doesn't seem to be related at all to this PR, unless the flash is getting left in a weird state and causing the test to fail subsequent runs.

features-storage-tests-kvstore-filesystemstore_tests | Testing Edge Cases
features-storage-tests-kvstore-general_tests_phase_1 | TDB_kvstore_init

Similar failures are in the latest nightly: http://mbed-os-ci.s3-website-eu-west-1.amazonaws.com/?prefix=jenkins-ci/ARMmbed/mbed-os/mbed-os-ci-nightly/artifacts/master/400/greentea-test/41/FAIL/K64F/ - only ARM showed this failure

As this is now on master, it's affecting PRs, see the failure in one of the recent PR here: https://mbed-os.mbedcloudtesting.com/blue/organizations/jenkins/mbed-os-ci_greentea-test/detail/mbed-os-ci_greentea-test/3093/tests/ - here GCC ARM failed

@ARMmbed/mbed-os-storage Please review

Target(s) affected by this defect ?

At least k64f shows this one in the recent test runs

Toolchain(s) (name and version) displaying this defect ?

So far I've seen it with ARM and GCC ARM

What version of Mbed-os are you using (tag or sha) ?

fd22997 - master

What version(s) of tools are you using. List all that apply (E.g. mbed-cli)

versions we use in CI

How is this defect reproduced ?

Run the tests reported as failures above

[0xc0170]

Similar failures for "needs: CI" PRs:

Internal links below

https://mbed-os.mbedcloudtesting.com/blue/organizations/jenkins/mbed-os-ci_greentea-test/detail/mbed-os-ci_greentea-test/3090/tests/

https://mbed-os.mbedcloudtesting.com/blue/organizations/jenkins/mbed-os-ci_greentea-test/detail/mbed-os-ci_greentea-test/3086/tests/

I'll reference all PRs affected so we restart the jobs once fixed.

[ciarmcom]

Internal Jira reference: https://jira.arm.com/browse/MBOTRIAGE-2337

[0xc0170]

From the latest nightly, this line errors: https://github.com/ARMmbed/mbed-os/blob/master/features/storage/TESTS/kvstore/general_tests_phase_1/main.cpp#L463 and the next line

[1573694720.86][CONN][RXD] >>> Running case #18: 'TDB_set_key_init_deinit'...
[1573694720.96][CONN][INF] found KV pair in stream: {{__testcase_start;TDB_set_key_init_deinit}}, queued...
mbedgt: :463::FAIL: Expected 0 Was 263
[1573694724.16][CONN][RXD] :463::FAIL: Expected 0 Was 263
mbedgt: :464::FAIL: Expected 'data' Was ''
[1573694724.16][CONN][RXD] :464::FAIL: Expected 'data' Was ''
[1573694724.26][CONN][RXD] >>> 'TDB_set_key_init_deinit': 0 passed, 2 failed with reason 'Test Cases Failed'

[adbridge]

@SeppoTakalo are you still investigating this ?

[0xc0170]

Master has this test skipped, we should keep this opened as the real issue still to be investigated. Is that correct @SeppoTakalo ?

[SeppoTakalo]

Yes, test is now skipped, but the issue is real.
So this ticket must be kept open.