Skip a few tests if not enough memory can be allocated for them #7465

davidsaada · 2018-07-10T16:16:57Z

Description

Some tests need large amounts of memory in order to work. These tests will fail on low end boards, having low memory. This is also shown in issue #7115 where these tests fail on the TMPM066 board. This PR fixes the problem by simply skipping the test if there's not enough memory. All allocations use the std::nothrow directive.
~~PR depends on #7448.~~

Pull request type

[x] Fix
[ ] Refactor
[ ] New target
[ ] Feature
[ ] Breaking change

davidsaada · 2018-07-10T16:44:40Z

@soramame21 Can you test if this fixes #7115?

adbridge · 2018-07-10T17:06:43Z

@davidsaada I don't think you should have commit e0ca511 in this PR ? Isn't that what #7448 is ?

cmonr · 2018-07-11T03:13:28Z

@adbridge It'll go away once a rebase is done to this PR.
Plus, even if a rebase isn't done, because the hashes are the same, git will not duplicate the commits.

(I've done something similar and it worked out)

soramame21 · 2018-07-13T07:11:54Z

I had passed the test cases related to this fix. please see the log.

mbed test -m TMPM066 -t GCC_ARM -vv -n features-tests-filesystem-util_block_device

[1531465438.32][CONN][RXD] >>> Running 3 test cases...
[1531465438.37][CONN][INF] found KV pair in stream: {{__testcase_name;Testing slicing of a block device}}, queued...
[1531465438.43][CONN][INF] found KV pair in stream: {{__testcase_name;Testing chaining of block devices}}, queued...
[1531465438.49][CONN][RXD]
[1531465438.49][CONN][INF] found KV pair in stream: {{__testcase_name;Testing profiling of block devices}}, queued...
[1531465438.56][CONN][RXD] >>> Running case #1: 'Testing slicing of a block device'...
[1531465438.62][CONN][INF] found KV pair in stream: {{__testcase_start;Testing slicing of a block device}}, queued...
[1531465438.72][CONN][RXD] :41::SKIP: Not enough memory for test
[1531465438.72][CONN][INF] found KV pair in stream: {{__testcase_finish;Testing slicing of a block device;1;0}}, queued...
[1531465438.78][CONN][RXD] >>> 'Testing slicing of a block device': 1 passed, 0 failed
[1531465438.78][CONN][RXD]
[1531465438.85][CONN][RXD] >>> Running case #2: 'Testing chaining of block devices'...
[1531465438.91][CONN][INF] found KV pair in stream: {{__testcase_start;Testing chaining of block devices}}, queued...
[1531465439.01][CONN][RXD] :146::SKIP: Not enough memory for test
[1531465439.01][CONN][INF] found KV pair in stream: {{__testcase_finish;Testing chaining of block devices;1;0}}, queued...
[1531465439.08][CONN][RXD] >>> 'Testing chaining of block devices': 1 passed, 0 failed
[1531465439.08][CONN][RXD]
[1531465439.14][CONN][RXD] >>> Running case #3: 'Testing profiling of block devices'...
[1531465439.20][CONN][INF] found KV pair in stream: {{__testcase_start;Testing profiling of block devices}}, queued...
[1531465439.31][CONN][RXD] :216::SKIP: Not enough memory for test
[1531465439.31][CONN][INF] found KV pair in stream: {{__testcase_finish;Testing profiling of block devices;1;0}}, queued...
[1531465439.37][CONN][RXD] >>> 'Testing profiling of block devices': 1 passed, 0 failed
[1531465439.37][CONN][RXD]
[1531465439.41][CONN][RXD] >>> Test cases: 3 passed, 0 failed
[1531465439.44][CONN][INF] found KV pair in stream: {{__testcase_summary;3;0}}, queued...
[1531465439.47][CONN][INF] found KV pair in stream: {{max_heap_usage;0}}, queued...
[1531465439.47][HTST][ERR] orphan event in main phase: {{max_heap_usage;0}}, timestamp=1531465439.466000
[1531465439.49][CONN][INF] found KV pair in stream: {{reserved_heap;0}}, queued...
[1531465439.49][HTST][ERR] orphan event in main phase: {{reserved_heap;0}}, timestamp=1531465439.488000
[1531465439.51][CONN][INF] found KV pair in stream: {{end;success}}, queued...
[1531465439.52][HTST][INF] __notify_complete(True)
[1531465439.52][HTST][INF] __exit_event_queue received
[1531465439.52][HTST][INF] test suite run finished after 1.26 sec...
[1531465439.52][CONN][INF] found KV pair in stream: {{__exit;0}}, queued...
[1531465439.52][CONN][INF] received special event '__host_test_finished' value='True', finishing
[1531465439.53][HTST][INF] CONN exited with code: 0
[1531465439.53][HTST][INF] Some events in queue
[1531465439.53][HTST][INF] stopped consuming events
[1531465439.53][HTST][INF] host test result() call skipped, received: True
[1531465439.53][HTST][WRN] missing __exit event from DUT
[1531465439.53][HTST][INF] calling blocking teardown()
[1531465439.53][HTST][INF] teardown() finished
[1531465439.53][HTST][INF] {{result;success}}
mbedgt: checking for GCOV data...
mbedgt: mbed-host-test-runner: stopped and returned 'OK'
mbedgt: test on hardware with target id: 701100000000000000000000000000000000000097969904
mbedgt: test suite 'features-tests-filesystem-util_block_device' ..................................... OK in 41.27 sec
        test case: 'Testing chaining of block devices' ............................................... OK in 0.10 sec
        test case: 'Testing profiling of block devices' .............................................. OK in 0.11 sec
        test case: 'Testing slicing of a block device' ............................................... OK in 0.10 sec
mbedgt: test case summary: 3 passes, 0 failures
mbedgt: all tests finished!
mbedgt: shuffle seed: 0.8397540719
mbedgt: test suite report:
+-----------------+---------------+---------------------------------------------+--------+--------------------+-------------+
| target          | platform_name | test suite                                  | result | elapsed_time (sec) | copy_method |
+-----------------+---------------+---------------------------------------------+--------+--------------------+-------------+
| TMPM066-GCC_ARM | TMPM066       | features-tests-filesystem-util_block_device | OK     | 41.27              | default     |
+-----------------+---------------+---------------------------------------------+--------+--------------------+-------------+
mbedgt: test suite results: 1 OK
mbedgt: test case report:
+-----------------+---------------+---------------------------------------------+------------------------------------+--------+--------+--------+--------------------+
| target          | platform_name | test suite                                  | test case                          | passed | failed | result | elapsed_time (sec) |
+-----------------+---------------+---------------------------------------------+------------------------------------+--------+--------+--------+--------------------+
| TMPM066-GCC_ARM | TMPM066       | features-tests-filesystem-util_block_device | Testing chaining of block devices  | 1      | 0      | OK     | 0.1                |
| TMPM066-GCC_ARM | TMPM066       | features-tests-filesystem-util_block_device | Testing profiling of block devices | 1      | 0      | OK     | 0.11               |
| TMPM066-GCC_ARM | TMPM066       | features-tests-filesystem-util_block_device | Testing slicing of a block device  | 1      | 0      | OK     | 0.1                |
+-----------------+---------------+---------------------------------------------+------------------------------------+--------+--------+--------+--------------------+
mbedgt: test case results: 3 OK
mbedgt: completed in 44.20 sec

davidsaada · 2018-07-13T07:49:09Z

Thanks @soramame21. Also got report that it solves #7439

davidsaada · 2018-07-16T14:24:39Z

Rebased. No longer depending on #7448.

cmonr · 2018-07-18T13:44:02Z

Fixes #6008

0xc0170 · 2018-07-25T12:32:50Z

@mprse @fkjagodzinski @maciejbocianski Can you review these test changes ?

0xc0170 · 2018-07-31T12:59:00Z

Waiting for core team to review, meanwhile

/morph build

geky

I don't know if I count as core team, but this is a good solution 👍

EDIT: GitHub says I do, but I'm not sure that's correct. @SenRamakri?

cmonr

Interesting. Curious to see how well this works. LGTM.

cmonr · 2018-07-31T15:09:32Z

/morph build

mbed-ci · 2018-07-31T15:30:47Z

Build : SUCCESS

Build number : 2705
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7465/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build
/morph mbed2-build

mbed-ci · 2018-07-31T17:57:24Z

Build : SUCCESS

Build number : 2708
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7465/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build
/morph mbed2-build

mbed-ci · 2018-07-31T18:27:42Z

Exporter Build : SUCCESS

Build number : 2334
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7465/

mbed-ci · 2018-07-31T20:40:10Z

Exporter Build : FAILURE

Build number : 2337
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7465/

cmonr · 2018-07-31T22:45:20Z

That's what I get for double building...
/morph export-build

mbed-ci · 2018-07-31T23:41:24Z

Test : SUCCESS

Build number : 2436
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7465/2436

mbed-ci · 2018-08-01T02:02:30Z

Exporter Build : SUCCESS

Build number : 2344
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7465/

mbed-ci · 2018-08-01T04:01:27Z

Test : SUCCESS

Build number : 2439
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7465/2439

…h_memory Skip a few tests if not enough memory can be allocated for them

cmonr added the needs: preceding PR label Jul 10, 2018

cmonr requested review from a team July 10, 2018 16:33

cmonr added the needs: review label Jul 10, 2018

davidsaada mentioned this pull request Jul 11, 2018

Failing some filesystem tests for a specific tool-chain #7439

Closed

Skip a few tests if not enough memory can be allocated for them.

a224ca7

davidsaada force-pushed the david_tests_not_enough_memory branch from 539ab2a to a224ca7 Compare July 16, 2018 14:22

cmonr removed the needs: preceding PR label Jul 16, 2018

cmonr requested a review from studavekar July 16, 2018 15:53

davidsaada mentioned this pull request Jul 18, 2018

CI tests failing for filesystem tests on targets with 32K RAM #6008

Closed

davidsaada mentioned this pull request Jul 18, 2018

test: some cases are failing with GCC_ARM for Toshiba's TMPM3H6 #7535

Closed

maciejbocianski approved these changes Jul 30, 2018

View reviewed changes

fkjagodzinski approved these changes Jul 30, 2018

View reviewed changes

geky approved these changes Jul 31, 2018

View reviewed changes

cmonr approved these changes Jul 31, 2018

View reviewed changes

cmonr added needs: CI release-version: 5.9.5 and removed needs: review labels Jul 31, 2018

0xc0170 added ready for merge and removed needs: CI labels Aug 1, 2018

0xc0170 merged commit d5f70f0 into ARMmbed:master Aug 1, 2018

0xc0170 removed the ready for merge label Aug 1, 2018

davidsaada deleted the david_tests_not_enough_memory branch August 1, 2018 13:25

aashishc1988 mentioned this pull request Aug 3, 2018

skip the flashsim test if not enough memory cannot be allocated for it. #7696

Merged

pan- pushed a commit to pan-/mbed that referenced this pull request Aug 22, 2018

Merge pull request ARMmbed#7465 from davidsaada/david_tests_not_enoug…

b154fd8

…h_memory Skip a few tests if not enough memory can be allocated for them

davidsaada mentioned this pull request Sep 11, 2018

Are FILESYSTEM tests available with small targets ? #7407

Closed

Skip a few tests if not enough memory can be allocated for them #7465

Skip a few tests if not enough memory can be allocated for them #7465

Uh oh!

Conversation

davidsaada commented Jul 10, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Pull request type

Uh oh!

davidsaada commented Jul 10, 2018

Uh oh!

adbridge commented Jul 10, 2018

Uh oh!

cmonr commented Jul 11, 2018

Uh oh!

soramame21 commented Jul 13, 2018

Uh oh!

davidsaada commented Jul 13, 2018

Uh oh!

davidsaada commented Jul 16, 2018

Uh oh!

cmonr commented Jul 18, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

0xc0170 commented Jul 25, 2018

Uh oh!

0xc0170 commented Jul 31, 2018

Uh oh!

geky left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmonr left a comment

Choose a reason for hiding this comment

Uh oh!

cmonr commented Jul 31, 2018

Uh oh!

mbed-ci commented Jul 31, 2018

Build : SUCCESS

Triggering tests

Uh oh!

mbed-ci commented Jul 31, 2018

Build : SUCCESS

Triggering tests

Uh oh!

mbed-ci commented Jul 31, 2018

Exporter Build : SUCCESS

Uh oh!

mbed-ci commented Jul 31, 2018

Exporter Build : FAILURE

Uh oh!

cmonr commented Jul 31, 2018

Uh oh!

mbed-ci commented Jul 31, 2018

Test : SUCCESS

Uh oh!

mbed-ci commented Aug 1, 2018

Exporter Build : SUCCESS

Uh oh!

mbed-ci commented Aug 1, 2018

Test : SUCCESS

Uh oh!

Uh oh!

davidsaada commented Jul 10, 2018 •

edited

Loading

cmonr commented Jul 18, 2018 •

edited

Loading

geky left a comment •

edited

Loading