Skip to content

Commit bb3f687

Browse files
committed
Fix tests with missing EXIT or SIGNAL tests
Some tests had previously been passing despite actually causing a crash, due to the required output being too loosely specified. This ensures the bare minimum: that every test has an expected EXIT code and SIGNAL result. The codes suggested were taken from the test's current output, and only applied for CORE tests, but hand inspection suggests these choices are reasonable.
1 parent f17c989 commit bb3f687

File tree

167 files changed

+264
-3
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

167 files changed

+264
-3
lines changed

regression/ansi-c/Forward_Declaration2/test.desc

+1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
CORE
22
main.c
33

4+
^EXIT=(1|64)$
45
^SIGNAL=0$
56
^CONVERSION ERROR$
67
--

regression/ansi-c/Incomplete_Type1/test.desc

+1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
CORE
22
main.c
33

4+
^EXIT=(1|64)$
45
^SIGNAL=0$
56
^CONVERSION ERROR$
67
--

regression/ansi-c/function_return1/test.desc

+1
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ CORE
22
main.c
33
--verbosity 2
44
^main.c:3:1: warning: function has return void but a return statement returning signed int$
5+
^EXIT=0$
56
^SIGNAL=0$
67
--
78
^warning: ignoring

regression/ansi-c/static2/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
main.c
33
main2.c --function foo
44
^main symbol `foo' is ambiguous$
5+
^EXIT=(1|64)$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/ansi-c/static3/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
main.c
33
main2.c --function foo
44
^main symbol `foo' is ambiguous$
5+
^EXIT=(1|64)$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetArray/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetArray.class
33
--function NondetArray.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetArray2/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ CORE
22
NondetArray2.class
33
--function NondetArray2.main --unwind 5
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
--
79
Disabled pending fixing warnings for array-of with zero length:

regression/cbmc-java/NondetArray3/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ CORE
22
NondetArray3.class
33
--function NondetArray3.main --unwind 5
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
--
79
Disabled pending fixing warnings for array-of with zero length:

regression/cbmc-java/NondetArray4/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetArray4.class
33
--function NondetArray4.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetAssume1/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetAssume1.class
33
--function NondetAssume1.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetAssume2/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetAssume2.class
33
--function NondetAssume2.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetBoolean/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetBoolean.class
33
--function NondetBoolean.main
44
^VERIFICATION FAILED$
5+
^EXIT=10$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetByte/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetByte.class
33
--function NondetByte.main
44
^VERIFICATION FAILED$
5+
^EXIT=10$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetCastToObject/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetCastToObject.class
33
--function NondetCastToObject.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetChar/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetChar.class
33
--function NondetChar.main
44
^VERIFICATION FAILED$
5+
^EXIT=10$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetDirectFromMethod/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetDirectFromMethod.class
33
--function NondetDirectFromMethod.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetDouble/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetDouble.class
33
--function NondetDouble.main
44
^VERIFICATION FAILED$
5+
^EXIT=10$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetFloat/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetFloat.class
33
--function NondetFloat.main
44
^VERIFICATION FAILED$
5+
^EXIT=10$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetGenericArray/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ CORE
22
NondetGenericArray.class
33
--function NondetGenericArray.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
--
79
Disabled pending fixing warnings for array-of with zero length:

regression/cbmc-java/NondetGenericRecursive/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetGenericRecursive.class
33
--function NondetGenericRecursive.main
44
^VERIFICATION FAILED$
5+
^EXIT=10$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetGenericRecursive2/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetGenericRecursive2.class
33
--function NondetGenericRecursive2.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetGenericWithNull/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetGenericWithNull.class
33
--function NondetGenericWithNull.main
44
^VERIFICATION FAILED$
5+
^EXIT=10$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetGenericWithoutNull/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetGenericWithoutNull.class
33
--function NondetGenericWithoutNull.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetInit/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,6 @@ CORE
22
Test.class
33
--function Test.check
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--

regression/cbmc-java/NondetInit/test_lazy.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,6 @@ CORE
22
Test.class
33
--function Test.check --lazy-methods
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--

regression/cbmc-java/NondetInit2/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,5 @@ CORE
22
Test.class
33
--function Test.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$

regression/cbmc-java/NondetInit3/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,5 @@ CORE
22
Test.class
33
--function Test.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$

regression/cbmc-java/NondetInt/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetInt.class
33
--function NondetInt.main
44
^VERIFICATION FAILED$
5+
^EXIT=10$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetLong/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetLong.class
33
--function NondetLong.main
44
^VERIFICATION FAILED$
5+
^EXIT=10$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/NondetShort/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
NondetShort.class
33
--function NondetShort.main
44
^VERIFICATION FAILED$
5+
^EXIT=10$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/address_space_size_limit1/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,6 @@ CORE
22
Test.class
33
--object-bits 4
44
too many addressed objects
5+
^EXIT=6$
6+
^SIGNAL=0$
57
--

regression/cbmc-java/array2/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,7 @@ CORE
22
test.class
33
--function test.f --cover location
44
\d+ of \d+ covered
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring

regression/cbmc-java/exceptions26/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ CORE
22
test.class
33

44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring
79
--

regression/cbmc-java/exceptions27/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ CORE
22
test.class
33

44
VERIFICATION SUCCESSFUL
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
^warning: ignoring
79
--

regression/cbmc-java/inherited_static_field1/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ CORE
22
Test.class
33
--function Test.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
--
79
This checks that jbmc knows that test.x and parent.x refer to the same field.

regression/cbmc-java/inherited_static_field10/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ Test.class
44
Stub static field x found for non-stub type java::Test\. In future this will be a fatal error\.
55
assertion at file Test\.java line 6 .* FAILURE
66
^VERIFICATION FAILED$
7+
^EXIT=10$
8+
^SIGNAL=0$
79
--
810
Stub static field x found for non-stub type java::Parent\. In future this will be a fatal error\.
911
--

regression/cbmc-java/inherited_static_field2/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ CORE
22
Test.class
33
--function Test.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
--
79
This checks that jbmc knows that Test.x and InterfaceWithStatic.x are the same field.

regression/cbmc-java/inherited_static_field3/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ CORE
22
Test.class
33
--function Test.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
--
79
This is the same as test inherited_static_field1, except Test's parent is opaque.

regression/cbmc-java/inherited_static_field4/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ Test.class
33
--function Test.main
44
assertion at file Test\.java line 6 .* FAILURE
55
^VERIFICATION FAILED$
6+
^EXIT=10$
7+
^SIGNAL=0$
68
--
79
--
810
This checks that jbmc knows that the hidden field Grandparent.x is not the same

regression/cbmc-java/inherited_static_field5/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ CORE
22
Test.class
33
--function Test.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
--
79
This is the same as inherited_static_field2, except that Test's parent is opaque. It must pass because

regression/cbmc-java/inherited_static_field6/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ Test.class
33
--function Test.main
44
assertion at file Test\.java line 6 .* FAILURE
55
^VERIFICATION FAILED$
6+
^EXIT=10$
7+
^SIGNAL=0$
68
--
79
--
810
This test is the same as inherited_static_field4, except that Test's parent is opaque.

regression/cbmc-java/inherited_static_field7/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ CORE
22
Test.class
33
--function Test.main
44
^VERIFICATION SUCCESSFUL$
5+
^EXIT=0$
6+
^SIGNAL=0$
57
--
68
--
79
This test must pass, as there is only one opaque parent (interface_with_static).

regression/cbmc-java/inherited_static_field8/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ Test.class
44
assertion at file Test\.java line 6 .* SUCCESS
55
assertion at file Test\.java line 7 .* FAILURE
66
^VERIFICATION FAILED$
7+
^EXIT=10$
8+
^SIGNAL=0$
79
--
810
--
911
This test is the same as inherited_static_field4, except that Test's grandparent is opaque.

regression/cbmc-java/inherited_static_field9/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@ Stub static field x found for non-stub type java::Test\. In future this will be
55
Stub static field x found for non-stub type java::Parent\. In future this will be a fatal error\.
66
assertion at file Test\.java line 6 .* FAILURE
77
^VERIFICATION FAILED$
8+
^EXIT=10$
9+
^SIGNAL=0$
810
--
911
--
1012
This tests the corner case where static fields are found on non-stub types, here caused

regression/cbmc-java/lambda1/test.desc

+2
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,5 @@ lambda function reference lambda\$k\$5 in class \"Lambdatest\"
1010
lambda function reference lambda\$l\$6 in class \"Lambdatest\"
1111
lambda function reference lambda\$m\$7 in class \"Lambdatest\"
1212
lambda function reference lambda\$static\$0 in class \"B\"
13+
^EXIT=0$
14+
^SIGNAL=0$

regression/cbmc-java/lazyloading_synthetic_method_cleanup1/check_clinit_normally_present.desc

+2
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,5 @@ Test.class
33
--show-goto-functions --function Test.main
44
java::Unused::clinit_wrapper
55
Unused\.<clinit>\(\)
6+
^EXIT=0$
7+
^SIGNAL=0$
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
CORE
22
Test.class
33
--lazy-methods --show-goto-functions --function Test.main
4+
^EXIT=0$
5+
^SIGNAL=0$
46
--
57
java::Unused::clinit_wrapper
68
Unused\.<clinit>\(\)

regression/cbmc-java/lazyloading_synthetic_method_cleanup1/check_runs_with_lazy_loading.desc

+2
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,5 @@ CORE
22
Test.class
33
--lazy-methods --function Test.main
44
VERIFICATION SUCCESSFUL
5+
^EXIT=0$
6+
^SIGNAL=0$

regression/cbmc-java/lazyloading_synthetic_method_cleanup2/check_clinit_normally_present.desc

+2
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,5 @@ Test.class
44
java::Unused1::clinit_wrapper
55
java::Unused2::clinit_wrapper
66
Unused2\.<clinit>\(\)
7+
^EXIT=0$
8+
^SIGNAL=0$

0 commit comments

Comments
 (0)