Skip to content

Tests without any checks [C standard test] #323

Closed
@belous-dp

Description

@belous-dp
Collaborator

Description
Tests generated for functions with struct as return value don't contain assertions (EXPECT_, ASSERT_).

That happens only with "verbose formatting" disabled.

To Reproduce
Steps to reproduce the behavior:

  1. Set "Verbose formatting" setting disabled
  2. Open a file floating_point.c in the 'c-example' project
  3. Generate tests for the function fp_array:
struct FParray {
    float data[2];
};

struct FParray fp_array(int a) {
    if (a < 0) {
        struct FParray res = {{1.23, 3.21}};
        return res;
    }
    struct FParray res = {{12.3, 32.1}};
    return res;
}

Expected behavior
Tests are supposed to contain gtest assertions like EXPECT_, ASSERT_. For example, with the above source code I want to get tests like that:

TEST(regression, fp_array_test_1)
{
    struct FParray actual = fp_array(0);
    struct FParray expected = {
        .data = {1.230000e+01, 3.210000e+01}};
    EXPECT_EQ(sizeof(expected.data), sizeof(actual.data));
    size_t n = 0;
    if (sizeof(expected.data) > 0) {
        EXPECT_EQ(sizeof(expected.data) / sizeof(expected.data[0]), sizeof(actual.data) / sizeof(actual.data[0]));
        n = sizeof(expected.data) / sizeof(expected.data[0]);
    }
    for (int i = 0; i < n; i ++) {
        EXPECT_NEAR(expected.data[i], actual.data[i], utbot_abs_error);
    }
}

TEST(regression, fp_array_test_2) { ... }

Actual behavior
Instead, the generated tests do not contain any checks:

TEST(regression, fp_array_test_1)
{
    struct FParray actual = fp_array(0);
    struct FParray expected = {
        .data = {1.230000e+01, 3.210000e+01}};
}

TEST(regression, fp_array_test_2)
{
    struct FParray actual = fp_array(-1);
    struct FParray expected = {
        .data = {1.230000e+00, 3.210000e+00}};
}

Behaviour with "verbose formatting" on
Generated tests:

TEST(regression, fp_array_test_1)
{
    // Construct input
    int a = 0;



    // Expected output
    struct FParray expected = {
        .data = {1.230000e+01, 3.210000e+01}};

    // Trigger the function
    struct FParray actual = fp_array(a);

    // Check results
    for (int it_2_0 = 0; it_2_0 < 2; it_2_0 ++) {
        EXPECT_NEAR(expected.data[it_2_0], actual.data[it_2_0], utbot_abs_error);
    }
}

TEST(regression, fp_array_test_2)
{
    // Construct input
    int a = -1;



    // Expected output
    struct FParray expected = {
        .data = {1.230000e+00, 3.210000e+00}};

    // Trigger the function
    struct FParray actual = fp_array(a);

    // Check results
    for (int it_3_0 = 0; it_3_0 < 2; it_3_0 ++) {
        EXPECT_NEAR(expected.data[it_3_0], actual.data[it_3_0], utbot_abs_error);
    }
}

Comments
The same thing happens with any function with a struct as a return value. Examples: simple_structs.c, complex_structs.c.

The same with unions:
Source code:

union MainUnion {
    union InnerUnion {
        union InInnerUnion {
            unsigned int u;
            long long l;
        };
        char c;
        union InInnerUnion ininner;
        short s;
    } inner;

    int x;
    // union InnerUnion inner;
    long long y;
};

union MainUnion union_as_return_type(int a) {
    if (a == 0) {
        union MainUnion res = {{.c='0'}};
        return res;
    }

    if (a == 1) {
        union MainUnion res = {{.ininner={.l=1}}};
        return res;
    }

    union MainUnion res = {.x=2};
    return res;
}

Generated tests ("verbose formatting off):

TEST(regression, union_as_return_type_test_1)
{
    union MainUnion actual = union_as_return_type(4);
    union MainUnion expected = from_bytes<MainUnion>({2, 0, 0, 0, 0, 0, 0, 0});
}

TEST(regression, union_as_return_type_test_2)
{
    union MainUnion actual = union_as_return_type(1);
    union MainUnion expected = from_bytes<MainUnion>({1, 0, 0, 0, 0, 0, 0, 0});
}

TEST(regression, union_as_return_type_test_3)
{
    union MainUnion actual = union_as_return_type(0);
    union MainUnion expected = from_bytes<MainUnion>({48, 0, 0, 0, 0, 0, 0, 0});
}

Maybe unions can be compared via memcmp.

Environment
UTBotCpp version 2022.7.0, tested locally using Docker+CLion+VSCode and tested online using utbot.

Activity

moved this to Todo in UTBot C/C++on Jul 14, 2022
changed the title [-]Tests without the assertions[/-] [+]Tests without the assertions [C standard test][/+] on Jul 15, 2022
changed the title [-]Tests without the assertions [C standard test][/-] [+]Tests without the assertions in short formatting [C standard test][/+] on Jul 18, 2022
changed the title [-]Tests without the assertions in short formatting [C standard test][/-] [+]Tests without any checks [C standard test][/+] on Jul 18, 2022
added
need to checkCheck if the issue can be reproduced in last version
on Nov 15, 2022
self-assigned this
on Nov 16, 2022
tyuldashev

tyuldashev commented on Nov 17, 2022

@tyuldashev
Collaborator

Not reproducible anymore. Checked on v2022.11.298.

Repository owner moved this from Todo to Done in UTBot C/C++on Nov 17, 2022
added
verifiedBug fix is verified
and removed
need to checkCheck if the issue can be reproduced in last version
on Nov 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't workingverifiedBug fix is verified

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @alexey-utkin@tyuldashev@ladisgin@belous-dp

      Issue actions

        Tests without any checks [C standard test] · Issue #323 · UnitTestBot/UTBotCpp