Fuzzing only test generation for Builder pattern produces an empty test class

[alisevych]

Description

Test generation using Fuzzing only mode - the Simplest - for a method instantiating a class using Builder pattern - produces an empty test class.
Sarif report fails to be generated.
FormattingModelInconsistencyException was thrown - twice (not every run).

To Reproduce

Steps to reproduce the behavior:

1. Run IntelliJ IDEA Ultimate 2022.1.4 with UTBot Java plugin installed
2. Create a Gradle project with JDK 11
3. Create the following class:

public class Person {

    private String lastName;
    private String firstName;

    private Person() {
        // private constructor
    }

    public String getLastName() {
        return lastName;
    }

    public String getFirstName() {
        return firstName;
    }

    public static Builder newBuilder() {
        return new Person().new Builder();
    }

    public class Builder {

        private Builder() {
            // private constructor
        }

        public Builder setLastName(String lastName) {
            Person.this.lastName = lastName;

            return this;
        }

        public Builder setFirstName(String firstName) {
            Person.this.firstName = firstName;

            return this;
        }

        public Person build() {
            return Person.this;
        }
    }

    public Person getSomeone(String name) {
        return Person.newBuilder()
                .setFirstName(name)
                .setLastName("Last")
                .build();
    }
}

4. Generate a test for getSomeone method with UTBot

Expected behavior

Tests are supposed to be generated.

Actual behavior

Empty PersonTest.java class is generated and Error dialog with "Failed to save tests report" is displayed.

Once there was FormattingModelInconsistencyException thrown

Visual proofs (screenshots, logs, images)

com.intellij.formatting.FormatterImpl$FormattingModelInconsistencyException: Invalid root block PSI element
	at com.intellij.formatting.FormatterImpl.validateModel(FormatterImpl.java:707)
	at com.intellij.formatting.FormatterImpl.format(FormatterImpl.java:221)
	at com.intellij.psi.impl.source.codeStyle.CodeFormatterFacade.processRange(CodeFormatterFacade.java:117)
	at com.intellij.psi.impl.source.codeStyle.CodeFormatterFacade.processElement(CodeFormatterFacade.java:67)
	at com.intellij.formatting.service.CoreFormattingService.formatElement(CoreFormattingService.java:44)
	at com.intellij.formatting.service.FormattingServiceUtil.formatElement(FormattingServiceUtil.java:67)
	at com.intellij.psi.impl.source.codeStyle.CodeStyleManagerImpl.reformat(CodeStyleManagerImpl.java:82)
	at com.intellij.psi.impl.source.codeStyle.CodeStyleManagerImpl.reformat(CodeStyleManagerImpl.java:65)
	at org.utbot.intellij.plugin.generator.CodeGenerationController.reformat(CodeGenerationController.kt:337)
	at org.utbot.intellij.plugin.generator.CodeGenerationController.access$reformat(CodeGenerationController.kt:83)
	at org.utbot.intellij.plugin.generator.CodeGenerationController$generateCodeAndReport$1$1$1$2.run(CodeGenerationController.kt:308)
	at com.intellij.openapi.command.WriteCommandAction.lambda$runWriteCommandAction$4(WriteCommandAction.java:362)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.lambda$doRunWriteCommandAction$1(WriteCommandAction.java:150)
	at com.intellij.openapi.application.impl.ApplicationImpl.runWriteAction(ApplicationImpl.java:1015)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.lambda$doRunWriteCommandAction$2(WriteCommandAction.java:148)
	at com.intellij.openapi.command.impl.CoreCommandProcessor.executeCommand(CoreCommandProcessor.java:219)
	at com.intellij.openapi.command.impl.CoreCommandProcessor.executeCommand(CoreCommandProcessor.java:184)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.doRunWriteCommandAction(WriteCommandAction.java:157)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.run(WriteCommandAction.java:124)
	at com.intellij.openapi.command.WriteCommandAction.runWriteCommandAction(WriteCommandAction.java:362)
	at org.utbot.intellij.plugin.generator.CodeGenerationController$generateCodeAndReport$1$1$1.run(CodeGenerationController.kt:307)
	at org.utbot.intellij.plugin.generator.CodeGenerationController$run$$inlined$let$lambda$3.compute(actions.kt:58)
	at com.intellij.openapi.application.impl.ApplicationImpl.lambda$runWriteAction$13(ApplicationImpl.java:1024)
	at com.intellij.openapi.application.impl.ApplicationImpl.runWriteActionWithClass(ApplicationImpl.java:1003)
	at com.intellij.openapi.application.impl.ApplicationImpl.runWriteAction(ApplicationImpl.java:1024)
	at org.utbot.intellij.plugin.generator.CodeGenerationController.run(CodeGenerationController.kt:584)
	at org.utbot.intellij.plugin.generator.CodeGenerationController.access$run(CodeGenerationController.kt:83)
	at org.utbot.intellij.plugin.generator.CodeGenerationController$generateCodeAndReport$1$1.run(CodeGenerationController.kt:293)
	at org.utbot.intellij.plugin.generator.CodeGenerationController$run$$inlined$let$lambda$4.run(ApplicationUtils.kt:59)
	at com.intellij.openapi.application.TransactionGuardImpl.runWithWritingAllowed(TransactionGuardImpl.java:215)
	at com.intellij.openapi.application.TransactionGuardImpl.access$100(TransactionGuardImpl.java:22)
	at com.intellij.openapi.application.TransactionGuardImpl$1.run(TransactionGuardImpl.java:197)
	at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:873)
	at com.intellij.openapi.application.impl.ApplicationImpl$3.run(ApplicationImpl.java:511)
	at com.intellij.openapi.application.impl.FlushQueue.doRun(FlushQueue.java:69)
	at com.intellij.openapi.application.impl.FlushQueue.runNextEvent(FlushQueue.java:112)
	at com.intellij.openapi.application.impl.FlushQueue.flushNow(FlushQueue.java:42)
	at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:776)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:727)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
	at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:746)
	at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:898)
	at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:746)
	at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$6(IdeEventQueue.java:439)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computePrioritized(CoreProgressManager.java:803)
	at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$7(IdeEventQueue.java:438)
	at com.intellij.openapi.application.TransactionGuardImpl.performActivity(TransactionGuardImpl.java:106)
	at com.intellij.ide.IdeEventQueue.performActivity(IdeEventQueue.java:604)
	at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$8(IdeEventQueue.java:436)
	at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:873)
	at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:484)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:207)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:128)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:117)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:113)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:105)
	at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:92)

Environment

IntelliJ IDEA 2022.1.4
Gradle project with JDK 11

Additional context

When Symbolic engine is turned on - even minimally - a test is generated.
Not reproducing on Gradle project with JDK 8,
Not reproducing on Maven and IntelliJ projects with JDK 11

[alisevych]

@Markoutte Please pay attention to the FormattingModelInconsistencyException - reproduced again with the same scenario.

[Markoutte]

There's no exception before PR #573. Before it creates empty test with error messages about failed concrete execution. @sofurihafe please, take a look.

[Markoutte]

@alisevych fyi, looks like this issue isn't reproducible anymore. Could you verify this?

[alisevych]

@Markoutte Closing as not reproducing: a successful test is generated by Fuzzing