Skip to content

Support test methods with Kotlin suspend modifier #1914

@IRus

Description

@IRus

Goals

Support running suspend test in JUnit Jupiter:

class Kit {
    @Test 
    suspend fun foo() {
        delay(1000) // suspend call
        assertEquals(1, 1)
    } 
}

Currently, such test can be written this way:

class Kit {
    @Test 
    fun foo() = runBlocking {
        delay(1000) // suspend call
        assertEquals(1, 1)
        assertThrows { /* ... */ } 
        Unit // test should return void, but `assertThrows` returns `Throwable`, so `foo` returns `Throwable` too 
    } 
}

Also, will be nice to provide CoroutineScope through params, or as receiver in extension:

class Kit {
    suspend fun foo(scope: CoroutinesScope) {  /* ... */  } // (1)
    suspend fun CoroutinesScope.foo() {  /* ... */  } // (2)
}

1 and 2 actually the same on bytecode level. suspend is optional.

And finally, support for runBlockingTest:

class Kit {
    suspend fun foo(scope: TestCoroutinesScope) {  /* ... */  }
    suspend fun TestCoroutinesScope.foo() {  /* ... */  } 
}

What can be done currently

ParameterResolver can be used to provide stubs for Continuation, CoroutineScope and TestCoroutineScope. These stub arguments can be replaced with real arguments in invocation.

Problems

Current extensions points not enough to implement this feature as extensions, since:

  1. Discovery. Jupiter discovers tests that returns void, but suspend fun returns Object;
  2. Invocation. InvocationInterceptor in 5.5-M1(SNAPSHOT) don't providing mechanism to override actual invocation, only to decoration of existing invocation. Conversion of method to kotlinFunction, and then executing using callSuspend is necessary to execute suspend fun.

Also, my slides about this topic.

Activity

sormuras

sormuras commented on Jun 4, 2019

@sormuras
Member

Related to #1851

added this to the 5.7 M1 milestone on Mar 19, 2020
marcphilipp

marcphilipp commented on Mar 19, 2020

@marcphilipp
Member

@IRus This is not resolved by #2042, is it?

IRus

IRus commented on Mar 20, 2020

@IRus
Author

@marcphilipp Yes, not resolved, this is different issues

modified the milestones: 5.7 M1, 5.7 Backlog on Mar 21, 2020
xeruf

xeruf commented on Feb 4, 2021

@xeruf

Please resolve this! My tests looked all fine until I discovered that they were not run at all anymore after adding suspending functions :/

34 remaining items

JavierSegoviaCordoba

JavierSegoviaCordoba commented on May 19, 2025

@JavierSegoviaCordoba

@marcphilipp which is the reason to use runBlocking over runTest?

marcphilipp

marcphilipp commented on May 19, 2025

@marcphilipp
Member

You're referring to https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-test/kotlinx.coroutines.test/run-test.html, right? I wasn't sure it was a good fit because it returns TestResult which is not consumable (unless you manually use runTest) and it skips delays which I thought would be rather surprising. It may be possible to convince me to change it, though, if you have good arguments. 🙂

rgmz

rgmz commented on May 19, 2025

@rgmz

Thanks for adding this, @marcphilipp!

... and it skips delays which I thought would be rather surprising.

IMO fast-forwarding delays is desirable behaviour. I don't want, or necessarily need, my ExponentialBackOff test cases to actually take 2/4/8/16/32/64/128... seconds.

IRus

IRus commented on May 19, 2025

@IRus
Author

Using runTest would imply passing a TestScope to the test functions somehow; otherwise, the user can’t call methods like advanceTimeBy.

Specifying TestScope also requires more or the same syntax than simply writing runTest.

Additionally, this would imply that suspend tests would only work if the user explicitly provides kotlinx-coroutines-test, or it must be included in junit-platform-common, which I assume is undesirable.

opatry

opatry commented on May 19, 2025

@opatry

Also, having TestScope being provided to the test would allow using advanceUntilIdle and stuff like that which is very desirable also.

ephemient

ephemient commented on May 19, 2025

@ephemient

I would have used kotlin.coroutines.jvm.internal.runSuspend instead of kotlinx.coroutines.runBlocking - it's in kotlin-stdlib-jvm so it's always present, and if a user wants a real CoroutineScope or TestScope they can use runBlocking or runTest themselves.

JavierSegoviaCordoba

JavierSegoviaCordoba commented on May 20, 2025

@JavierSegoviaCordoba

@marcphilipp if I remember correctly, it is highly recommended to use "real" delay on tests instead of utility functions to add waits and/or move forward in time. Maybe @dkhalanskyjb can help with this, as I vaguely remember his comments.

runBlocking would make those tests too slow, so runTest is the only way to go. Additionally it is necessary to be able to change the dispatcher somehow.

It would be amazing if we could do something like this annotation to set the dispatcher for the whole test class:

@TestDispatcher(FooDispatcher::class)
class BarTest {

    ...
}

Even I would find a bit weird to need to replace the dispatcher only on a single test instead of the whole class, @TestDispatcher(FooDispatcher::class) should work if it is annotating a test method too.

dkhalanskyjb

dkhalanskyjb commented on May 20, 2025

@dkhalanskyjb

if I remember correctly, it is highly recommended to use "real" delay on tests instead of utility functions to add waits and/or move forward in time.

When possible (which is most of the time), yes, that's the recommended approach: Kotlin/kotlinx.coroutines#3919

marcphilipp

marcphilipp commented on May 23, 2025

@marcphilipp
Member

@JavierSegoviaCordoba Could you please create a new issue for runTest and @TestDispatcher?

JavierSegoviaCordoba

JavierSegoviaCordoba commented on Jul 15, 2025

@JavierSegoviaCordoba

@marcphilipp created here: #4768

IRus

IRus commented on Jul 15, 2025

@IRus
Author

@marcphilipp I think @ephemient having a very good point about using runSuspend (which is internal, but implementation is very straightforward and can be re-implemented easily).

runBlocking doing some clever things internally: https://madhead.me/posts/chm-not-concurrent-runblocking-not-blocking/

And runSuspend just blocking current thread until coroutine finished without any extra hidden behavior

IRus

IRus commented on Jul 17, 2025

@IRus
Author

Another nice property of runSuspend - it can be implemented without kotlinx.corountes, so JUnit wouldn't imply runtime dependency on kotlinx-coroutines-core. Which of course likely be present for people that want to use suspend tests, but not having plus one dependency is usually good thing

marcphilipp

marcphilipp commented on Jul 17, 2025

@marcphilipp
Member

Let's please continue the discussion in #4768

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Support test methods with Kotlin suspend modifier · Issue #1914 · junit-team/junit-framework