Event System for extending PHPUnit

[sebastianbergmann]

History and Background

The TestListener Interface

First, there was the PHPUnit\Framework\TestListener interface:

interface TestListener
{
    public function addError(Test $test, Throwable $t, float $time): void;

    public function addWarning(Test $test, Warning $e, float $time): void;

    public function addFailure(Test $test, AssertionFailedError $e, float $time): void;

    public function addIncompleteTest(Test $test, Throwable $t, float $time): void;

    public function addRiskyTest(Test $test, Throwable $t, float $time): void;

    public function addSkippedTest(Test $test, Throwable $t, float $time): void;

    public function startTestSuite(TestSuite $suite): void;

    public function endTestSuite(TestSuite $suite): void;

    public function startTest(Test $test): void;

    public function endTest(Test $test, float $time): void;
}

The TestListener interface violates the Interface Segregation Principle, the "I" in "SOLID". This means that it requires the implementation of many methods, even if the client is not interested in the events they represent.

But the TestListener interface also has a fundamental design flaw: it passes around the Test and TestSuite objects, allowing clients to manipulate the outcome of a test run, for instance. To make things worse, client implementations of TestListener exist that bypass the public API of Test and TestSuite to perform such manipulation. In short, TestListener implementations were use to do more than "just listen".

The TestListener interface is deprecated since PHPUnit 8, it will be removed in PHPUnit 10.

The Hook Interfaces

Back in 2018 and with PHPUnit 7.1, we attempted to make extending PHPUnit's test runner easier with the introduction of the PHPUnit\Runner\Hook interfaces. Here is an example of one of these interfaces:

interface AfterSuccessfulTestHook
{
    public function executeAfterSuccessfulTest(string $test, float $time): void;
}

As you can see, we learned from the painful experience we made with the TestListener interface. The PHPUnit\Runner\Hook interfaces follow the Interface Segregation Principle and clients no longer get access to the real test objects that are used by PHPUnit to run the tests.

However, we did not "think big enough" and the PHPUnit\Runner\Hook interfaces were too limited. Even we did not manage to migrate PHPUnit's own TestListener implementations to the PHPUnit\Runner\Hook interfaces. They were a step in the right direction, but they fell short of actually being useful.

The PHPUnit\Runner\Hook interfaces will be removed in PHPUnit 11.

The New Event System

At the EU-FOSSA 2 Hackathon in October 2019, Sebastian Bergmann, Ewout Pieter den Ouden, Andreas Möller, Arne Blankerts, and Stefan Priebsch got together and designed a new system for extending PHPUnit based on events.

---

Photo: Arne Blankerts, Andreas Möller, and Ewout Pieter den Ouden work on the new event system for PHPUnit

---

Since then, Arne Blankerts and Andreas Möller worked on implementing this new event-based system in a branch. This issue is created as we are getting close to be able to merge this branch.

[Naktibalda]

Hi Sebastian,

Factory methods for value objects of TestSuite and TestCase classes make TestCase and TestSuite non-extensible.

phpunit/src/Event/Value/TestSuite/TestSuite.php

Lines 55 to 145 in 1c56cf3

	public static function fromTestSuite(FrameworkTestSuite $testSuite): self
	{
	$groups = [];
	foreach ($testSuite->getGroupDetails() as $groupName => $tests) {
	if (!isset($groups[$groupName])) {
	$groups[$groupName] = [];
	}
	foreach ($tests as $test) {
	$groups[$groupName][] = get_class($test);
	}
	}
	$tests = [];
	foreach ($testSuite->tests() as $test) {
	if ($test instanceof TestCase || $test instanceof PhptTestCase) {
	$tests[] = $test->valueObjectForEvents();
	}
	}
	if ($testSuite instanceof DataProviderTestSuite) {
	[$className, $methodName] = explode('::', $testSuite->getName());
	try {
	$reflector = new ReflectionMethod($className, $methodName);
	return new TestSuiteForTestMethodWithDataProvider(
	$testSuite->getName(),
	$testSuite->count(),
	$groups,
	$testSuite->provides(),
	$testSuite->requires(),
	$testSuite->sortId(),
	TestCollection::fromArray($tests),
	$testSuite->warnings(),
	$className,
	$methodName,
	$reflector->getFileName(),
	$reflector->getStartLine(),
	);
	// @codeCoverageIgnoreStart
	} catch (ReflectionException $e) {
	throw new Exception(
	$e->getMessage(),
	(int) $e->getCode(),
	$e
	);
	}
	// @codeCoverageIgnoreEnd
	}
	if (class_exists($testSuite->getName())) {
	try {
	$reflector = new ReflectionClass($testSuite->getName());
	return new TestSuiteForTestClass(
	$testSuite->getName(),
	$testSuite->count(),
	$groups,
	$testSuite->provides(),
	$testSuite->requires(),
	$testSuite->sortId(),
	TestCollection::fromArray($tests),
	$testSuite->warnings(),
	$reflector->getFileName(),
	$reflector->getStartLine(),
	);
	// @codeCoverageIgnoreStart
	} catch (ReflectionException $e) {
	throw new Exception(
	$e->getMessage(),
	(int) $e->getCode(),
	$e
	);
	}
	// @codeCoverageIgnoreEnd
	}
	return new TestSuiteWithName(
	$testSuite->getName(),
	$testSuite->count(),
	$groups,
	$testSuite->provides(),
	$testSuite->requires(),
	$testSuite->sortId(),
	TestCollection::fromArray($tests),
	$testSuite->warnings(),
	);
	}

phpunit/src/Event/Value/Test/TestMethod.php

Lines 43 to 67 in 1c56cf3

	public static function fromTestCase(TestCase $testCase): self
	{
	$className = $testCase::class;
	$methodName = $testCase->getName(false);
	$testData = self::dataFor($testCase);
	if ($testCase instanceof ErrorTestCase ||
	$testCase instanceof IncompleteTestCase ||
	$testCase instanceof SkippedTestCase ||
	$testCase instanceof WarningTestCase) {
	$className = $testCase->className();
	$methodName = $testCase->methodName();
	}
	$location = self::sourceLocationFor($className, $methodName);
	return new self(
	$className,
	$methodName,
	$location['file'],
	$location['line'],
	self::metadataFor($className, $methodName),
	$testData,
	);
	}

Would it be possible to implement mechanism for extending class to provide factory method for value objects?

[sebastianbergmann]

Please explain why you want to extend these classes.

[top-master]

No matter if hook, or event.

I don't think that any of this can help for situations like #4960

And should not be used as an excuse to close feature-requests.