Skip to content

[Marshaller] Introduce component #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

[Marshaller] Introduce component #4

wants to merge 1 commit into from

Conversation

mtarld
Copy link
Owner

@mtarld mtarld commented Mar 21, 2023

No description provided.

@mtarld mtarld force-pushed the marshaller branch 6 times, most recently from f598032 to e791c07 Compare March 22, 2023 08:24
@mtarld mtarld closed this Mar 22, 2023
mtarld added a commit that referenced this pull request Oct 9, 2023
mtarld pushed a commit that referenced this pull request Dec 12, 2023
…uration in PHPUnit (cafferata)

This PR was merged into the 5.4 branch.

Discussion
----------

[Mailer] Stop using the (local) AWS shared configuration in PHPUnit

| Q             | A
| ------------- | ---
| Branch?       | 5.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Issues        | -
| License       | MIT

Replaces symfony#52958

Found an (other) way to stop using the (local) [AWS shared configuration](https://async-aws.com/authentication/credentials-file.html) in the Composer package [async-aws/core](https://github.com/async-aws/core) in the PHPUnit tests. 🥳

_This was an great Symfony hackday pair session with `@TmEuMail`-2020._

Also big shout out to `@xabbuh` and `@OskarStark` for the heads up in the right direction! 👍

```shell
./phpunit src/Symfony/Component/Mailer
```

**Before**

<details>
    <summary>PHPUnit output</summary>

```shell
PHPUnit 9.6.15 by Sebastian Bergmann and contributors.

Testing /symfony/symfony/src/Symfony/Component/Mailer
F...F...F.F...F...F............................................  63 / 495 ( 12%)
............................................................... 126 / 495 ( 25%)
............................................................... 189 / 495 ( 38%)
............................................................... 252 / 495 ( 50%)
............................................................... 315 / 495 ( 63%)
............................................................... 378 / 495 ( 76%)
.......................................SSS...SS.............S.. 441 / 495 ( 89%)
..................SS..................................          495 / 495 (100%)

Time: 00:02.082, Memory: 24.00 MB

There were 6 failures:

1) Symfony\Component\Mailer\Bridge\Amazon\Tests\Transport\SesApiAsyncAwsTransportTest::testToString with data set #0 (Symfony\Component\Mailer\Bridge\Amazon\Transport\SesApiAsyncAwsTransport Object (...), 'ses+api://ACCESS_KEY@us-east-1')
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'ses+api://ACCESS_KEY@us-east-1'
+'ses+api://ACCESS_KEY@eu-west-1'

/symfony/symfony/src/Symfony/Component/Mailer/Bridge/Amazon/Tests/Transport/SesApiAsyncAwsTransportTest.php:34

2) Symfony\Component\Mailer\Bridge\Amazon\Tests\Transport\SesApiAsyncAwsTransportTest::testToString with data set #4 (Symfony\Component\Mailer\Bridge\Amazon\Transport\SesApiAsyncAwsTransport Object (...), 'ses+api://ACCESS_KEY@us-east-1')
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'ses+api://ACCESS_KEY@us-east-1'
+'ses+api://ACCESS_KEY@eu-west-1'

/symfony/symfony/src/Symfony/Component/Mailer/Bridge/Amazon/Tests/Transport/SesApiAsyncAwsTransportTest.php:34

3) Symfony\Component\Mailer\Bridge\Amazon\Tests\Transport\SesApiAsyncAwsTransportTest::testSend
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'https://email.us-east-1.amazonaws.com/v2/email/outbound-emails'
+'https://email.eu-west-1.amazonaws.com/v2/email/outbound-emails'

/symfony/symfony/src/Symfony/Component/Mailer/Bridge/Amazon/Tests/Transport/SesApiAsyncAwsTransportTest.php:79
/symfony/symfony/src/Symfony/Component/HttpClient/MockHttpClient.php:70
/symfony/symfony/vendor/async-aws/core/src/AbstractApi.php:161
/symfony/symfony/vendor/async-aws/ses/src/SesClient.php:67
/symfony/symfony/src/Symfony/Component/Mailer/Bridge/Amazon/Transport/SesHttpAsyncAwsTransport.php:55
/symfony/symfony/src/Symfony/Component/Mailer/Transport/AbstractTransport.php:69
/symfony/symfony/src/Symfony/Component/Mailer/Bridge/Amazon/Tests/Transport/SesApiAsyncAwsTransportTest.php:119

4) Symfony\Component\Mailer\Bridge\Amazon\Tests\Transport\SesHttpAsyncAwsTransportTest::testToString with data set #0 (Symfony\Component\Mailer\Bridge\Amazon\Transport\SesHttpAsyncAwsTransport Object (...), 'ses+https://ACCESS_KEY@us-east-1')
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'ses+https://ACCESS_KEY@us-east-1'
+'ses+https://ACCESS_KEY@eu-west-1'

/symfony/symfony/src/Symfony/Component/Mailer/Bridge/Amazon/Tests/Transport/SesHttpAsyncAwsTransportTest.php:34

5) Symfony\Component\Mailer\Bridge\Amazon\Tests\Transport\SesHttpAsyncAwsTransportTest::testToString with data set #4 (Symfony\Component\Mailer\Bridge\Amazon\Transport\SesHttpAsyncAwsTransport Object (...), 'ses+https://ACCESS_KEY@us-east-1')
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'ses+https://ACCESS_KEY@us-east-1'
+'ses+https://ACCESS_KEY@eu-west-1'

/symfony/symfony/src/Symfony/Component/Mailer/Bridge/Amazon/Tests/Transport/SesHttpAsyncAwsTransportTest.php:34

6) Symfony\Component\Mailer\Bridge\Amazon\Tests\Transport\SesHttpAsyncAwsTransportTest::testSend
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'https://email.us-east-1.amazonaws.com/v2/email/outbound-emails'
+'https://email.eu-west-1.amazonaws.com/v2/email/outbound-emails'

/symfony/symfony/src/Symfony/Component/Mailer/Bridge/Amazon/Tests/Transport/SesHttpAsyncAwsTransportTest.php:79
/symfony/symfony/src/Symfony/Component/HttpClient/MockHttpClient.php:70
/symfony/symfony/vendor/async-aws/core/src/AbstractApi.php:161
/symfony/symfony/vendor/async-aws/ses/src/SesClient.php:67
/symfony/symfony/src/Symfony/Component/Mailer/Bridge/Amazon/Transport/SesHttpAsyncAwsTransport.php:55
/symfony/symfony/src/Symfony/Component/Mailer/Transport/AbstractTransport.php:69
/symfony/symfony/src/Symfony/Component/Mailer/Bridge/Amazon/Tests/Transport/SesHttpAsyncAwsTransportTest.php:112

FAILURES!
Tests: 495, Assertions: 923, Failures: 6, Skipped: 8.
```

</details>

**After**

<details>
    <summary>PHPUnit output</summary>

```shell
PHPUnit 9.6.15 by Sebastian Bergmann and contributors.

Testing /symfony/symfony/src/Symfony/Component/Mailer
...............................................................  63 / 495 ( 12%)
............................................................... 126 / 495 ( 25%)
............................................................... 189 / 495 ( 38%)
............................................................... 252 / 495 ( 50%)
............................................................... 315 / 495 ( 63%)
............................................................... 378 / 495 ( 76%)
.......................................SSS...SS.............S.. 441 / 495 ( 89%)
..................SS..................................          495 / 495 (100%)

Time: 00:02.387, Memory: 24.00 MB

OK, but incomplete, skipped, or risky tests!
Tests: 495, Assertions: 943, Skipped: 8.
```

</details>

Commits
-------

b156c33 [Mailer] Stop using the (local) AWS shared configuration in the PHPUnit tests.
fabpot pushed a commit that referenced this pull request Feb 3, 2024
…read from socket (xdanik)

This PR was merged into the 5.4 branch.

Discussion
----------

[Mailer] Throw `TransportException` when unable to read from socket

| Q             | A
| ------------- | ---
| Branch?       | 5.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? |no
| Issues        | None
| License       | MIT

We are seeing error `fgets(): SSL: Connection reset by peer` multiple times a day from connection to Office 365 SMTP server (smtp.office365.com:587).
It's certainly related to some kind of timeout as we are sending emails from long running queue dispatcher and error shows up only occasionally and never with the first message. We are not seeing this issue with any other SMTP server, but we have not tested much past smtp.mandrillapp.com and local MailHog.

We have tried adjusting the `$pingThreshold` and `$restartThreshold` options, but without much success (well `$restartThreshold = 1` resolves the issue, but it also forces the transport to close connection after each message).

Stack trace:
```
#0 /var/www/vendor/symfony/mailer/Transport/Smtp/Stream/AbstractStream.php(77): fgets(Resource(stream))
#1 /var/www/vendor/symfony/mailer/Transport/Smtp/SmtpTransport.php(315): Symfony\Component\Mailer\Transport\Smtp\Stream\AbstractStream->readLine()
#2 /var/www/vendor/symfony/mailer/Transport/Smtp/SmtpTransport.php(181): Symfony\Component\Mailer\Transport\Smtp\SmtpTransport->getFullResponse()
#3 /var/www/vendor/symfony/mailer/Transport/Smtp/SmtpTransport.php(140): Symfony\Component\Mailer\Transport\Smtp\SmtpTransport->executeCommand("RSET
", Array(1))
#4 /var/www/vendor/symfony/mailer/Mailer.php(45): Symfony\Component\Mailer\Transport\Smtp\SmtpTransport->send(Object(Symfony\Component\Mime\Email), Null)
#5 (our queue dispatcher): Symfony\Component\Mailer\Mailer->send(Object(Symfony\Component\Mime\Email))
```

App is running on PHP 8.0.28 on Debian Linux x64, Mailer v5.4.22.

I would gladly written some tests for this, but I don't know how to simulate calls to low-level stream functions like fgets.

Commits
-------

44d5b57 [Mailer] Throw TransportException when unable to read from socket
mtarld pushed a commit that referenced this pull request Mar 17, 2024
…hen publishing a message. (jwage)

This PR was squashed before being merged into the 6.4 branch.

Discussion
----------

[Messenger] [Amqp] Handle AMQPConnectionException when publishing a message.

| Q             | A
| ------------- | ---
| Branch?       | 6.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Issues        | Fix symfony#36538 Fix symfony#48241
| License       | MIT

If you have a message handler that dispatches messages to another queue, you can encounter `AMQPConnectionException` with the message "Library error: a SSL error occurred" or "a socket error occurred"  depending on if you are using tls or not or if you are running behind a load balancer or not.

You can manually reproduce this issue by dispatching a message where the handler then dispatches another message to a different queue, then go to rabbitmq admin and close the connection manually, then dispatch another message and when the message handler goes to dispatch the other message, you will get this exception:

```
a socket error occurred
#0 /vagrant/vendor/symfony/amqp-messenger/Transport/AmqpTransport.php(60): Symfony\Component\Messenger\Bridge\Amqp\Transport\AmqpSender->send()
#1 /vagrant/vendor/symfony/messenger/Middleware/SendMessageMiddleware.php(62): Symfony\Component\Messenger\Bridge\Amqp\Transport\AmqpTransport->send()
#2 /vagrant/vendor/symfony/messenger/Middleware/FailedMessageProcessingMiddleware.php(34): Symfony\Component\Messenger\Middleware\SendMessageMiddleware->handle()
#3 /vagrant/vendor/symfony/messenger/Middleware/DispatchAfterCurrentBusMiddleware.php(61): Symfony\Component\Messenger\Middleware\FailedMessageProcessingMiddleware->handle()
#4 /vagrant/vendor/symfony/messenger/Middleware/RejectRedeliveredMessageMiddleware.php(41): Symfony\Component\Messenger\Middleware\DispatchAfterCurrentBusMiddleware->handle()
#5 /vagrant/vendor/symfony/messenger/Middleware/AddBusNameStampMiddleware.php(37): Symfony\Component\Messenger\Middleware\RejectRedeliveredMessageMiddleware->handle()
#6 /vagrant/vendor/symfony/messenger/Middleware/TraceableMiddleware.php(40): Symfony\Component\Messenger\Middleware\AddBusNameStampMiddleware->handle()
#7 /vagrant/vendor/symfony/messenger/MessageBus.php(70): Symfony\Component\Messenger\Middleware\TraceableMiddleware->handle()
#8 /vagrant/vendor/symfony/messenger/TraceableMessageBus.php(38): Symfony\Component\Messenger\MessageBus->dispatch()
#9 /vagrant/src/Messenger/MessageBus.php(37): Symfony\Component\Messenger\TraceableMessageBus->dispatch()
#10 /vagrant/vendor/symfony/mailer/Mailer.php(66): App\Messenger\MessageBus->dispatch()
symfony#11 /vagrant/src/Mailer/Mailer.php(83): Symfony\Component\Mailer\Mailer->send()
symfony#12 /vagrant/src/Mailer/Mailer.php(96): App\Mailer\Mailer->send()
symfony#13 /vagrant/src/MessageHandler/Trading/StrategySubscriptionMessageHandler.php(118): App\Mailer\Mailer->sendEmail()
symfony#14 /vagrant/src/MessageHandler/Trading/StrategySubscriptionMessageHandler.php(72): App\MessageHandler\Trading\StrategySubscriptionMessageHandler->handle()
symfony#15 /vagrant/vendor/symfony/messenger/Middleware/HandleMessageMiddleware.php(152): App\MessageHandler\Trading\StrategySubscriptionMessageHandler->__invoke()
symfony#16 /vagrant/vendor/symfony/messenger/Middleware/HandleMessageMiddleware.php(91): Symfony\Component\Messenger\Middleware\HandleMessageMiddleware->callHandler()
symfony#17 /vagrant/vendor/symfony/messenger/Middleware/SendMessageMiddleware.php(71): Symfony\Component\Messenger\Middleware\HandleMessageMiddleware->handle()
symfony#18 /vagrant/vendor/symfony/messenger/Middleware/FailedMessageProcessingMiddleware.php(34): Symfony\Component\Messenger\Middleware\SendMessageMiddleware->handle()
symfony#19 /vagrant/vendor/symfony/messenger/Middleware/DispatchAfterCurrentBusMiddleware.php(68): Symfony\Component\Messenger\Middleware\FailedMessageProcessingMiddleware->handle()
symfony#20 /vagrant/vendor/symfony/messenger/Middleware/RejectRedeliveredMessageMiddleware.php(41): Symfony\Component\Messenger\Middleware\DispatchAfterCurrentBusMiddleware->handle()
symfony#21 /vagrant/vendor/symfony/messenger/Middleware/AddBusNameStampMiddleware.php(37): Symfony\Component\Messenger\Middleware\RejectRedeliveredMessageMiddleware->handle()
symfony#22 /vagrant/vendor/symfony/messenger/Middleware/TraceableMiddleware.php(40): Symfony\Component\Messenger\Middleware\AddBusNameStampMiddleware->handle()
symfony#23 /vagrant/vendor/symfony/messenger/MessageBus.php(70): Symfony\Component\Messenger\Middleware\TraceableMiddleware->handle()
symfony#24 /vagrant/vendor/symfony/messenger/TraceableMessageBus.php(38): Symfony\Component\Messenger\MessageBus->dispatch()
symfony#25 /vagrant/vendor/symfony/messenger/RoutableMessageBus.php(54): Symfony\Component\Messenger\TraceableMessageBus->dispatch()
symfony#26 /vagrant/vendor/symfony/messenger/Worker.php(162): Symfony\Component\Messenger\RoutableMessageBus->dispatch()
symfony#27 /vagrant/vendor/symfony/messenger/Worker.php(109): Symfony\Component\Messenger\Worker->handleMessage()
symfony#28 /vagrant/vendor/symfony/messenger/Command/ConsumeMessagesCommand.php(238): Symfony\Component\Messenger\Worker->run()
symfony#29 /vagrant/vendor/symfony/console/Command/Command.php(326): Symfony\Component\Messenger\Command\ConsumeMessagesCommand->execute()
symfony#30 /vagrant/vendor/symfony/console/Application.php(1096): Symfony\Component\Console\Command\Command->run()
symfony#31 /vagrant/vendor/symfony/framework-bundle/Console/Application.php(126): Symfony\Component\Console\Application->doRunCommand()
symfony#32 /vagrant/vendor/symfony/console/Application.php(324): Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand()
symfony#33 /vagrant/vendor/symfony/framework-bundle/Console/Application.php(80): Symfony\Component\Console\Application->doRun()
symfony#34 /vagrant/vendor/symfony/console/Application.php(175): Symfony\Bundle\FrameworkBundle\Console\Application->doRun()
symfony#35 /vagrant/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php(49): Symfony\Component\Console\Application->run()
symfony#36 /vagrant/vendor/autoload_runtime.php(29): Symfony\Component\Runtime\Runner\Symfony\ConsoleApplicationRunner->run()
symfony#37 /vagrant/bin/console(11): require_once('...')
symfony#38 {main}
```

TODO:

- [x] Add test for retry logic when publishing messages

Commits
-------

f123370 [Messenger] [Amqp] Handle AMQPConnectionException when publishing a message.
mtarld pushed a commit that referenced this pull request Aug 14, 2024
…rsimpsons)

This PR was merged into the 5.4 branch.

Discussion
----------

[Yaml] 🐛 throw ParseException on invalid date

| Q             | A
| ------------- | ---
| Branch?       | 5.4 <!-- see below -->
| Bug fix?      | yes
| New feature?  | no <!-- please update src/**/CHANGELOG.md files -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Issues        | None <!-- prefix each issue number with "Fix #", no need to create an issue if none exists, explain below instead -->
| License       | MIT

(found in symfony-tools/docs-builder#179)

When parsing the following yaml:
```
date: 6418-75-51
```

`symfony/yaml` will throw an exception:
```
$ php main.php
PHP Fatal error:  Uncaught Exception: Failed to parse time string (6418-75-51) at position 6 (5): Unexpected character in /tmp/symfony-yaml/vendor/symfony/yaml/Inline.php:714
Stack trace:
#0 /tmp/symfony-yaml/vendor/symfony/yaml/Inline.php(714): DateTimeImmutable->__construct()
#1 /tmp/symfony-yaml/vendor/symfony/yaml/Inline.php(312): Symfony\Component\Yaml\Inline::evaluateScalar()
#2 /tmp/symfony-yaml/vendor/symfony/yaml/Inline.php(80): Symfony\Component\Yaml\Inline::parseScalar()
#3 /tmp/symfony-yaml/vendor/symfony/yaml/Parser.php(790): Symfony\Component\Yaml\Inline::parse()
#4 /tmp/symfony-yaml/vendor/symfony/yaml/Parser.php(341): Symfony\Component\Yaml\Parser->parseValue()
#5 /tmp/symfony-yaml/vendor/symfony/yaml/Parser.php(86): Symfony\Component\Yaml\Parser->doParse()
#6 /tmp/symfony-yaml/vendor/symfony/yaml/Yaml.php(77): Symfony\Component\Yaml\Parser->parse()
#7 /tmp/symfony-yaml/main.php(8): Symfony\Component\Yaml\Yaml::parse()
#8 {main}
  thrown in /tmp/symfony-yaml/vendor/symfony/yaml/Inline.php on line 714
```

This is because the "month" is invalid. Fixing the "month" will trigger about the same issue because the "day" would be invalid.

With the current change it will throw a `ParseException`.

Commits
-------

6d71a7e 🐛 throw ParseException on invalid date
@mtarld mtarld deleted the marshaller branch November 15, 2024 10:38
mtarld pushed a commit that referenced this pull request Dec 3, 2024
… not throw exception (lyrixx)

This PR was merged into the 5.4 branch.

Discussion
----------

[HttpKernel] Ensure `HttpCache::getTraceKey()` does not throw exception

| Q             | A
| ------------- | ---
| Branch?       | 5.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Issues        | -
| License       | MIT

We have such logs in our logs. It's in our raw PHP logs. They are not caught by monolog, it's too early

```
[11-Oct-2024 01:23:33 UTC] PHP Fatal error:  Uncaught Symfony\Component\HttpFoundation\Exception\SuspiciousOperationException: Invalid method override "__CONSTRUCT". in /var/www/redirection.io/backend/blue/vendor/symfony/http-foundation/Request.php:1234
Stack trace:
#0 /var/www/redirection.io/backend/blue/vendor/symfony/http-kernel/HttpCache/HttpCache.php(728): Symfony\Component\HttpFoundation\Request->getMethod()
#1 /var/www/redirection.io/backend/blue/vendor/symfony/http-kernel/HttpCache/HttpCache.php(207): Symfony\Component\HttpKernel\HttpCache\HttpCache->getTraceKey()
#2 /var/www/redirection.io/backend/blue/vendor/symfony/http-kernel/Kernel.php(188): Symfony\Component\HttpKernel\HttpCache\HttpCache->handle()
#3 /var/www/redirection.io/backend/blue/web/app.php(9): Symfony\Component\HttpKernel\Kernel->handle()
#4 {main}
  thrown in /var/www/redirection.io/backend/blue/vendor/symfony/http-foundation/Request.php on line 1234

```

I managed to reproduced locally.
* Before the patch, without the http_cache, symfony returns a 405
* After the patch, without the http_cache, symfony returns a 405
* Before the patch, with the http_cache, symfony returns a 500, without any information (too early)
* After the patch, with the http_cache, symfony returns a 405

Commits
-------

a2ebbe0 [HttpKernel] Ensure HttpCache::getTraceKey() does not throw exception
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant