Exception type changed on master

[jakobnissen]

The following code:

hcat(reshape([1, 2], 2, 1), [1])

Throws an ArgumentError on 1.10.0-alpha1, but a DimensionMismatch on master. This breaks BioSequences.jl's tests which relies on @test_throws ArgumentError ....

The change happened in #49322

[timholy]

I have the vague sense (could be misremembering) that changing error types is considered acceptable breakage, and in this case DimensionMismatch does seem more appropriate.

Really, the better way to write tests is @test_throws "some text from the error" f(args...). That doesn't fix all problems, but it's more robust and better at testing what we actually want to deliver to the user.

[jakobnissen]

I agree that DimensionMismatch is more appropriate, and in this case I can easily fix the particular test in question.
But I must admit, I find it increasingly frustrating and disappointing how difficult Julia makes it to find out what is reliable, intentional behaviour, and to write Julia code that doesn't randomly break over time with new releases of Julia, despite them nominally being non-breaking. Neither I nor Tim are exactly Julia newbies, so the fact that neither of us actually knew for sure whether (the rather fundamental) @test_throws can be relied on is telling. And I'm not going to feel more confident in @test_throws "some test" f() is going to be less vulnerable to random breakage either - presumably changing the text of an exception in a minor release is fair game, too.

And what is the implication of this? That exception handling using try should not test for the exception type, as demonstrated in the manual? That @test_throws ought to only be written @test_throws Exception f() if you don't know for sure that f will only throw errors from your own code?

Anyway. Perhaps instead of griping here, I should make PRs to the manual and the docstring of @test_throws not to use it for exceptions that are not under your own control.