Skip to content

JoinNullable deserialisation fix #70

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 28, 2025

Conversation

NPCRUS
Copy link
Contributor

@NPCRUS NPCRUS commented Feb 25, 2025

In the current implementation, JoinNullable in order to determine that R shouldBe Option or Some is deserialising whole set and then counting nulls. In cases where you have user written TypeMapper, that doesn't do nulls tricks(like inside Dialects.scala), such:

enum Enumeration {
 case A, B, C
}
object Enumeration {
 given TypeMapper[Enumeration] = StringType.bimap(_.toString(), Enumeration.valueOf)
}

then leftJoin query, where the right side can be nullable instead of returning (A, None) will fail with exception, check new test in DataTypesTests

new implementation will check first if the rest of columns are null without deserialising them into type and only if one of them is not - will try to construct

comments on everything are welcome!

fixes #65

@lihaoyi
Copy link
Member

lihaoyi commented Feb 26, 2025

Looks reasonable to me, just run the autofix commands then we can merge this I think

- generate doc
@NPCRUS
Copy link
Contributor Author

NPCRUS commented Feb 28, 2025

@lihaoyi ping

@lihaoyi lihaoyi merged commit b6f864f into com-lihaoyi:main Feb 28, 2025
6 checks passed
@lihaoyi
Copy link
Member

lihaoyi commented Feb 28, 2025

Thanks! Feel free to ping me when you're done with your PRs and I can cut a stable version for you to depend on

@NPCRUS NPCRUS deleted the JoinNullable-deserialization-fix branch March 8, 2025 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

on leftJoin TypeMapper for T is being applied on Option[T]
4 participants