Closed
Description
Is your feature request related to a problem or challenge?
We are working to add complete StringView support in DataFusion, which permits potentially much faster processing of string data. See #10918 for more background.
Today, most DataFusion string functions support DataType::Utf8 and DataType::LargeUtf8 and when called with a StringView argument DataFusion will cast the argument back to DataType::Utf8 which is expensive.
To realize the full speed of StringView, we need to ensure that all string functions support the DataType::Utf8View directly.
Describe the solution you'd like
Port all string functions
- Implement native support StringView for character length #11676
- Initial support for regex_replace on
StringViewArray#11556 - Support
starts_withforUtf8View#11786 - Update the
ASCIIscalar function to supportUtf8View#11834 - Update the
BTRIMscalar function to supportUtf8View#11835 - Update the
CONCATscalar function to supportUtf8View#11836 - Update
concat_wsscalar function to supportUtf8View#11837 - Update
CONTAINSscalar function to supportUtf8View#11838 - Update
ENDS_WITHscalar function to supportUtf8View#11852 - Update
INITCAPscalar function to supportUtf8View#11853 - Update
levenshteinscalar function to supportUtf8View#11854 - Update
LOWERscalar function to supportUtf8View#11855 - Update
LTRIMscalar function to supportUtf8View#11856 - Update
LPADscalar function to supportUtf8View#11857 - Update
OCTET_LENGTHscalar function to supportUtf8View#11858 - Update
SPLIT_PARTscalar function to support Utf8View #11950 - Update
STRPOSscalar function to support Utf8View #11951 - Update
SUBSTRscalar function to support Utf8View #11952 - Update
TRANSLATEscalar function to support Utf8View #11953 - Update
FIND_IN_SETscalar function to support Utf8View #11954 - Implement native support StringView for
REPEAT#11962 - Support Utf8View for string function
bit_length#13195
Describe alternatives you've considered
No response
Additional context
See coordination plan with @tshauck and myself here: #11787 (comment)