BUG: colon in URL gets cut off during call of to_html() when applying format(hyperlinks='html')

[thevoiddancer]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
df = pd.DataFrame([['www.google.com:80']])
styler = df.style.format(hyperlinks='html')
print(styler.to_html())

Issue Description

User wants to use format(hyperlinks='html') to automatically convert hyperlinks-styled text (starting with http:, https: or www.) to a hyperlinks. However, the URL contains a colon which gets cut off during the conversion, in both the hyperlink href and the text.

I've located the source of error to _render_href method, more specifically to the regex used. Included is the demonstration of the cutting off using that regex.

Also included is the demonstration of a fix, adding a colon:

Same issue happens with other characters such as # or + (I didn't check for others)

Expected Behavior

Input string is:
'www.google.com:80'

Expected output is:
www.google.com:80

Current output is:
www.google.com:80

Installed Versions

INSTALLED VERSIONS

commit : 06d2301
python : 3.8.10.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19042
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : Croatian_Croatia.1252

pandas : 1.4.1
numpy : 1.21.2
pytz : 2021.1
dateutil : 2.8.2
pip : 21.1.1
setuptools : 56.0.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 3.0.3
IPython : 8.1.1
pandas_datareader: None
bs4 : 4.10.0
bottleneck : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.5.1
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : 1.4.25
tables : None
tabulate : 0.8.9
xarray : None
xlrd : None
xlwt : None
zstandard : None

[attack68]

seems like a valid concern. PR is welcome. :)

note this stakcoverlfow might be relevant here:

All the gory details can be found in the current RFC on the topic: RFC 3986 (Uniform Resource Identifier (URI): Generic >Syntax)

Based on this related answer, you are looking at a list that looks like: A-Z, a-z, 0-9, -, ., _, ~, :, /, ?, #, [, ], @, !, $, &, ', (, ), *, +, ,, ;, %, and =. Everything else must be url-encoded. Also, some of these characters can only exist in very specific spots in a URI and outside of those spots must be url-encoded (e.g. % can only be used in conjunction with url encoding as in %20), the RFC has all of these specifics.

[demonictoaster]

According to RFC3986 (see pages 11-12), the following characters are allowed in a URI:

Reserved Characters

reserved = gen-delims / sub-delims
gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@"
sub-delims = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "="

Unreserved Characters

unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"

I am happy to extend the regex pattern in the _render_href method to capture URLs containing any of the above-mentioned characters.

[kianelbo]

I spent sometime tinkering with the regex and seems that the perfect regex beating every corner case almost doesn't exist. However, I think adding the aforementioned characters to the last capturing group of the regex, significantly expands its coverage, so I decided to open a PR. I'd be glad if you take a look at it.
Btw I'm a total newbie, so I apologize if I've handled it too naively.

[thevoiddancer]

I spent sometime tinkering with the regex and seems that the perfect regex beating every corner case almost doesn't exist. However, I think adding the aforementioned characters to the last capturing group of the regex, significantly expands its coverage, so I decided to open a PR. I'd be glad if you take a look at it. Btw I'm a total newbie, so I apologize if I've handled it too naively.

That would be my solution as well (you beat me to it), with the addition that I would probably add the same characters to the first group as well, perhaps too naively.

[kianelbo]

That would be my solution as well (you beat me to it), with the addition that I would probably add the same characters to the first group as well, perhaps too naively.

Sorry @thevoiddancer I thought you are not going to submit a PR :(
And thanks for the suggestion. According to rfc1738, the net_loc portion of urls can be like user:password@host. So I'll add : and @ to the middle group, but I don't think adding other characters such as # to this part would be a good idea.