Skip to content

BUG: to_dict() unexpectedly casts np.float32 to python float #50125

Closed
@ghost

Description

EDIT: After more investigation, I found out the root cause of the issue described below: #50125 (comment)

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np

np.random.seed(42)
df = pd.DataFrame(np.random.rand(2,2).astype(np.float32))
print("original float32 data:\n", df.to_dict())
df = df.round(3)
print("to_records() after rounding:\n", df.to_records())
print("to_dict() after rounding:\n", df.to_dict())

Issue Description

The reproducible example returns:

original float32 data:
 {0: {0: 0.3745401203632355, 1: 0.7319939136505127}, 1: {0: 0.9507142901420593, 1: 0.5986585021018982}}
to_records() after rounding:
 [(0, 0.375, 0.951) (1, 0.732, 0.599)]
to_dict() after rounding:
 {0: {0: 0.375, 1: 0.7319999933242798}, 1: {0: 0.9509999752044678, 1: 0.5989999771118164}}

We can see that .round(3) has been properly carried through .to_records() but not through .to_dict().
to_dict() does not return the original data either, but something like the rounding result with extra decimals.

This does not happens if we use np.float64.

(Similar to issue #35124)

Expected Behavior

Result when using np.float64:

original float64 data:
 {0: {0: 0.3745401188473625, 1: 0.7319939418114051}, 1: {0: 0.9507143064099162, 1: 0.5986584841970366}}
to_records() after rounding:
 [(0, 0.375, 0.951) (1, 0.732, 0.599)]
to_dict() after rounding:
 {0: {0: 0.375, 1: 0.732}, 1: {0: 0.951, 1: 0.599}}

Installed Versions

INSTALLED VERSIONS

commit : 8dab54d
python : 3.10.8.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19042
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 10, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United Kingdom.1252

pandas : 1.5.2
numpy : 1.23.5
pytz : 2022.6
dateutil : 2.8.2
setuptools : 65.5.1
pip : 22.3.1
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : 8.7.0
pandas_datareader: None
bs4 : None
bottleneck : None
brotli :
fastparquet : None
fsspec : 2022.11.0
gcsfs : None
matplotlib : 3.6.2
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 9.0.0
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.9.3
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : 2022.7

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugIO DataIO issues that don't fit into a more specific label

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions