Skip to content

Support JSON data type #698

Open
Open
@tilschuenemann

Description

@tilschuenemann

First of all: Thank you for maintaining this project, it really is a nice addition to pandas!

Is your feature request related to a problem? Please describe.
GBQ supports the JSON data type, but unfortunately pandas-gbq doesn't allow writing (and I believe reading) from a table that features a JSON data type column.

Describe the solution you'd like

  1. Add support for both reading and writing dataframes that feature JSON columns.
  2. Add support for manually specifying JSON columns in the table schema.

Describe alternatives you've considered
Alternatives are:

  • writing data as a string and then loading it as a JSON inside a SQL query to access its key-values
    • the downside here is that the json object can't be used for early query processing, introducing the conversion overhead
  • using the native google.cloud.bigquery module

Activity

added
type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.
on Nov 14, 2023
subramad

subramad commented on Apr 24, 2024

@subramad

Is there any update on this?

tswast

tswast commented on Jul 18, 2024

@tswast
Collaborator

@chelsea-lin is working on a JSON dtype for https://github.com/googleapis/python-db-dtypes-pandas which is blocking this for read_gbq support. Otherwise, we'd end up returning strings (see: googleapis/python-bigquery#1876 (comment)).

Regarding loads via to_gbq, I believe BigQuery's parquet load jobs don't support JSON columns in the table either, so we'll need to address this as well.

tanjt107

tanjt107 commented on Apr 23, 2025

@tanjt107

It seems that this issue has been resolved natively on the GBQ side.
googleapis/python-db-dtypes-pandas#284

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the googleapis/python-bigquery-pandas API.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @tswast@Linchin@tilschuenemann@tanjt107@subramad

      Issue actions

        Support JSON data type · Issue #698 · googleapis/python-bigquery-pandas