Skip to content

proposal: encoding/json: allow returning nil from MarshalJSON to omit the field #50480

Closed
@mitar

Description

@mitar

Currently even if one implements MarshalJSON on a struct there is no way to prevent its inclusion into the parent struct. E.g.:

type Foo struct {
	Field Field `json:"field"`
}

type Field struct {
	DataPublic bool
	Value      interface{}
}

func (f Field) MarshalJSON() ([]byte, error) {
	if f.DataPublic {
		return json.Marshal(f.Value)
	} else {
		return []byte("null"), nil
	}
}

(Full example: https://go.dev/play/p/A0wKfuZgURw)

The best one can do is return null. But what if one wants to fully omit the field? This is not possible. Or one has to implement MarshalJSON on the parent struct, but that is not nice from composability perspective (one would have to implement this for every struct which includes Field).

I propose that it should be allowed to return nil, nil from MarshalJSON to signal that the field should be omitted in the parent struct. Currently returning nil, nil does fails with error unexpected end of JSON input because the resulting JSON is malformed. That means that nobody can rely on this behavior really for correct JSON output (maybe only to catch errors?), so I do not think we would be breaking any real backwards compatibility if we introduce this, or at least I believe benefits are bigger here. In any case, if you want previous behavior, you can return []byte{} and then it will continue to error in the same way. Doing JSON marshal of a struct itself which returns nil is compatible enough in my view: if caller just assumes they are getting []byte slice, nil will mostly behave for them as an empty slice. If they are able to handle it anyway.

Implementing this would also provide a solution for the issue that omitempty does not work with structs: one could implement MarshalJSON which checks if the value is zero and return nil if it is so. This would be useful only when implementor of the type is the user of it at the same time, but one can always make a new type as a user and then implement MarshalJSON there. I think it is an useful step in addressing that.

Returning nil to omit the field seems to be also something others expected to work and has been also proposed in the past, but inside a bigger discussion, so this can serve as a proposal for just this feature.

Activity

added this to the Proposal milestone on Jan 6, 2022
changed the title [-]proposal: encoding/json: Allow returning nil from MarshalJSON to omit the field[/-] [+]proposal: encoding/json: allow returning nil from MarshalJSON to omit the field[/+] on Jan 7, 2022
earthboundkid

earthboundkid commented on Jan 18, 2022

@earthboundkid
Contributor

I worry this could result in an accidental omission. How about return nil, json.OmitFieldErr instead?

LittleFox94

LittleFox94 commented on Apr 28, 2022

@LittleFox94

What is the state for this, would a PR for this be accepted?

I really like the return nil, json.OmitFieldErr idea, though it should be either ErrOmitField to match conventions or strip the Err completely - maybe json.OmitFieldHint or something?

LittleFox94

LittleFox94 commented on Apr 28, 2022

@LittleFox94

I just saw this comment is created from one of the ideas #11939, explicitly for return nil, nil and return nil, json.$someErr is one of the other proposals in that original issue.

icholy

icholy commented on Apr 30, 2022

@icholy

@mitar what happens if MarshalJSON returns nil without omitempty.

type Nil struct{}

func (Nil) MarshalJSON() ([]byte, error) {
	return nil, nil
}

func main() {
	data, _ := json.Marshal(Nil{})
	fmt.Println(string(data)) // what's the expected output?
}
icholy

icholy commented on Apr 30, 2022

@icholy

@mitar I implemented your idea here https://github.com/icholy/json if you want to play with it https://go.dev/play/p/7Cf6BgrNGpP

edit: I've also implemented the err_empty branch which requires the error return. Ex:

type MarshalNil struct{}

func (MarshalNil) MarshalJSON() ([]byte, error) {
	return []byte("null"), json.ErrEmpty
}

This type will marshal to null unless it's a struct field with omitempty set.

smikulcik

smikulcik commented on May 9, 2022

@smikulcik

Regarding #50480 (comment)

Wouldn't nil, nil yield the invalid JSON of empty string?

rsc

rsc commented on Jun 1, 2022

@rsc
Contributor

This proposal is a duplicate of a previously discussed proposal, as noted above,
and there is no significant new information to justify reopening the discussion.
The issue has therefore been declined as a duplicate.
— rsc for the proposal review group

mitar

mitar commented on Jun 1, 2022

@mitar
ContributorAuthor

Sorry, where exactly has this already been discussed? So I found a mention in one long issue (on omitempty and structs), a much broader issue, where this was proposed but not acted upon, while this issue itself tries to be focused on exactly one proposal. So which discussion is this reopening? This is just trying to focus the discussion which have not happened. (And this issue has also 14 upvotes.)

joeshaw

joeshaw commented on Jun 1, 2022

@joeshaw
Contributor

@mitar #45669 is the one that's been put on the docket for this.

There are lots of different proposals to address the same underlying problem, but it's not clear yet which is the best approach to take. Hopefully discussion there will address it.

mitar

mitar commented on Jun 1, 2022

@mitar
ContributorAuthor

Hm, I saw that one, but I disagree that this one would be addressed with omitzero. But the opposite holds true: implementing this proposal would also provide a way to solve the issue which omitzero attempts to address. But omitzero (nor omitempty) do not influence JSON marshaling when struct implements MarshalJSON. And this issue is about that. If a struct has MarshalJSON, then you cannot omit it anymore. omitzero will not fix this.

moved this to Declined in Proposalson Aug 10, 2022
locked and limited conversation to collaborators on Jun 1, 2023
removed this from Proposalson Jun 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @joeshaw@rsc@earthboundkid@mitar@icholy

        Issue actions

          proposal: encoding/json: allow returning nil from MarshalJSON to omit the field · Issue #50480 · golang/go