proposal: encoding/json: allow returning nil from MarshalJSON to omit the field

[mitar]

Currently even if one implements MarshalJSON on a struct there is no way to prevent its inclusion into the parent struct. E.g.:

type Foo struct {
	Field Field `json:"field"`
}

type Field struct {
	DataPublic bool
	Value      interface{}
}

func (f Field) MarshalJSON() ([]byte, error) {
	if f.DataPublic {
		return json.Marshal(f.Value)
	} else {
		return []byte("null"), nil
	}
}

(Full example: https://go.dev/play/p/A0wKfuZgURw)

The best one can do is return null. But what if one wants to fully omit the field? This is not possible. Or one has to implement MarshalJSON on the parent struct, but that is not nice from composability perspective (one would have to implement this for every struct which includes Field).

I propose that it should be allowed to return nil, nil from MarshalJSON to signal that the field should be omitted in the parent struct. Currently returning nil, nil does fails with error unexpected end of JSON input because the resulting JSON is malformed. That means that nobody can rely on this behavior really for correct JSON output (maybe only to catch errors?), so I do not think we would be breaking any real backwards compatibility if we introduce this, or at least I believe benefits are bigger here. In any case, if you want previous behavior, you can return []byte{} and then it will continue to error in the same way. Doing JSON marshal of a struct itself which returns nil is compatible enough in my view: if caller just assumes they are getting []byte slice, nil will mostly behave for them as an empty slice. If they are able to handle it anyway.

Implementing this would also provide a solution for the issue that omitempty does not work with structs: one could implement MarshalJSON which checks if the value is zero and return nil if it is so. This would be useful only when implementor of the type is the user of it at the same time, but one can always make a new type as a user and then implement MarshalJSON there. I think it is an useful step in addressing that.

Returning nil to omit the field seems to be also something others expected to work and has been also proposed in the past, but inside a bigger discussion, so this can serve as a proposal for just this feature.

[earthboundkid]

I worry this could result in an accidental omission. How about return nil, json.OmitFieldErr instead?

[LittleFox94]

What is the state for this, would a PR for this be accepted?

I really like the return nil, json.OmitFieldErr idea, though it should be either ErrOmitField to match conventions or strip the Err completely - maybe json.OmitFieldHint or something?

[LittleFox94]

I just saw this comment is created from one of the ideas #11939, explicitly for return nil, nil and return nil, json.$someErr is one of the other proposals in that original issue.

[icholy]

@mitar what happens if MarshalJSON returns nil without omitempty.

type Nil struct{}

func (Nil) MarshalJSON() ([]byte, error) {
	return nil, nil
}

func main() {
	data, _ := json.Marshal(Nil{})
	fmt.Println(string(data)) // what's the expected output?
}

[icholy]

@mitar I implemented your idea here https://github.com/icholy/json if you want to play with it https://go.dev/play/p/7Cf6BgrNGpP

edit: I've also implemented the err_empty branch which requires the error return. Ex:

type MarshalNil struct{}

func (MarshalNil) MarshalJSON() ([]byte, error) {
	return []byte("null"), json.ErrEmpty
}

This type will marshal to null unless it's a struct field with omitempty set.

[smikulcik]

Regarding #50480 (comment)

Wouldn't nil, nil yield the invalid JSON of empty string?

[rsc]

This proposal is a duplicate of a previously discussed proposal, as noted above,
and there is no significant new information to justify reopening the discussion.
The issue has therefore been declined as a duplicate.
— rsc for the proposal review group

[mitar]

Sorry, where exactly has this already been discussed? So I found a mention in one long issue (on omitempty and structs), a much broader issue, where this was proposed but not acted upon, while this issue itself tries to be focused on exactly one proposal. So which discussion is this reopening? This is just trying to focus the discussion which have not happened. (And this issue has also 14 upvotes.)

[joeshaw]

@mitar #45669 is the one that's been put on the docket for this.

There are lots of different proposals to address the same underlying problem, but it's not clear yet which is the best approach to take. Hopefully discussion there will address it.

[mitar]

Hm, I saw that one, but I disagree that this one would be addressed with omitzero. But the opposite holds true: implementing this proposal would also provide a way to solve the issue which omitzero attempts to address. But omitzero (nor omitempty) do not influence JSON marshaling when struct implements MarshalJSON. And this issue is about that. If a struct has MarshalJSON, then you cannot omit it anymore. omitzero will not fix this.