PyStructSequence constructor ignores unknown field names

[serhiy-storchaka]

Bug report

PyStructSequence constructor takes two arguments: tuple and dict. The tuple specifies values for "visible" fields (i.e. what you get when interpret PyStructSequence as a tuple), the dict specifies values for fields that are accessible only by name (they are like __slot__ attributes).

>>> import time
>>> time.struct_time((2023, 10, 2, 17, 50, 53, 0, 275, 0), {'tm_zone': 'GMT', 'tm_gmtoff': 0})
time.struct_time(tm_year=2023, tm_mon=10, tm_mday=2, tm_hour=17, tm_min=50, tm_sec=53, tm_wday=0, tm_yday=275, tm_isdst=0)

The problem is that all invalid names are silently ignored. It includes names of "visible" fields and typos.

>>> time.struct_time((0,)*9, {'tm_zone': 'GMT', 'tm_gmtoff': 0, 'tm_year': 2023, 'invalid': 1})
time.struct_time(tm_year=0, tm_mon=0, tm_mday=0, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=0, tm_yday=0, tm_isdst=0)

Linked PRs

• gh-110235: Raise TypeError for duplicate/unknown fields in PyStructSequence constructor #110258

[serhiy-storchaka]

The original code was added by @mwhudson in bpo-526072 (#36205, ce358e3). Because it was only for pickling support, omitting some validity checks was acceptable.

[serhiy-storchaka]

@XuehaiPan, it may be interesting to you, since you already do something similar in the __replace__().

The difference is that this code cannot modify the input dict, because it is not kwargs. But it can count the found names.

[XuehaiPan]

The problem is that all invalid names are silently ignored. It includes names of "visible" fields and typos.

Any thoughts on allowing the present visible fields in dict to override the values in the input sequence? Currently, we ignore visible field names in dict.

PyStructSequence.__new__(cls: Self, sequence: Iterable[Any], dict: dict[str, Any]) -> Self

Since we need n_visible_fields <= len(sequence) <= n_fields in __new__(), all visible fields should be present in the sequence argument. If a field value is not allowed to come from both sequence and dict, the dict argument should only contain invisible field names. Also, we need to check if len(sequence) > n_visible_fields, the given invisible field provided in sequence should not present again in dict.

---

For collections.namedtuple, it will raise TypeError for duplicate arguments:

namedtuple.__new__(cls: Self, *args: Any, **kwargs: Any) -> Self

>>> import collections
>>> MyTuple = collections.namedtuple('MyTuple', ['a', 'b', 'c', 'd'])
>>> MyTuple(1, 2, 3, 4, a=5, b=6)
TypeError: MyTuple.__new__() got multiple values for argument 'a'

[XuehaiPan]

I opened a PR to resolve this.

• gh-110235: Raise TypeError for duplicate/unknown fields in PyStructSequence constructor #110258