Hom: introduce a check argument to simplify the unpickling detection logic

[nthiery]

As was noticed in #14793, some sanity checks need to be disabled in Hom when called upon unpickling because the input may include an unitialized parent. Since #14793, this is achieved by looking at the parent to detect if it is unitilialized.

As an alternative, this ticket proposes to add a check=True/False
optional argument to Hom, and to use it upon unpickling. The advantages are:

• The logic is quite simpler, slightly faster, and more robust.

• This check argument is of general purpose, and indeed immediately
  put to use when Hom calls itself recursively.

• It made it easier for the first feature below

A potential caveat: pickles created since #14793 might not unpickle
properly. This was very recent; do we care?

Other changes in this ticket:

• Hom now uses X in category as sanity check rather than
  X.category().is_subcategory(category). This is more expressive,
  and indeed a necessary preliminary step for Categories over a base ring category #15801 which makes those
  two idioms not always equivalent (example: X is a QQ-module,
  X.category() is Modules(Rings) and category is
  Modules(QQ))

• A bug fix for Homsets between simplicial complexes over some higher
  category (Hom was not checking its input).

CC: @simon-king-jena @tscrim

Component: categories

Keywords: homset

Author: Nicolas M. Thiéry, Simon King

Branch: e1e916c

Reviewer: Simon King, Nicolas M. Thiéry

Issue created by migration from https://trac.sagemath.org/ticket/16275

[nthiery]

Branch: u/nthiery/hom__introduce_a_check_argument_to_simplify_the_unpickling_detection_logic

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

1ff993b

#16275: trivial commit to use format

[sagetrac-git]

Commit: 1ff993b

[tscrim]

comment:3

I agree with these changes, although I don't understand pickling well enough to answer those issues. Simon, could you render judgement on this or clarify the issue?

[nthiery]

comment:4

Just in case this helps, here is an extract of how quotients of
multivariate polynomial rings are unpickled. We see in the
construction of si8 Hom being called on the parents si5 and si7, where
si5 (the resulting quotient ring) has been allocated by
unpickle_newobj, but not yet initialized by unpickle_build:

    sage: P.<x,y> = QQ['x,y']
    sage: Q = P.quotient([x^2-1,y^2-1])
    sage: Q.an_element()
    sage: explain_pickle(dumps(Q))
    ...
    si4 = pg_dynamic_class('QuotientRing_generic_with_category', (pg_QuotientRing_generic, pg_getattr(si3, 'parent_class')), None, None, pg_QuotientRing_generic)
    si5 = unpickle_newobj(si4, ())
    ...
    pg_Hom = unpickle_global('sage.categories.homset', 'Hom')
    pg_unpickle_MPolynomialRing_libsingular = unpickle_global('sage.rings.polynomial.multi_polynomial_libsingular', 'unpickle_MPolynomialRing_libsingular')
    ...
    si7 = pg_unpickle_MPolynomialRing_libsingular(pg1, ('x', 'y'), si6)
    ...
    si8 = pg_unpickle_map(pg_RingMap_lift, pg_Hom(si5, si7, pg_unreduce(pg_Sets, (), {}), False), {}, {'_codomain':si7, 'S':si7, '_domain':si5, '_repr_type_str':None})
    ...
    unpickle_build(si5, {'_QuotientRing_nc__lift':si8, '_embedding':None, '_initial_action_list':[], '_convert_method_name':None, '_QuotientRing_nc__cover':si9, '_category':si3, '_names':('xbar', 'ybar'), 'cover':pg_CachedMethodPickle(si5, 'cover', si9), 'lifting_map':pg_CachedMethodPickle(si5, 'lifting_map', si8), '_QuotientRing_nc__R':si7, 'element_class':pg_dynamic_class('QuotientRing_generic_with_category.element_class', (pg_QuotientRingElement, pg_getattr(si3, 'element_class')), None, None, None), '_generators':{}, '_QuotientRing_nc__I':si10, '_base':pg1, '_pickle_version':1r, '_element_constructor':pg_unpickleMethod('_element_constructor_', si5, si4), '_initial_convert_list':[], '_element_init_pass_parent':False, '_cdata':None, '_initial_coerce_list':[]})
    si5

[sagetrac-git]

Changed commit from 1ff993b to 0074f02

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

0074f02

#16275: added doctest illustrating an unpickling where Hom is called on an uninitialized parent

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

c1e91f3	#16275: update ticket number in the comments
ae3ca80	This is handled as well by the generic of Homset.__reduce__ method,

[sagetrac-git]

Changed commit from 0074f02 to ae3ca80

[nthiery]

comment:7

With the latest commit, all tests pass. Running the long tests now.