diff --git a/doc/sphinx/guide/rangesets.rst b/doc/sphinx/guide/rangesets.rst index 0ff2c7e6..11880d40 100644 --- a/doc/sphinx/guide/rangesets.rst +++ b/doc/sphinx/guide/rangesets.rst @@ -14,23 +14,30 @@ RangeSet class -------------- The :class:`.RangeSet` class implements a mutable, ordered set of cluster node -indexes (one dimension) featuring a fast range-based API. This class is used -by the :class:`.NodeSet` class (see :ref:`class-NodeSet`). Since version 1.6, -:class:`.RangeSet` really derives from standard Python set class (`Python -sets`_), and thus provides methods like :meth:`.RangeSet.union`, +indexes (over a single dimension) featuring a fast range-based API. This class +is used by the :class:`.NodeSet` class (see :ref:`class-NodeSet`). Since +version 1.6, :class:`.RangeSet` actually derives from the standard Python set +class (`Python sets`_), and thus provides methods like :meth:`.RangeSet.union`, :meth:`.RangeSet.intersection`, :meth:`.RangeSet.difference`, :meth:`.RangeSet.symmetric_difference` and their in-place versions :meth:`.RangeSet.update`, :meth:`.RangeSet.intersection_update`, :meth:`.RangeSet.difference_update()` and :meth:`.RangeSet.symmetric_difference_update`. -Since v1.6, padding of ranges (eg. *003-009*) can be managed through a public -:class:`.RangeSet` instance variable named padding. It may be changed at any -time. Padding is a simple display feature per RangeSet object, thus current -padding value is not taken into account when computing set operations. Also -since v1.6, :class:`.RangeSet` is itself an iterator over its items as -integers (instead of strings). To iterate over string items as before (with -optional padding), you can now use the :meth:`.RangeSet.striter()` method. +In v1.9, the implementation of zero-based padding of indexes (e.g. `001`) has +been improved. The inner set contains indexes as strings with the padding +included, which allows the use of mixed length zero-padded indexes (eg. using +both `01` and `001` is valid and supported in the same object). Prior to v1.9, +zero-padding was a simple display feature of fixed length per +:class:`.RangeSet` object, and indexes where stored as integers in the inner +set. + +To iterate over indexes as strings with zero-padding included, you can now +iterate over the :class:`.RangeSet` object (:meth:`.RangeSet.__iter__()`), +or still use the :meth:`.RangeSet.striter()` method which has not changed. +To iterate over the set's indexes as integers, you may use the new method +:meth:`.RangeSet.intiter()`, which is the equivalent of iterating over the +:class:`.RangeSet` object before v1.9. .. _class-RangeSetND: @@ -47,6 +54,8 @@ tuples, for instance:: >>> from ClusterShell.RangeSet import RangeSet, RangeSetND >>> r1 = RangeSet("1-5/2") + >>> list(r1) + ['1', '3', '5'] >>> r2 = RangeSet("10-12") >>> r3 = RangeSet("0-4/2") >>> r4 = RangeSet("10-12") @@ -57,7 +66,8 @@ tuples, for instance:: 0-5; 10-12 >>> print list(rnd) - [(0, 10), (0, 11), (0, 12), (1, 10), (1, 11), (1, 12), (2, 10), (2, 11), (2, 12), (3, 10), (3, 11), (3, 12), (4, 10), (4, 11), (4, 12), (5, 10), (5, 11), (5, 12)] + [('0', '10'), ('0', '11'), ('0', '12'), ('1', '10'), ('1', '11'), ('1', '12'), ('2', '10'), ('2', '11'), ('2', '12'), ('3', '10'), ('3', '11'), ('3', '12'), ('4', '10'), ('4', '11'), ('4', '12'), ('5', '10'), ('5', '11'), ('5', '12')] + >>> r1 = RangeSetND([(0, 4), (0, 5), (1, 4), (1, 5)]) >>> len(r1) 4 @@ -70,7 +80,7 @@ tuples, for instance:: >>> str(r) '1; 4-5\n' >>> list(r) - [(1, 4), (1, 5)] + [('1', '4'), ('1', '5')] .. _Python sets: http://docs.python.org/library/sets.html diff --git a/doc/sphinx/tools/nodeset.rst b/doc/sphinx/tools/nodeset.rst index 1803aa49..1ab1b3b5 100644 --- a/doc/sphinx/tools/nodeset.rst +++ b/doc/sphinx/tools/nodeset.rst @@ -268,7 +268,7 @@ are automatically padded with zeros as well. For example:: $ nodeset -e node[08-11] node08 node09 node10 node11 - + $ nodeset -f node001 node002 node003 node005 node[001-003,005] @@ -278,23 +278,20 @@ also supported, for example:: $ nodeset -e node[000-012/4] node000 node004 node008 node012 -Nevertheless, care should be taken when dealing with padding, as a zero-padded -node name has priority over a normal one, for example:: +Since v1.9, mixed length padding is allowed, for example:: - $ nodeset -f node1 node02 - node[01-02] + $ nodeset -f node2 node01 node001 + node[2,01,001] -To clarify, *nodeset* will always try to coalesce node names by their -numerical index first (without taking care of any zero-padding), and then will -use the first zero-padding rule encountered. In the following example, the -first zero-padding rule found is *node01*'s one:: +When mixed length zero-padding is encountered, indexes with smaller padding +length are returned first, as you can see in the example above (``2`` comes +before ``01``). - $ nodeset -f node01 node002 - node[01-02] +Since v1.9, when using node sets with multiple dimensions, each dimension (or +axis) may also use mixed length zero-padding:: -That said, you can see it is not possible to mix *node01* and *node001* in the -same node set (not supported by the :class:`.NodeSet` class), but that would -be a tricky case anyway! + $ nodeset -f foo1bar1 foo1bar00 foo1bar01 foo004bar1 foo004bar00 foo004bar01 + foo[1,004]bar[1,00-01] Leading and trailing digits @@ -325,7 +322,13 @@ Examples with both bracket leading and trailing digits:: $ nodeset --autostep=auto -f node-00[1-6]0 node-[0010-0060/10] -Still, using this syntax can be error-prone especially if used with node sets +Example with leading digit and mixed length zero padding (supported since +v1.9):: + + $ nodeset -f node1[00-02,000-032/8] + node[100-102,1000,1008,1016,1024,1032] + +Using this syntax can be error-prone especially if used with node sets without 0-padding or with the */step* syntax and also requires additional processing by the parser. In general, we recommend writing the whole rangeset inside the brackets. @@ -350,7 +353,7 @@ the union operation will be computed, for example:: $ nodeset -f node[1-3] node[4-7] node[1-7] - + $ nodeset -f node[1-3] node[2-7] node[5-8] node[1-8] diff --git a/lib/ClusterShell/NodeSet.py b/lib/ClusterShell/NodeSet.py index cac5f66d..2127affe 100644 --- a/lib/ClusterShell/NodeSet.py +++ b/lib/ClusterShell/NodeSet.py @@ -165,44 +165,38 @@ def set_autostep(self, val): def _iter(self): """Iterator on internal item tuples - (pattern, indexes, padding, autostep).""" + (pattern, indexes, autostep).""" for pat, rset in sorted(self._patterns.items()): if rset: autostep = rset.autostep if rset.dim() == 1: assert isinstance(rset, RangeSet) - padding = rset.padding for idx in rset: - yield pat, (idx,), (padding,), autostep + yield pat, (idx,), autostep else: - for args, padding in rset.iter_padding(): - yield pat, args, padding, autostep + for rvec in rset: + yield pat, rvec, autostep else: - yield pat, None, None, None + yield pat, None, None def _iterbase(self): """Iterator on single, one-item NodeSetBase objects.""" - for pat, ivec, pad, autostep in self._iter(): + for pat, ivec, autostep in self._iter(): rset = None # 'no node index' by default if ivec is not None: assert len(ivec) > 0 if len(ivec) == 1: - rset = RangeSet.fromone(ivec[0], pad[0] or 0, autostep) + rset = RangeSet.fromone(ivec[0], autostep) else: - rset = RangeSetND([ivec], pad, autostep) + rset = RangeSetND([ivec], autostep) yield NodeSetBase(pat, rset) def __iter__(self): """Iterator on single nodes as string.""" # Does not call self._iterbase() + str() for better performance. - for pat, ivec, pads, _ in self._iter(): + for pat, ivec, _ in self._iter(): if ivec is not None: - # For performance reasons, add a special case for 1D RangeSet - if len(ivec) == 1: - yield pat % ("%0*d" % (pads[0] or 0, ivec[0])) - else: - yield pat % tuple(["%0*d" % (pad or 0, i) \ - for pad, i in zip(pads, ivec)]) + yield pat % ivec else: yield pat % () @@ -214,14 +208,13 @@ def __iter__(self): def nsiter(self): """Object-based NodeSet iterator on single nodes.""" - for pat, ivec, pads, autostep in self._iter(): + for pat, ivec, autostep in self._iter(): nodeset = self.__class__() if ivec is not None: if len(ivec) == 1: - pad = pads[0] or 0 - nodeset._add_new(pat, RangeSet.fromone(ivec[0], pad)) + nodeset._add_new(pat, RangeSet.fromone(ivec[0])) else: - nodeset._add_new(pat, RangeSetND([ivec], pads, autostep)) + nodeset._add_new(pat, RangeSetND([ivec], autostep)) else: nodeset._add_new(pat, None) yield nodeset @@ -1045,12 +1038,18 @@ def _scan_string(self, nsstr, autostep): pfxlen, sfxlen = len(pfx), len(sfx) if sfxlen > 0: - # amending trailing digits generates /steps - sfx, rng = self._amend_trailing_digits(sfx, rng) + try: + # amending trailing digits generates /steps + sfx, rng = self._amend_trailing_digits(sfx, rng) + except RangeSetParseError as ex: + raise NodeSetParseRangeError(ex) if pfxlen > 0: - # this method supports /steps - pfx, rng = self._amend_leading_digits(pfx, rng) + try: + # this method supports /steps + pfx, rng = self._amend_leading_digits(pfx, rng) + except RangeSetParseError as ex: + raise NodeSetParseRangeError(ex) if pfx: # scan any nonempty pfx as a single node (no bracket) pfx, pfxrvec = self._scan_string_single(pfx, autostep) diff --git a/lib/ClusterShell/RangeSet.py b/lib/ClusterShell/RangeSet.py index 884f5421..c44adf4d 100644 --- a/lib/ClusterShell/RangeSet.py +++ b/lib/ClusterShell/RangeSet.py @@ -71,8 +71,9 @@ class RangeSet(set): RangeSet basic constructors: >>> rset = RangeSet() # empty RangeSet - >>> rset = RangeSet("5,10-42") # contains 5, 10 to 42 - >>> rset = RangeSet("0-10/2") # contains 0, 2, 4, 6, 8, 10 + >>> rset = RangeSet("5,10-42") # contains '5', '10' to '42' + >>> rset = RangeSet("0-10/2") # contains '0', '2', '4', '6', '8', '10' + >>> rset = RangeSet("00-10/2") # contains '00', '02', '04', '06', '08', '10' Also any iterable of integers can be specified as first argument: @@ -80,16 +81,17 @@ class RangeSet(set): 1,3,6-8 >>> rset2 = RangeSet(rset) - Padding of ranges (eg. "003-009") can be managed through a public RangeSet - instance variable named padding. It may be changed at any time. Padding is - a simple display feature per RangeSet object, thus current padding value is - not taken into account when computing set operations. - RangeSet is itself an iterator over its items as integers (instead of - strings). However, this behavior is going to change in the next version - v1.9. For compatibility, please use the explicit method - :meth:`.RangeSet.intiter` to iterate over the set's indexes as integers. - To iterate over string items (with zero-padding if present), please use the - :meth:`RangeSet.striter` method. + Padding of ranges (eg. "003-009") is inferred from input arguments and + managed automatically. This is new in ClusterShell v1.9, where mixed lengths + zero padding is now supported within the same RangeSet. The instance + variable `padding` has become a property that can still be used to either + get the max padding length in the set, or force a fixed length zero-padding + on the set. + + RangeSet is itself a set and as such, provides an iterator over its items + as strings (strings are used since v1.9). It is recommended to use the + explicit iterators :meth:`RangeSet.intiter` and :meth:`RangeSet.striter` + when iterating over a RangeSet. RangeSet provides methods like :meth:`RangeSet.union`, :meth:`RangeSet.intersection`, :meth:`RangeSet.difference`, @@ -99,7 +101,7 @@ class RangeSet(set): :meth:`RangeSet.symmetric_difference_update` which conform to the Python Set API. """ - _VERSION = 3 # serial version number + _VERSION = 4 # serial version number def __init__(self, pattern=None, autostep=None): """Initialize RangeSet object. @@ -107,17 +109,15 @@ def __init__(self, pattern=None, autostep=None): :param pattern: optional string pattern :param autostep: optional autostep threshold """ - if pattern is None or isinstance(pattern, str): - set.__init__(self) - else: - set.__init__(self, pattern) + set.__init__(self) + + if pattern is not None and not isinstance(pattern, str): + pattern = ",".join("%s" % i for i in pattern) if isinstance(pattern, RangeSet): self._autostep = pattern._autostep - self.padding = pattern.padding else: self._autostep = None - self.padding = None self.autostep = autostep #: autostep threshold public instance attribute if isinstance(pattern, str): @@ -127,6 +127,7 @@ def _parse(self, pattern): """Parse string of comma-separated x-y/step -like ranges""" # Comma separated ranges for subrange in pattern.split(','): + subrange = subrange.strip() # ignore whitespaces if subrange.find('/') < 0: baserange, step = subrange, 1 else: @@ -143,11 +144,12 @@ def _parse(self, pattern): raise RangeSetParseError(subrange, "invalid step usage") begin = end = baserange else: - begin, end = baserange.split('-', 1) + # ignore whitespaces in a range + begin, end = (n.strip() for n in baserange.split('-', 1)) # compute padding and return node range info tuple try: - pad = 0 + pad = endpad = 0 if int(begin) != 0: begins = begin.lstrip("0") if len(begin) - len(begins) > 0: @@ -161,6 +163,12 @@ def _parse(self, pattern): ends = end.lstrip("0") else: ends = end + # explicit padding for begin and end must match + if len(end) - len(ends) > 0: + endpad = len(end) + if (pad > 0 or endpad > 0) and len(begin) != len(end): + raise RangeSetParseError(subrange, + "padding length mismatch") stop = int(ends) except ValueError: if len(subrange) == 0: @@ -185,8 +193,13 @@ def fromlist(cls, rnglist, autostep=None): @classmethod def fromone(cls, index, pad=0, autostep=None): - """Class method that returns a new RangeSet of one single item or - a single range (from integer or slice object).""" + """ + Class method that returns a new RangeSet of one single item or + a single range. Accepted input arguments can be: + - integer and padding length + - slice object and padding length + - string (1.9+) with padding automatically detected (pad is ignored) + """ inst = RangeSet(autostep=autostep) # support slice object with duck-typing try: @@ -197,6 +210,29 @@ def fromone(cls, index, pad=0, autostep=None): inst.add_range(index.start or 0, index.stop, index.step or 1, pad) return inst + @property + def padding(self): + """Get largest padding value of whole set""" + result = None + for si in self: + idx, digitlen = int(si), len(si) + # explicitly padded? + if digitlen > 1 and si[0] == '0': + # result always grows bigger as we iterate over a sorted set + # with largest padded values at the end + result = digitlen + return result + + @padding.setter + def padding(self, value): + """Force padding length on the whole set""" + if value is None: + value = 1 + cpyset = set(self) + self.clear() + for i in cpyset: + self.add(int(i), pad=value) + def get_autostep(self): """Get autostep value (property)""" if self._autostep >= AUTOSTEP_DISABLED: @@ -225,7 +261,8 @@ def dim(self): def _sorted(self): """Get sorted list from inner set.""" - return sorted(set.__iter__(self)) + # for mixed padding support, sort by both string length and index + return sorted(set.__iter__(self), key=lambda x: (len(x), x)) def __iter__(self): """Iterate over each element in RangeSet, currently as integers, with @@ -240,15 +277,19 @@ def intiter(self): return iter(self._sorted()) def striter(self): - """Iterate over each (optionally padded) string element in RangeSet.""" - pad = self.padding or 0 - for i in self._sorted(): - yield "%0*d" % (pad, i) + """Iterate over each element in RangeSet as strings with optional + zero-padding.""" + return iter(self._sorted()) + + def intiter(self): + """Iterate over each element in RangeSet as integer. + Zero padding info is ignored.""" + for e in self._sorted(): + yield int(e) def contiguous(self): """Object-based iterator over contiguous range sets.""" - pad = self.padding or 0 - for sli in self._contiguous_slices(): + for sli, pad in self._contiguous_slices(): yield RangeSet.fromone(slice(sli.start, sli.stop, sli.step), pad) def __reduce__(self): @@ -274,20 +315,25 @@ def __setstate__(self, dic): # workaround for object pickled from Python < 2.5 setattr(self, '_ranges', [(slice(start, stop, step), pad) \ for (start, stop, step), pad in self_ranges]) - # convert to v3 - for sli, pad in getattr(self, '_ranges'): - self.add_range(sli.start, sli.stop, sli.step, pad) - delattr(self, '_ranges') - delattr(self, '_length') - # add padding if unpickling old instances - if not hasattr(self, 'padding'): - setattr(self, 'padding', None) + if hasattr(self, '_ranges'): + # convert to v3 + for sli, pad in getattr(self, '_ranges'): + self.add_range(sli.start, sli.stop, sli.step, pad) + delattr(self, '_ranges') + delattr(self, '_length') + + if getattr(self, '_version', 0) == 3: # 1.6 - 1.8 + padding = getattr(self, 'padding', 0) + # convert integer set to string set + cpyset = set(self) + self.clear() + for i in cpyset: + self.add(i, pad=padding) # automatic conversion def _strslices(self): """Stringify slices list (x-y/step format)""" - pad = self.padding or 0 - for sli in self.slices(): + for sli, pad in self._folded_slices(): if sli.start + 1 == sli.stop: yield "%0*d" % (pad, sli.start) else: @@ -306,134 +352,128 @@ def __str__(self): # could be used to recreate a RangeSet with the same value __repr__ = __str__ - def _contiguous_slices(self): - """Internal iterator over contiguous slices in RangeSet.""" - k = j = None - for i in self._sorted(): - if k is None: - k = j = i - if i - j > 1: - yield slice(k, j + 1, 1) - k = i - j = i - if k is not None: - yield slice(k, j + 1, 1) + def _slices_padding(self, autostep=AUTOSTEP_DISABLED): + """Iterator over (slices, padding). - def _folded_slices(self): - """Internal generator that is able to retrieve ranges organized by - step.""" - if len(self) == 0: - return - - prng = None # pending range - istart = None # processing starting indices - step = 0 # processing step - for sli in self._contiguous_slices(): - start = sli.start - stop = sli.stop - unitary = (start + 1 == stop) # one indices? - if istart is None: # first loop - if unitary: - istart = start + Iterator over RangeSet slices, either a:b:1 slices if autostep + is disabled (default), or a:b:step slices if autostep is specified. + """ + # + # Now support mixed lengths zero-padding (v1.9) + cur_pad = 0 + cur_padded = False + cur_start = None + cur_step = None + last_idx = None + + for si in self._sorted(): + + # numerical index and length of digits + idx, digitlen = int(si), len(si) + + # is current digit zero-padded? + padded = (digitlen > 1 and si[0] == '0') + + if cur_start is not None: + padding_mismatch = False + step_mismatch = False + + # check conditions to yield + # - padding mismatch + # - step check (step=1 is just a special case if contiguous) + + if cur_padded: + # currently strictly padded, our next item could be + # unpadded but with the same length + if digitlen != cur_pad: + padding_mismatch = True else: - prng = [start, stop, 1] - istart = stop - 1 - i = k = istart - elif step == 0: # istart is set but step is unknown - if not unitary: - if prng is not None: - # yield and replace pending range - yield slice(*prng) + # current not padded, and because the set is sorted, + # it should stay that way + if padded: + padding_mismatch = True + + if not padding_mismatch: + # does current range lead to broken step? + if cur_step is not None: + # only consider it if step is defined + if cur_step != idx - last_idx: + step_mismatch = True + + if padding_mismatch or step_mismatch: + if cur_step is not None: + # stepped is True when autostep setting does apply + stepped = (cur_step == 1) or (last_idx - cur_start >= autostep * cur_step) + step = cur_step else: - yield slice(istart, istart + 1, 1) - prng = [start, stop, 1] - istart = k = stop - 1 - continue - i = start - else: # step > 0 - assert step > 0 - i = start - # does current range lead to broken step? - if step != i - k or not unitary: - #Python2.6+: j = i if step == i - k else k - if step == i - k: - j = i - else: - j = k - # stepped is True when autostep setting does apply - stepped = (j - istart >= self._autostep * step) - if prng: # yield pending range? - if stepped: - prng[1] -= 1 - else: - istart += step - yield slice(*prng) - prng = None - if step != i - k: - # case: step value has changed + stepped = True + step = 1 + if stepped: - yield slice(istart, k + 1, step) + yield slice(cur_start, last_idx + 1, step), cur_pad if cur_padded else 0 + cur_start = idx + cur_padded = padded + cur_pad = digitlen else: - for j in range(istart, k - step + 1, step): - yield slice(j, j + 1, 1) - if not unitary: - yield slice(k, k + 1, 1) - if unitary: - if stepped: - istart = i = k = start + if padding_mismatch: + stop = last_idx + 1 else: - istart = k - else: - prng = [start, stop, 1] - istart = i = k = stop - 1 - elif not unitary: - # case: broken step by contiguous range - if stepped: - # yield 'range/step' by taking first indices of new range - yield slice(istart, i + 1, step) - i += 1 - else: - # autostep setting does not apply in that case - for j in range(istart, i - step + 1, step): - yield slice(j, j + 1, 1) - if stop > i + 1: # current->pending only if not unitary - prng = [i, stop, 1] - istart = i = k = stop - 1 - step = i - k - k = i - # exited loop, process pending range or indices... - if step == 0: - if prng: - yield slice(*prng) + stop = last_idx - step + 1 + + for j in range(cur_start, stop, step): + yield slice(j, j + 1, 1), cur_pad if cur_padded else 0 + + if padding_mismatch: + cur_start = idx + cur_padded = padded + cur_pad = digitlen + else: + cur_start = last_idx + + cur_step = idx - last_idx if step_mismatch else None + last_idx = idx + continue + else: - yield slice(istart, istart + 1, 1) - else: - assert step > 0 - stepped = (k - istart >= self._autostep * step) - if prng: - if stepped: - prng[1] -= 1 - else: - istart += step - yield slice(*prng) - prng = None - if stepped: - yield slice(istart, i + 1, step) + # first index + cur_padded = padded + cur_pad = digitlen + cur_start = idx + cur_step = None + last_idx = idx + continue + + cur_step = idx - last_idx + last_idx = idx + + if cur_start is not None: + if cur_step is not None: + # stepped is True when autostep setting does apply + stepped = (last_idx - cur_start >= self._autostep * cur_step) + else: + stepped = True + + if stepped or cur_step == 1: + yield slice(cur_start, last_idx + 1, cur_step), cur_pad if cur_padded else 0 else: - for j in range(istart, i + 1, step): - yield slice(j, j + 1, 1) + for j in range(cur_start, idx + 1, cur_step): + yield slice(j, j + 1, 1), cur_pad if cur_padded else 0 + + def _contiguous_slices(self): + """Internal iterator over contiguous slices in RangeSet.""" + return self._slices_padding() + + def _folded_slices(self): + """Internal generator over ranges organized by step.""" + return self._slices_padding(self._autostep) def slices(self): """ - Iterate over RangeSet ranges as Python slice objects. + Iterate over RangeSet ranges as Python slide objects. + NOTE: zero-padding info is not provided """ - # return an iterator - if self._autostep >= AUTOSTEP_DISABLED: - # autostep disabled: call simpler method to return only a:b slices - return self._contiguous_slices() - else: - # autostep enabled: call generic method to return a:b:step slices - return self._folded_slices() + for sli, pad in self._folded_slices(): + yield sli def __getitem__(self, index): """ @@ -442,7 +482,6 @@ def __getitem__(self, index): if isinstance(index, slice): inst = RangeSet() inst._autostep = self._autostep - inst.padding = self.padding inst.update(self._sorted()[index]) return inst elif isinstance(index, int): @@ -486,17 +525,15 @@ def add_range(self, start, stop, step=1, pad=0): assert pad >= 0 assert stop - start < 1e9, "range too large" - # inherit padding info only if currently not defined - if pad is not None and pad > 0 and self.padding is None: - self.padding = pad - - set.update(self, range(start, stop, step)) + if pad == 0: + set.update(self, ("%d" % i for i in range(start, stop, step))) + else: + set.update(self, ("%0*d" % (pad, i) for i in range(start, stop, step))) def copy(self): """Return a shallow copy of a RangeSet.""" cpy = self.__class__() cpy._autostep = self._autostep - cpy.padding = self.padding cpy.update(self) return cpy @@ -608,7 +645,7 @@ def __contains__(self, element): if isinstance(element, set): return element.issubset(self) - return set.__contains__(self, int(element)) + return set.__contains__(self, str(element)) # Subset and superset test @@ -697,11 +734,7 @@ def difference_update(self, other, strict=False): # Python dict-like mass mutations: update, clear def update(self, iterable): - """Add all integers from an iterable (such as a list).""" - if isinstance(iterable, RangeSet): - # keep padding unless it has not been defined yet - if self.padding is None and iterable.padding is not None: - self.padding = iterable.padding + """Add all indexes (as strings) from an iterable (such as a list).""" assert not isinstance(iterable, str) set.update(self, iterable) @@ -711,46 +744,66 @@ def updaten(self, rangesets): """ for rng in rangesets: if isinstance(rng, set): - self.update(rng) + self.update(str(i) for i in rng) # 1.9+: force cast to str else: self.update(RangeSet(rng)) - # py2.5+ - #self.update(rng if isinstance(rng, set) else RangeSet(rng)) def clear(self): """Remove all elements from this RangeSet.""" set.clear(self) - self.padding = None # Single-element mutations: add, remove, discard def add(self, element, pad=0): """Add an element to a RangeSet. This has no effect if the element is already present. + + ClusterShell 1.9+ uses strings instead of integers to better manage + zero-padded ranges with mixed lengths. This method supports either a + string or an integer with padding info. + + :param element: the element to add (integer or string) + :param pad: zero padding length (integer); ignored if element is string """ - # inherit padding info only if currently not defined - if pad is not None and pad > 0 and self.padding is None: - self.padding = pad + if isinstance(element, str): + set.add(self, element) + else: + set.add(self, "%0*d" % (pad, int(element))) - set.add(self, int(element)) + def remove(self, element, pad=0): + """Remove an element from a RangeSet. - def remove(self, element): - """Remove an element from a RangeSet; it must be a member. + ClusterShell 1.9+ uses strings instead of integers to better manage + zero-padded ranges with mixed lengths. This method supports either a + string or an integer with padding info. - :param element: the element to remove + :param element: the element to remove (integer or string) + :param pad: zero padding length (integer); ignored if element is string :raises KeyError: element is not contained in RangeSet :raises ValueError: element is not castable to integer """ - set.remove(self, int(element)) + if isinstance(element, str): + set.remove(self, element) + else: + set.remove(self, "%0*d" % (pad, int(element))) - def discard(self, element): - """Remove element from the RangeSet if it is a member. + def discard(self, element, pad=0): + """Discard an element from a RangeSet if it is a member. If the element is not a member, do nothing. + + ClusterShell 1.9+ uses strings instead of integers to better manage + zero-padded ranges with mixed lengths. This method supports either a + string or an integer with padding info. + + :param element: the element to remove (integer or string) + :param pad: zero padding length (integer); ignored if element is string """ try: - i = int(element) - set.discard(self, i) + if isinstance(element, str): + set.discard(self, element) + else: + set.discard(self, "%0*d" % (pad, int(element))) except ValueError: pass # ignore other object types @@ -896,7 +949,9 @@ def _iter(self): @precond_fold() def iter_padding(self): - """Iterate through individual items as tuples with padding info.""" + """Iterate through individual items as tuples with padding info. + As of v1.9, this method returns the largest padding value of each + items, as mixed length padding is allowed.""" for vec in self._veclist: for ivec in product(*vec): yield ivec, [rg.padding for rg in vec] @@ -967,8 +1022,7 @@ def __getitem__(self, index): for rgvec in self._veclist: iveclist += product(*rgvec) assert(len(iveclist) == len(self)) - rnd = RangeSetND(iveclist[index], pads=self.pads(), - autostep=self.autostep) + rnd = RangeSetND(iveclist[index], autostep=self.autostep) return rnd elif isinstance(index, int): @@ -1079,7 +1133,7 @@ def rgveckeyfunc(rgvec): # (3) lower first index first # (4) lower last index first return (-reduce(mul, [len(rg) for rg in rgvec]), \ - tuple((-len(rg), rg[0], rg[-1]) for rg in rgvec)) + tuple((-len(rg), int(rg[0]), int(rg[-1])) for rg in rgvec)) self._veclist.sort(key=rgveckeyfunc) @precond_fold() @@ -1137,13 +1191,11 @@ def _fold_multivariate_expand(self): """Multivariate nD folding: expand [phase 1]""" max_length = sum([reduce(mul, [len(rg) for rg in rgvec]) \ for rgvec in self._veclist]) - # Simple heuristic that makes us faster + # Simple heuristic to make us faster if len(self._veclist) * (len(self._veclist) - 1) / 2 > max_length * 10: # *** nD full expand is preferred *** - pads = self.pads() - self._veclist = [[RangeSet.fromone(i, pad=pads[axis], - autostep=self.autostep) - for axis, i in enumerate(tvec)] + self._veclist = [[RangeSet.fromone(i, autostep=self.autostep) + for i in tvec] for tvec in set(self._iter())] return diff --git a/tests/CLINodesetTest.py b/tests/CLINodesetTest.py index 32c4ae61..a9ce38b7 100644 --- a/tests/CLINodesetTest.py +++ b/tests/CLINodesetTest.py @@ -739,7 +739,7 @@ def test_040_wildcards(self): self._nodeset_t(["-s", "other", "-f", "*!*[033]"], None, b"nova[030-032,034-489]\n") self._nodeset_t(["-s", "other", "--autostep=3", "-f", "*!*[033-099/2]"], - None, b"nova[030-031,032-100/2,101-489]\n") + None, b"nova[030-032,034-100/2,101-489]\n") class CLINodesetGroupResolverTest3(CLINodesetTestBase): diff --git a/tests/NodeSetTest.py b/tests/NodeSetTest.py index 137cd3df..85651a1c 100644 --- a/tests/NodeSetTest.py +++ b/tests/NodeSetTest.py @@ -18,6 +18,12 @@ class NodeSetTest(unittest.TestCase): + def _assertEqual(self, pattern, result=None): + ns = NodeSet(pattern) + if result is None: + result = pattern + self.assertEqual(str(ns), result) + def _assertNode(self, nodeset, nodename): """helper to assert single node presence""" self.assertEqual(str(nodeset), nodename) @@ -425,15 +431,13 @@ def test_numerical_bracket_folding(self): # see also NodeSetErrorTest.py for unsupported trailing digits w/ steps - # /!\ padding mismatch cases: current behavior - nodeset = NodeSet("prod-0[10-345]") # padding mismatch - self.assertEqual(str(nodeset), "prod-[010-345]") - nodeset = NodeSet("prod-1[10-345]") # no mismatch there + # /!\ padding mismatch cases: mixed padding allowed since 1.9 + nodeset = NodeSet("prod-1[10-345]") # no padding so no mismatch there: OK self.assertEqual(str(nodeset), "prod-[110-1345]") - nodeset = NodeSet("prod-02[10-345]") # padding mismatch - self.assertEqual(str(nodeset), "prod-[0210-2345]") - nodeset = NodeSet("prod-02[10-34,069-099]") # padding mismatch - self.assertEqual(str(nodeset), "prod-[02010-02034,02069-02099]") + nodeset = NodeSet("prod-02[10-34,069-099]") # no padding mismatch within a range: OK + self.assertEqual(str(nodeset), "prod-[0210-0234,02069-02099]") + self._assertNS("prod-0[10-345]", NodeSetParseRangeError) # padding length mismatch in a range + self._assertNS("prod-02[10-345]", NodeSetParseRangeError) # padding length mismatch in a range # numerical folding with nD nodesets nodeset = NodeSet("x01[0-1]y01[0-1]z01[0-1]") @@ -516,8 +520,12 @@ def test_numerical_bracket_folding(self): self.assertEqual(str(nodeset), "3abc[16156,16256,16356,16456]d") nodeset = NodeSet("0[3,6,9]1abc16[1-4]56d") self.assertEqual(str(nodeset), "[031,061,091]abc[16156,16256,16356,16456]d") - nodeset = NodeSet("0123[0-100]L6") - self.assertEqual(str(nodeset), "[01230-123100]L6") + + # bogus range with padding, we are stricter in v1.9+ + self._assertNS("0123[0-100]L6", NodeSetParseRangeError) + # ok when no mismatch within a given range + nodeset = NodeSet("[01230-99999,100000-123100]L6") + self.assertEqual(str(nodeset), "[01230-99999,100000-123100]L6") nodeset = NodeSet("0123[000-100]L6") self.assertEqual(str(nodeset), "[0123000-0123100]L6") @@ -1227,7 +1235,8 @@ def testAddAdjust(self): self.assertEqual(str(nodeset), "green[1-7/2]") self.assertEqual(len(nodeset), 4) nodeset.add("green[6-17/2]") - self.assertEqual(str(nodeset), "green[1-5/2,6-7,8-16/2]") + #self.assertEqual(str(nodeset), "green[1-5/2,6-7,8-16/2]") # <1.9 + self.assertEqual(str(nodeset), "green[1-5/2,6-8,10-16/2]") # 1.9+ self.assertEqual(len(nodeset), 10) def testRemove(self): @@ -1330,20 +1339,24 @@ def testContainsUsingPadding(self): """test NodeSet contains() when using padding""" nodeset = NodeSet("white[001,030]") nodeset.add("white113") - self.assertTrue(NodeSet("white30") in nodeset) + self.assertFalse(NodeSet("white30") in nodeset) self.assertTrue(NodeSet("white030") in nodeset) # case: nodeset without padding info is compared to a # padding-initialized range self.assertTrue(NodeSet("white113") in nodeset) self.assertTrue(NodeSet("white[001,113]") in nodeset) - self.assertTrue(NodeSet("gene0113") in NodeSet("gene[001,030,113]")) + self.assertTrue(NodeSet("gene113") in NodeSet("gene[001,030,113]")) + self.assertFalse(NodeSet("gene0113") in NodeSet("gene[001,030,113]")) self.assertTrue(NodeSet("gene0113") in NodeSet("gene[0001,0030,0113]")) - self.assertTrue(NodeSet("gene0113") in NodeSet("gene[098-113]")) + self.assertTrue(NodeSet("gene113") in NodeSet("gene[098-113]")) + self.assertFalse(NodeSet("gene0113") in NodeSet("gene[098-113]")) self.assertTrue(NodeSet("gene0113") in NodeSet("gene[0098-0113]")) # case: len(str(ielem)) >= rgpad nodeset = NodeSet("white[001,099]") nodeset.add("white100") nodeset.add("white1000") + self.assertTrue(NodeSet("white100") in nodeset) + self.assertFalse(NodeSet("white0100") in nodeset) self.assertTrue(NodeSet("white1000") in nodeset) def test_issuperset(self): @@ -1355,10 +1368,11 @@ def test_issuperset(self): self.assertTrue(nodeset.issuperset(NodeSet("tronic[0140-0200]"))) self.assertTrue(nodeset.issuperset("tronic0070")) self.assertFalse(nodeset.issuperset("tronic0034")) - # check padding issue - since 1.6 padding is ignored in this case - self.assertTrue(nodeset.issuperset("tronic36")) - self.assertTrue(nodeset.issuperset("tronic[36-40]")) - self.assertTrue(nodeset.issuperset(NodeSet("tronic[36-40]"))) + # check padding issue - fixed since 1.9 + self.assertFalse(nodeset.issuperset("tronic36")) # used to be true < 1.9 + self.assertFalse(nodeset.issuperset("tronic[36-40]")) # same + self.assertFalse(nodeset.issuperset(NodeSet("tronic[36-40]"))) # same + self.assertTrue(nodeset.issuperset(NodeSet("tronic[0036-0040]"))) # check gt self.assertTrue(nodeset > NodeSet("tronic[0100-0200]")) self.assertFalse(nodeset > NodeSet("tronic[0036-1630]")) @@ -1371,8 +1385,9 @@ def test_issuperset(self): self.assertTrue(nodeset > NodeSet("tronic[0100-0200]")) self.assertTrue(nodeset > NodeSet("lounge[36-400/2]")) self.assertTrue(nodeset.issuperset(NodeSet("lounge[36-400/2]," - "tronic[0100-660]"))) - self.assertTrue(nodeset > NodeSet("lounge[36-400/2],tronic[0100-660]")) + "tronic[0100-0660]"))) + self.assertTrue(nodeset > NodeSet("lounge[36-400/2],tronic[0100-0660]")) + self._assertNS("lounge[36-400/2],tronic[0100-660]", NodeSetParseRangeError) def test_issubset(self): """test NodeSet issubset()""" @@ -1393,9 +1408,11 @@ def test_issubset(self): self.assertFalse(nodeset <= NodeSet("artcore[3-980]")) self.assertFalse(nodeset <= NodeSet("artcore[2-998]")) self.assertEqual(len(nodeset), 997) - # check padding issue - since 1.6 padding is ignored in this case - self.assertTrue(nodeset.issubset("artcore[0001-1000]")) + # check padding issues - fixed since 1.9 + self.assertFalse(nodeset.issubset("artcore[0001-1000]")) # was true < 1.9 + self.assertFalse(nodeset.issubset("artcore30")) self.assertFalse(nodeset.issubset("artcore030")) + self.assertFalse(nodeset.issubset("artcore0030")) # multiple patterns case nodeset = NodeSet("tronic[0036-1630],lounge[20-660/2]") self.assertTrue(nodeset @@ -1716,6 +1733,10 @@ def testCopy(self): nodeset2 = nodeset.copy() self.assertEqual(nodeset, nodeset2) # content equality + # unpickling tests; generate data with: + # ns = NodeSet("bar[050-150,502-599],foo[1,4-50,80-100]") + # print(binascii.b2a_base64(pickle.dumps(ns))) + def test_unpickle_v1_3_py24(self): """test NodeSet unpickling (against v1.3/py24)""" nodeset = pickle.loads(binascii.a2b_base64("gAJjQ2x1c3RlclNoZWxsLk5vZGVTZXQKTm9kZVNldApxACmBcQF9cQIoVQdfbGVuZ3RocQNLAFUJX3BhdHRlcm5zcQR9cQUoVQh5ZWxsb3clc3EGKGNDbHVzdGVyU2hlbGwuTm9kZVNldApSYW5nZVNldApxB29xCH1xCShoA0sBVQlfYXV0b3N0ZXBxCkdUskmtJZTDfVUHX3Jhbmdlc3ELXXEMKEsESwRLAUsAdHENYXViVQZibHVlJXNxDihoB29xD31xEChoA0sIaApHVLJJrSWUw31oC11xESgoSwZLCksBSwB0cRIoSw1LDUsBSwB0cRMoSw9LD0sBSwB0cRQoSxFLEUsBSwB0cRVldWJVB2dyZWVuJXNxFihoB29xF31xGChoA0tlaApHVLJJrSWUw31oC11xGShLAEtkSwFLAHRxGmF1YlUDcmVkcRtOdWgKTnViLg==")) @@ -1800,6 +1821,16 @@ def test_unpickle_v1_7_3_py27(self): self.assertEqual(nodeset[1], "bar051") self.assertEqual(nodeset[-1], "foo100") + def test_unpickle_v1_8_4_py27(self): + """test NodeSet unpickling (against v1.8.4/py27)""" + nodeset = pickle.loads(binascii.a2b_base64("Y2NvcHlfcmVnCl9yZWNvbnN0cnVjdG9yCnAwCihjQ2x1c3RlclNoZWxsLk5vZGVTZXQKTm9kZVNldApwMQpjX19idWlsdGluX18Kb2JqZWN0CnAyCk50cDMKUnA0CihkcDUKUydmb2xkX2F4aXMnCnA2Ck5zUydfbGVuZ3RoJwpwNwpJMApzUydfcGF0dGVybnMnCnA4CihkcDkKUydmb28lcycKcDEwCmNDbHVzdGVyU2hlbGwuUmFuZ2VTZXQKUmFuZ2VTZXQKcDExCihTJzEsNC01MCw4MC0xMDAnCnAxMgp0cDEzClJwMTQKKGRwMTUKUydwYWRkaW5nJwpwMTYKTnNTJ19hdXRvc3RlcCcKcDE3CkYxZSsxMDAKc1MnX3ZlcnNpb24nCnAxOApJMwpzYnNTJ2JhciVzJwpwMTkKZzExCihTJzA1MC0xNTAsNTAyLTU5OScKcDIwCnRwMjEKUnAyMgooZHAyMwpnMTYKSTMKc2cxNwpGMWUrMTAwCnNnMTgKSTMKc2Jzc2cxNwpOc2cxOApJMgpzYi4=")) + self.assertEqual(nodeset, NodeSet("foo[1,4-50,80-100],bar[050-150,502-599]")) + self.assertEqual(str(nodeset), "bar[050-150,502-599],foo[1,4-50,80-100]") + self.assertEqual(len(nodeset), 268) + self.assertEqual(nodeset[0], "bar050") + self.assertEqual(nodeset[1], "bar051") + self.assertEqual(nodeset[-1], "foo100") + def test_pickle_current(self): """test NodeSet pickling (current version)""" dump = pickle.dumps(NodeSet("foo[1-100]")) @@ -2126,8 +2157,8 @@ def test_nd_issubset(self): self.assertFalse(nodeset <= NodeSet("artcore[3-980]-ib0")) self.assertFalse(nodeset <= NodeSet("artcore[2-998]-ib0")) self.assertEqual(len(nodeset), 997) - # check padding issue - since 1.6 padding is ignored in this case - self.assertTrue(nodeset.issubset("artcore[0001-1000]-ib0")) + # check padding issue - fixed in 1.9 + self.assertFalse(nodeset.issubset("artcore[0001-1000]-ib0")) # used to be true < 1.9 self.assertFalse(nodeset.issubset("artcore030-ib0")) # multiple patterns case nodeset = NodeSet("tronic[0036-1630],lounge[20-660/2]") @@ -2464,6 +2495,46 @@ def test_autostep_property(self): n1.autostep = 3 self.assertEqual(n1.copy().autostep, 3) + n1 = NodeSet("n09,n11") + self.assertEqual(str(n1), "n[09,11]") + self.assertEqual(len(n1), 2) + self.assertEqual(n1.autostep, None) + n1.autostep = 2 + self.assertEqual(str(n1), "n[09-11/2]") + n1.autostep = 3 + self.assertEqual(str(n1), "n[09,11]") + + n1 = NodeSet("n1,n3,n5,p03,p06,p09,p012,p015") + self.assertEqual(str(n1), "n[1,3,5],p[03,06,09,012,015]") + self.assertEqual(len(n1), 8) + self.assertEqual(n1.autostep, None) + n1.autostep = 2 + self.assertEqual(str(n1), "n[1-5/2],p[03-09/3,012-015/3]") + n1.autostep = 3 + self.assertEqual(str(n1), "n[1-5/2],p[03-09/3,012,015]") + + n1 = NodeSet("n1,n3,n5,p03,p06,p09") + self.assertEqual(str(n1), "n[1,3,5],p[03,06,09]") + self.assertEqual(len(n1), 6) + self.assertEqual(n1.autostep, None) + n1.autostep = 2 + self.assertEqual(str(n1), "n[1-5/2],p[03-09/3]") + self.assertEqual(n1.autostep, 2) + + n1 = NodeSet("n1,n03,n05") + self.assertEqual(str(n1), "n[1,03,05]") + self.assertEqual(len(n1), 3) + self.assertEqual(n1.autostep, None) + n1.autostep = 3 + self.assertEqual(str(n1), "n[1,03,05]") + + n1 = NodeSet("n1,n03,n05,n07") + self.assertEqual(str(n1), "n[1,03,05,07]") + self.assertEqual(len(n1), 4) + self.assertEqual(n1.autostep, None) + n1.autostep = 2 + self.assertEqual(str(n1), "n[1,03-07/2]") + def test_nd_autostep(self): """test NodeSet autostep (nD)""" n1 = NodeSet("p2n1,p2n3,p2n5") @@ -2737,9 +2808,6 @@ def test_fully_numeric(self): n1 = NodeSet("3,5,[7-10,40]") self.assertEqual(str(n1), "[3,5,7-10,40]") self.assertEqual(len(n1), 7) - n1 = NodeSet("0[7-9,10]") - self.assertEqual(str(n1), "[07-10]") - self.assertEqual(len(n1), 4) n1 = NodeSet("nova3,nova4,5,nova6") self.assertEqual(str(n1), "5,nova[3-4,6]") self.assertEqual(len(n1), 4) @@ -2749,13 +2817,15 @@ def test_fully_numeric(self): n1 = NodeSet("[0-10]") self.assertEqual(str(n1), "[0-10]") self.assertEqual(len(n1), 11) - n1 = NodeSet("0[0-10]") - self.assertEqual(str(n1), "[00-10]") - self.assertEqual(len(n1), 11) - n1 = NodeSet("[0-10]0") - self.assertEqual(str(n1), "[00,10,20,30,40,50,60,70,80,90,100]") - self.assertEqual(len(n1), 11) - n1 = NodeSet("0[0-10]0") - self.assertEqual(str(n1), - "[000,010,020,030,040,050,060,070,080,090,100]") self.assertEqual(len(n1), 11) + # leading 0 along with mixed lengths padding + self._assertEqual("[07-09,010]") + self._assertNS("0[7-10]", NodeSetParseRangeError) + self._assertNS("0[7-9,10]", NodeSetParseRangeError) # expanded to 7-10 first + self._assertEqual("0[7-9,010]", "[07-09,0010]") + self._assertEqual("0[07-09,10]", "[007-010]") + self._assertEqual("0[07-09,010]", "[007-009,0010]") + self._assertNS("0[0-10]", NodeSetParseRangeError) + # trailing 0 along with mixed lengths padding + self._assertNS("[0-10]0", NodeSetParseRangeError) + self._assertNS("0[0-10]0", NodeSetParseRangeError) diff --git a/tests/RangeSetNDTest.py b/tests/RangeSetNDTest.py index 7ff8f916..66c1cfa1 100644 --- a/tests/RangeSetNDTest.py +++ b/tests/RangeSetNDTest.py @@ -5,12 +5,16 @@ import sys import unittest +import warnings from ClusterShell.RangeSet import RangeSet, RangeSetND class RangeSetNDTest(unittest.TestCase): + def setUp(self): + warnings.simplefilter("always") + def _testRS(self, test, res, length): r1 = RangeSetND(test, autostep=3) self.assertEqual(str(r1), res) @@ -392,26 +396,26 @@ def test_xor(self): def test_getitem(self): rn1 = RangeSetND([["10", "10-13"], ["0-3", "1-2"]]) self.assertEqual(len(rn1), 12) - self.assertEqual(rn1[0], (0, 1)) - self.assertEqual(rn1[1], (0, 2)) - self.assertEqual(rn1[2], (1, 1)) - self.assertEqual(rn1[3], (1, 2)) - self.assertEqual(rn1[4], (2, 1)) - self.assertEqual(rn1[5], (2, 2)) - self.assertEqual(rn1[6], (3, 1)) - self.assertEqual(rn1[7], (3, 2)) - self.assertEqual(rn1[8], (10, 10)) - self.assertEqual(rn1[9], (10, 11)) - self.assertEqual(rn1[10], (10, 12)) - self.assertEqual(rn1[11], (10, 13)) + self.assertEqual(rn1[0], ('0', '1')) + self.assertEqual(rn1[1], ('0', '2')) + self.assertEqual(rn1[2], ('1', '1')) + self.assertEqual(rn1[3], ('1', '2')) + self.assertEqual(rn1[4], ('2', '1')) + self.assertEqual(rn1[5], ('2', '2')) + self.assertEqual(rn1[6], ('3', '1')) + self.assertEqual(rn1[7], ('3', '2')) + self.assertEqual(rn1[8], ('10', '10')) + self.assertEqual(rn1[9], ('10', '11')) + self.assertEqual(rn1[10], ('10', '12')) + self.assertEqual(rn1[11], ('10', '13')) self.assertRaises(IndexError, rn1.__getitem__, 12) # negative indices - self.assertEqual(rn1[-1], (10, 13)) - self.assertEqual(rn1[-2], (10, 12)) - self.assertEqual(rn1[-3], (10, 11)) - self.assertEqual(rn1[-4], (10, 10)) - self.assertEqual(rn1[-5], (3, 2)) - self.assertEqual(rn1[-12], (0, 1)) + self.assertEqual(rn1[-1], ('10', '13')) + self.assertEqual(rn1[-2], ('10', '12')) + self.assertEqual(rn1[-3], ('10', '11')) + self.assertEqual(rn1[-4], ('10', '10')) + self.assertEqual(rn1[-5], ('3', '2')) + self.assertEqual(rn1[-12], ('0', '1')) self.assertRaises(IndexError, rn1.__getitem__, -13) self.assertRaises(TypeError, rn1.__getitem__, "foo") @@ -447,11 +451,11 @@ def test_contiguous(self): def test_iter(self): rn0 = RangeSetND([['1-2', '3'], ['1-2', '4'], ['2-6', '6-9,11']]) self.assertEqual(len([r for r in rn0]), len(rn0)) - self.assertEqual([(2, 6), (2, 7), (2, 8), (2, 9), (2, 11), (3, 6), - (3, 7), (3, 8), (3, 9), (3, 11), (4, 6), (4, 7), - (4, 8), (4, 9), (4, 11), (5, 6), (5, 7), (5, 8), - (5, 9), (5, 11), (6, 6), (6, 7), (6, 8), (6, 9), - (6, 11), (1, 3), (1, 4), (2, 3), (2, 4)], + self.assertEqual([('2', '6'), ('2', '7'), ('2', '8'), ('2', '9'), ('2', '11'), ('3', '6'), + ('3', '7'), ('3', '8'), ('3', '9'), ('3', '11'), ('4', '6'), ('4', '7'), + ('4', '8'), ('4', '9'), ('4', '11'), ('5', '6'), ('5', '7'), ('5', '8'), + ('5', '9'), ('5', '11'), ('6', '6'), ('6', '7'), ('6', '8'), ('6', '9'), + ('6', '11'), ('1', '3'), ('1', '4'), ('2', '3'), ('2', '4')], [r for r in rn0]) def test_pads(self): @@ -463,16 +467,15 @@ def test_pads(self): self.assertEqual(str(rn1), "02-06; 006-009,411\n01-02; 003-004\n") self.assertEqual(len(rn1), 29) self.assertEqual(rn1.pads(), (2, 3)) - # Note: padding mismatch is NOT supported by ClusterShell - # We just track any regressions here (MAY CHANGE!) + # Note: mixed lenghts zero-padding supported in ClusterShell v1.9 rn1 = RangeSetND([['01-02', '003'], ['01-02', '0101'], ['02-06', '006-009,411']]) - # here 0101 padding is changed to 101 - self.assertEqual(str(rn1), '02-06; 006-009,411\n01-02; 003,101\n') + # before v1.9: 0101 padding was changed to 101 + self.assertEqual(str(rn1), '02-06; 006-009,411\n01-02; 003,0101\n') self.assertEqual(len(rn1), 29) - self.assertEqual(rn1.pads(), (2, 3)) + self.assertEqual(rn1.pads(), (2, 4)) rn1 = RangeSetND([['01-02', '0003'], ['01-02', '004'], ['02-06', '006-009,411']]) - # here 004 padding is changed to 0004 - self.assertEqual(str(rn1), '02-06; 006-009,411\n01-02; 0003-0004\n') + # before v1.9: 004 padding was wrongly changed to 0004 + self.assertEqual(str(rn1), '02-06; 006-009,411\n01-02; 004,0003\n') self.assertEqual(len(rn1), 29) self.assertEqual(rn1.pads(), (2, 4)) # pads() returns max padding length by axis diff --git a/tests/RangeSetTest.py b/tests/RangeSetTest.py index 6ce5647a..65ea0824 100644 --- a/tests/RangeSetTest.py +++ b/tests/RangeSetTest.py @@ -6,11 +6,15 @@ import binascii import pickle import unittest +import warnings from ClusterShell.RangeSet import RangeSet, RangeSetParseError class RangeSetTest(unittest.TestCase): + def setUp(self): + warnings.simplefilter("always") + def _testRS(self, test, res, length): r1 = RangeSet(test, autostep=3) self.assertEqual(str(r1), res) @@ -34,7 +38,7 @@ def testStepSimple(self): def testStepAdvanced(self): """test RangeSet advanced step usages""" - self._testRS("1-4/4,2-6/2", "1,2-6/2", 4) # 1.6 small behavior change + self._testRS("1-4/4,2-6/2", "1-2,4,6", 4) # 1.9 small behavior change self._testRS("6-24/6,9-21/6", "6-24/3", 7) self._testRS("0-24/2,9-21/2", "0-8/2,9-22,24", 20) self._testRS("0-24/2,9-21/2,100", "0-8/2,9-22,24,100", 21) @@ -49,13 +53,18 @@ def testStepAdvanced(self): self._testRS("1-16/3,1-16/6", "1-16/3", 6) self._testRS("1-16/6,1-16/3", "1-16/3", 6) self._testRS("1-16/3,3-19/6", "1,3-4,7,9-10,13,15-16", 9) - #self._testRS("1-16/3,3-19/4", "1,3-4,7,10-11,13,15-16,19", 10) # < 1.6 - self._testRS("1-16/3,3-19/4", "1,3,4-10/3,11-15/2,16,19", 10) # >= 1.6 + #self._testRS("1-16/3,3-19/4", "1,3-4,7,10-11,13,15-16,19", 10) # <1.6 + #self._testRS("1-16/3,3-19/4", "1,3,4-10/3,11-15/2,16,19", 10) # 1.6+ + self._testRS("1-16/3,3-19/4", "1,3-4,7,10-11,13,15-16,19", 10) # 1.9+ self._testRS("1-17/2,2-18/2", "1-18", 18) self._testRS("1-17/2,33-41/2,2-18/2", "1-18,33-41/2", 23) self._testRS("1-17/2,33-41/2,2-20/2", "1-18,20,33-41/2", 24) self._testRS("1-17/2,33-41/2,2-19/2", "1-18,33-41/2", 23) self._testRS("1968-1970,1972,1975,1978-1981,1984-1989", "1968-1970,1972-1978/3,1979-1981,1984-1989", 15) + # use of 0-padding in the step number is ignored + self._testRS("1-17/01", "1-17", 17) + self._testRS("1-17/02", "1-17/2", 9) + self._testRS("1-17/03", "1-16/3", 6) def test_bad_syntax(self): """test parse errors""" @@ -300,7 +309,7 @@ def testSubStep(self): r1 = RangeSet("1-100,102,105-242,800", autostep=3) r2 = RangeSet("1-1000/3", autostep=3) r1.difference_update(r2) - self.assertEqual(str(r1), "2-3,5-6,8-9,11-12,14-15,17-18,20-21,23-24,26-27,29-30,32-33,35-36,38-39,41-42,44-45,47-48,50-51,53-54,56-57,59-60,62-63,65-66,68-69,71-72,74-75,77-78,80-81,83-84,86-87,89-90,92-93,95-96,98,99-105/3,107-108,110-111,113-114,116-117,119-120,122-123,125-126,128-129,131-132,134-135,137-138,140-141,143-144,146-147,149-150,152-153,155-156,158-159,161-162,164-165,167-168,170-171,173-174,176-177,179-180,182-183,185-186,188-189,191-192,194-195,197-198,200-201,203-204,206-207,209-210,212-213,215-216,218-219,221-222,224-225,227-228,230-231,233-234,236-237,239-240,242,800") + self.assertEqual(str(r1), "2-3,5-6,8-9,11-12,14-15,17-18,20-21,23-24,26-27,29-30,32-33,35-36,38-39,41-42,44-45,47-48,50-51,53-54,56-57,59-60,62-63,65-66,68-69,71-72,74-75,77-78,80-81,83-84,86-87,89-90,92-93,95-96,98-99,102,105,107-108,110-111,113-114,116-117,119-120,122-123,125-126,128-129,131-132,134-135,137-138,140-141,143-144,146-147,149-150,152-153,155-156,158-159,161-162,164-165,167-168,170-171,173-174,176-177,179-180,182-183,185-186,188-189,191-192,194-195,197-198,200-201,203-204,206-207,209-210,212-213,215-216,218-219,221-222,224-225,227-228,230-231,233-234,236-237,239-240,242,800") self.assertEqual(len(r1), 160) r1 = RangeSet("1-1000", autostep=3) @@ -327,8 +336,8 @@ def testContains(self): self.assertEqual(len(r1), 240) self.assertTrue(99 in r1) self.assertTrue("99" in r1) - self.assertTrue("099" in r1) - self.assertRaises(TypeError, r1.__contains__, object()) + self.assertFalse("099" in r1) # fixed in 1.9+ + self.assertFalse(object() in r1) self.assertTrue(101 not in r1) self.assertEqual(len(r1), 240) r2 = RangeSet("1-100/3,40-60/3", autostep=3) @@ -340,16 +349,16 @@ def testContains(self): self.assertTrue(40 in r2) self.assertTrue(101 not in r2) r3 = RangeSet("0003-0143,0360-1000") - self.assertTrue(360 in r3) - self.assertTrue("360" in r3) + self.assertFalse(360 in r3) # fixed in 1.9+ + self.assertFalse("360" in r3) # fixed in 1.9+ self.assertTrue("0360" in r3) r4 = RangeSet("00-02") self.assertTrue("00" in r4) - self.assertTrue(0 in r4) - self.assertTrue("0" in r4) + self.assertFalse(0 in r4) # changed in 1.9+ + self.assertFalse("0" in r4) # fixed in 1.9+ self.assertTrue("01" in r4) - self.assertTrue(1 in r4) - self.assertTrue("1" in r4) + self.assertFalse(1 in r4) # changed in 1.9+ + self.assertFalse("1" in r4) # fixed in 1.9+ self.assertTrue("02" in r4) self.assertFalse("03" in r4) # @@ -400,31 +409,31 @@ def testIsSubSet(self): self.assertFalse(r1 < r2) self.assertFalse(r1 <= r2) self.assertFalse(r2 >= r1) - # since v1.6, padding is ignored when computing set operations + # fixed in v1.9 where mixed padding is now supported r1 = RangeSet("1-100") r2 = RangeSet("001-100") - self.assertTrue(r1.issubset(r2)) + self.assertFalse(r1.issubset(r2)) # used to be true < v1.9 def testGetItem(self): """test RangeSet.__getitem__()""" r1 = RangeSet("1-100,102,105-242,800") self.assertEqual(len(r1), 240) - self.assertEqual(r1[0], 1) - self.assertEqual(r1[1], 2) - self.assertEqual(r1[2], 3) - self.assertEqual(r1[99], 100) - self.assertEqual(r1[100], 102) - self.assertEqual(r1[101], 105) - self.assertEqual(r1[102], 106) - self.assertEqual(r1[103], 107) - self.assertEqual(r1[237], 241) - self.assertEqual(r1[238], 242) - self.assertEqual(r1[239], 800) + self.assertEqual(r1[0], '1') + self.assertEqual(r1[1], '2') + self.assertEqual(r1[2], '3') + self.assertEqual(r1[99], '100') + self.assertEqual(r1[100], '102') + self.assertEqual(r1[101], '105') + self.assertEqual(r1[102], '106') + self.assertEqual(r1[103], '107') + self.assertEqual(r1[237], '241') + self.assertEqual(r1[238], '242') + self.assertEqual(r1[239], '800') self.assertRaises(IndexError, r1.__getitem__, 240) self.assertRaises(IndexError, r1.__getitem__, 241) # negative indices - self.assertEqual(r1[-1], 800) - self.assertEqual(r1[-240], 1) + self.assertEqual(r1[-1], '800') + self.assertEqual(r1[-240], '1') for n in range(1, len(r1)): self.assertEqual(r1[-n], r1[len(r1)-n]) self.assertRaises(IndexError, r1.__getitem__, -len(r1)-1) @@ -432,19 +441,19 @@ def testGetItem(self): r2 = RangeSet("1-37/3,43-52/3,58-67/3,73-100/3,102-106/2") self.assertEqual(len(r2), 34) - self.assertEqual(r2[0], 1) - self.assertEqual(r2[1], 4) - self.assertEqual(r2[2], 7) - self.assertEqual(r2[12], 37) - self.assertEqual(r2[13], 43) - self.assertEqual(r2[14], 46) - self.assertEqual(r2[16], 52) - self.assertEqual(r2[17], 58) - self.assertEqual(r2[29], 97) - self.assertEqual(r2[30], 100) - self.assertEqual(r2[31], 102) - self.assertEqual(r2[32], 104) - self.assertEqual(r2[33], 106) + self.assertEqual(r2[0], '1') + self.assertEqual(r2[1], '4') + self.assertEqual(r2[2], '7') + self.assertEqual(r2[12], '37') + self.assertEqual(r2[13], '43') + self.assertEqual(r2[14], '46') + self.assertEqual(r2[16], '52') + self.assertEqual(r2[17], '58') + self.assertEqual(r2[29], '97') + self.assertEqual(r2[30], '100') + self.assertEqual(r2[31], '102') + self.assertEqual(r2[32], '104') + self.assertEqual(r2[33], '106') self.assertRaises(TypeError, r2.__getitem__, "foo") def testGetSlice(self): @@ -581,22 +590,22 @@ def testAdd(self): self.assertEqual(len(r1), 240) r1.add(801) self.assertEqual(len(r1), 241) - self.assertEqual(r1[0], 1) - self.assertEqual(r1[240], 801) + self.assertEqual(r1[0], '1') + self.assertEqual(r1[240], '801') r1.add(788) self.assertEqual(str(r1), "1-100,102,105-242,788,800-801") self.assertEqual(len(r1), 242) - self.assertEqual(r1[0], 1) - self.assertEqual(r1[239], 788) - self.assertEqual(r1[240], 800) + self.assertEqual(r1[0], '1') + self.assertEqual(r1[239], '788') + self.assertEqual(r1[240], '800') r1.add(812) self.assertEqual(len(r1), 243) # test forced padding r1 = RangeSet("1-100,102,105-242,800") r1.add(801, pad=3) self.assertEqual(len(r1), 241) - self.assertEqual(str(r1), "001-100,102,105-242,800-801") - r1.padding = 4 + self.assertEqual(str(r1), "1-100,102,105-242,800-801") + r1.padding = 4 # 1.8-1.9 compat: adjust padding of the whole set self.assertEqual(len(r1), 241) self.assertEqual(str(r1), "0001-0100,0102,0105-0242,0800-0801") @@ -647,10 +656,9 @@ def testRemove(self): self.assertEqual(len(r1), 239) self.assertEqual(str(r1), "1-99,102,105-242,800") self.assertRaises(KeyError, r1.remove, 101) - # test remove integer-castable type (convenience) + self.assertRaises(KeyError, r1.remove, "101") r1.remove("106") - # non integer castable cases raise ValueError (documented since 1.6) - self.assertRaises(ValueError, r1.remove, "foo") + self.assertRaises(KeyError, r1.remove, "foo") def testDiscard(self): """test RangeSet.discard()""" @@ -660,8 +668,9 @@ def testDiscard(self): self.assertEqual(len(r1), 239) self.assertEqual(str(r1), "1-99,102,105-242,800") r1.discard(101) # should not raise KeyError - # test remove integer-castable type (convenience) - r1.remove("106") + r1.discard('105') + self.assertEqual(len(r1), 238) + self.assertEqual(str(r1), "1-99,102,106-242,800") r1.discard("foo") def testClear(self): @@ -728,7 +737,7 @@ def testFromOneConstructor(self): def testIterator(self): """test RangeSet iterator""" - matches = [ 1, 3, 4, 5, 6, 7, 8, 11 ] + matches = ['1', '3', '4', '5', '6', '7', '8', '11'] rgs = RangeSet.fromlist([ "11", "3", "5-8", "1", "4" ]) cnt = 0 for rg in rgs: @@ -736,10 +745,12 @@ def testIterator(self): cnt += 1 self.assertEqual(cnt, len(matches)) # with padding + matches = ['001', '003', '004', '005', '006', '007', '008', '011'] rgs = RangeSet.fromlist([ "011", "003", "005-008", "001", "004" ]) cnt = 0 for rg in rgs: - self.assertTrue(isinstance(rg, int)) + self.assertFalse(isinstance(rg, int)) # true prior to v1.9 + self.assertTrue(isinstance(rg, str)) # true since v1.9 self.assertEqual(rg, matches[cnt]) cnt += 1 self.assertEqual(cnt, len(matches)) @@ -881,11 +892,11 @@ def testAddRange(self): r1.add_range(40, 65, 10) self.assertEqual(r1.autostep, 3) self.assertEqual(len(r1), 33) - self.assertEqual(str(r1), "1-29,30-60/10") + self.assertEqual(str(r1), "1-30,40-60/10") # One r1.add_range(103, 104) self.assertEqual(len(r1), 34) - self.assertEqual(str(r1), "1-29,30-60/10,103") + self.assertEqual(str(r1), "1-30,40-60/10,103") # Zero self.assertRaises(AssertionError, r1.add_range, 103, 103) @@ -928,15 +939,19 @@ def testCopy(self): self.assertEqual(str(r1), "115-117,130,167-170,4780-4999") self.assertEqual(str(r2), "115-118,130,166-170,4780-4999") + # unpickling tests; generate data with: + # rs = RangeSet("5,7-102,104,106-107") + # print(binascii.b2a_base64(pickle.dumps(rs))) + def test_unpickle_v1_3_py24(self): """test RangeSet unpickling (against v1.3/py24)""" rngset = pickle.loads(binascii.a2b_base64("gAIoY0NsdXN0ZXJTaGVsbC5Ob2RlU2V0ClJhbmdlU2V0CnEAb3EBfXECKFUHX2xlbmd0aHEDS2RVCV9hdXRvc3RlcHEER1SySa0llMN9VQdfcmFuZ2VzcQVdcQYoKEsFSwVLAUsAdHEHKEsHS2ZLAUsAdHEIKEtoS2hLAUsAdHEJKEtqS2tLAUsAdHEKZXViLg==")) self.assertEqual(rngset, RangeSet("5,7-102,104,106-107")) self.assertEqual(str(rngset), "5,7-102,104,106-107") self.assertEqual(len(rngset), 100) - self.assertEqual(rngset[0], 5) - self.assertEqual(rngset[1], 7) - self.assertEqual(rngset[-1], 107) + self.assertEqual(rngset[0], '5') + self.assertEqual(rngset[1], '7') + self.assertEqual(rngset[-1], '107') def test_unpickle_v1_3_py26(self): """test RangeSet unpickling (against v1.3/py26)""" @@ -944,9 +959,9 @@ def test_unpickle_v1_3_py26(self): self.assertEqual(rngset, RangeSet("5,7-102,104,106-107")) self.assertEqual(str(rngset), "5,7-102,104,106-107") self.assertEqual(len(rngset), 100) - self.assertEqual(rngset[0], 5) - self.assertEqual(rngset[1], 7) - self.assertEqual(rngset[-1], 107) + self.assertEqual(rngset[0], '5') + self.assertEqual(rngset[1], '7') + self.assertEqual(rngset[-1], '107') # unpickle_v1_4_py24 : unpickling fails as v1.4 does not have slice pickling workaround @@ -956,9 +971,9 @@ def test_unpickle_v1_4_py26(self): self.assertEqual(rngset, RangeSet("5,7-102,104,106-107")) self.assertEqual(str(rngset), "5,7-102,104,106-107") self.assertEqual(len(rngset), 100) - self.assertEqual(rngset[0], 5) - self.assertEqual(rngset[1], 7) - self.assertEqual(rngset[-1], 107) + self.assertEqual(rngset[0], '5') + self.assertEqual(rngset[1], '7') + self.assertEqual(rngset[-1], '107') def test_unpickle_v1_5_py24(self): """test RangeSet unpickling (against v1.5/py24)""" @@ -966,9 +981,9 @@ def test_unpickle_v1_5_py24(self): self.assertEqual(rngset, RangeSet("5,7-102,104,106-107")) self.assertEqual(str(rngset), "5,7-102,104,106-107") self.assertEqual(len(rngset), 100) - self.assertEqual(rngset[0], 5) - self.assertEqual(rngset[1], 7) - self.assertEqual(rngset[-1], 107) + self.assertEqual(rngset[0], '5') + self.assertEqual(rngset[1], '7') + self.assertEqual(rngset[-1], '107') def test_unpickle_v1_5_py26(self): """test RangeSet unpickling (against v1.5/py26)""" @@ -977,9 +992,9 @@ def test_unpickle_v1_5_py26(self): self.assertEqual(rngset, RangeSet("5,7-102,104,106-107")) self.assertEqual(str(rngset), "5,7-102,104,106-107") self.assertEqual(len(rngset), 100) - self.assertEqual(rngset[0], 5) - self.assertEqual(rngset[1], 7) - self.assertEqual(rngset[-1], 107) + self.assertEqual(rngset[0], '5') + self.assertEqual(rngset[1], '7') + self.assertEqual(rngset[-1], '107') def test_unpickle_v1_6_py24(self): """test RangeSet unpickling (against v1.6/py24)""" @@ -987,9 +1002,9 @@ def test_unpickle_v1_6_py24(self): self.assertEqual(rngset, RangeSet("5,7-102,104,106-107")) self.assertEqual(str(rngset), "5,7-102,104,106-107") self.assertEqual(len(rngset), 100) - self.assertEqual(rngset[0], 5) - self.assertEqual(rngset[1], 7) - self.assertEqual(rngset[-1], 107) + self.assertEqual(rngset[0], '5') + self.assertEqual(rngset[1], '7') + self.assertEqual(rngset[-1], '107') def test_unpickle_v1_6_py26(self): """test RangeSet unpickling (against v1.6/py26)""" @@ -997,9 +1012,27 @@ def test_unpickle_v1_6_py26(self): self.assertEqual(rngset, RangeSet("5,7-102,104,106-107")) self.assertEqual(str(rngset), "5,7-102,104,106-107") self.assertEqual(len(rngset), 100) - self.assertEqual(rngset[0], 5) - self.assertEqual(rngset[1], 7) - self.assertEqual(rngset[-1], 107) + self.assertEqual(rngset[0], '5') + self.assertEqual(rngset[1], '7') + self.assertEqual(rngset[-1], '107') + + def test_unpickle_v1_8_4_py27(self): + """test RangeSet unpickling (against v1.8.4/py27)""" + rngset = pickle.loads(binascii.a2b_base64("Y0NsdXN0ZXJTaGVsbC5SYW5nZVNldApSYW5nZVNldApwMAooUyc1LDctMTAyLDEwNCwxMDYtMTA3JwpwMQp0cDIKUnAzCihkcDQKUydwYWRkaW5nJwpwNQpOc1MnX2F1dG9zdGVwJwpwNgpGMWUrMTAwCnNTJ192ZXJzaW9uJwpwNwpJMwpzYi4=")) + self.assertEqual(rngset, RangeSet("5,7-102,104,106-107")) + self.assertEqual(str(rngset), "5,7-102,104,106-107") + self.assertEqual(len(rngset), 100) + self.assertEqual(rngset[0], '5') + self.assertEqual(rngset[1], '7') + self.assertEqual(rngset[-1], '107') + + rngset = pickle.loads(binascii.a2b_base64("Y0NsdXN0ZXJTaGVsbC5SYW5nZVNldApSYW5nZVNldApwMAooUycwMDAzLTAwNDAsMDA1OS0xNDAwJwpwMQp0cDIKUnAzCihkcDQKUydwYWRkaW5nJwpwNQpJNApzUydfYXV0b3N0ZXAnCnA2CkYxZSsxMDAKc1MnX3ZlcnNpb24nCnA3CkkzCnNiLg==")) + self.assertEqual(rngset, RangeSet("0003-0040,0059-1400")) + self.assertEqual(str(rngset), "0003-0040,0059-1400") + self.assertEqual(len(rngset), 1380) + self.assertEqual(rngset[0], '0003') + self.assertEqual(rngset[1], '0004') + self.assertEqual(rngset[-1], '1400') def test_pickle_current(self): """test RangeSet pickling (current version)""" @@ -1008,9 +1041,9 @@ def test_pickle_current(self): rngset = pickle.loads(dump) self.assertEqual(rngset, RangeSet("1-100")) self.assertEqual(str(rngset), "1-100") - self.assertEqual(rngset[0], 1) - self.assertEqual(rngset[1], 2) - self.assertEqual(rngset[-1], 100) + self.assertEqual(rngset[0], '1') + self.assertEqual(rngset[1], '2') + self.assertEqual(rngset[-1], '100') def testIntersectionLength(self): """test RangeSet intersection/length""" @@ -1053,9 +1086,9 @@ def testFolding(self): r1 = RangeSet("1,3-4,6,8", autostep=4) self.assertEqual(str(r1), "1,3-4,6,8") r1 = RangeSet("1,3-4,6,8", autostep=2) - self.assertEqual(str(r1), "1,3,4-8/2") + self.assertEqual(str(r1), "1-3/2,4,6-8/2") r1 = RangeSet("1,3-4,6,8", autostep=3) - self.assertEqual(str(r1), "1,3,4-8/2") + self.assertEqual(str(r1), "1,3-4,6,8") # empty set r1 = RangeSet(autostep=3) @@ -1122,12 +1155,111 @@ def test_intiter(self): cnt += 1 self.assertEqual(cnt, len(matches)) # with mixed length padding (add 01, 09 and 0001): not supported until 1.9 - #matches = [ 1, 9 ] + matches + [ 1 ] - #rgs = RangeSet.fromlist([ "011", "01", "003", "005-008", "001", "0001", "09", "004" ]) - #cnt = 0 - #for rg in rgs.intiter(): - # print(rg) - # self.assertTrue(isinstance(rg, int)) - # self.assertEqual(rg, matches[cnt]) - # cnt += 1 - #self.assertEqual(cnt, len(matches)) + matches = [ 1, 9 ] + matches + [ 1 ] + rgs = RangeSet.fromlist([ "011", "01", "003", "005-008", "001", "0001", "09", "004" ]) + cnt = 0 + for rg in rgs.intiter(): + self.assertTrue(isinstance(rg, int)) + self.assertEqual(rg, matches[cnt]) + cnt += 1 + self.assertEqual(cnt, len(matches)) + + def test_mixed_padding(self): + r0 = RangeSet("030-031,032-100/2,101-489", autostep=3) + self.assertEqual(str(r0), "030-032,034-100/2,101-489") + r1 = RangeSet("030-032,033-100/3,102", autostep=3) + self.assertEqual(str(r1), "030-033,036-102/3") + r2 = RangeSet("030-032,033-100/3,101", autostep=3) + self.assertEqual(str(r2), "030-033,036-099/3,101") + r3 = RangeSet("030-032,033-100/3,100", autostep=3) + self.assertEqual(str(r3), "030-033,036-099/3,100") + r5 = RangeSet("030-032,033-100/3,99-105/3,0001", autostep=3) + self.assertEqual(str(r5), "99,030-033,036-105/3,0001") + + def test_mixed_padding_mismatch(self): + self.assertRaises(RangeSetParseError, RangeSet, "1-044") + self.assertRaises(RangeSetParseError, RangeSet, "01-044") + self.assertRaises(RangeSetParseError, RangeSet, "001-44") + + self.assertRaises(RangeSetParseError, RangeSet, "0-9,1-044") + self.assertRaises(RangeSetParseError, RangeSet, "0-9,01-044") + self.assertRaises(RangeSetParseError, RangeSet, "0-9,001-44") + + self.assertRaises(RangeSetParseError, RangeSet, "030-032,033-99/3,100") + + def test_padding_property_compat(self): + r0 = RangeSet("0-10,15-20") + self.assertEqual(r0.padding, None) + r0.padding = 1 + self.assertEqual(r0.padding, None) + self.assertEqual(str(r0), "0-10,15-20") + r0.padding = 2 + self.assertEqual(r0.padding, 2) + self.assertEqual(str(r0), "00-10,15-20") + r0.padding = 3 + self.assertEqual(r0.padding, 3) + self.assertEqual(str(r0), "000-010,015-020") + # reset padding using None is allowed + r0.padding = None + self.assertEqual(r0.padding, None) + self.assertEqual(str(r0), "0-10,15-20") + + def test_strip_whitespaces(self): + r0 = RangeSet(" 1,5,39-42,100") + self._testRS(" 1,5,39-42,100", "1,5,39-42,100", 7) + self._testRS("1 ,5,39-42,100", "1,5,39-42,100", 7) + self._testRS("1, 5,39-42,100", "1,5,39-42,100", 7) + self._testRS("1,5 ,39-42,100", "1,5,39-42,100", 7) + self._testRS("1,5, 39-42,100", "1,5,39-42,100", 7) + self._testRS("1,5,39-42 ,100", "1,5,39-42,100", 7) + self._testRS("1,5,39-42, 100", "1,5,39-42,100", 7) + self._testRS("1,5,39-42,100 ", "1,5,39-42,100", 7) + self._testRS(" 1 ,5,39-42,100", "1,5,39-42,100", 7) + self._testRS("1 , 5 , 39-42 , 100", "1,5,39-42,100", 7) + self._testRS(" 1 , 5 , 39-42 , 100 ", "1,5,39-42,100", 7) + + # whitespaces within ranges + self._testRS("1 - 2", "1-2", 2) + self._testRS("01 - 02", "01-02", 2) + self._testRS("01- 02", "01-02", 2) + self._testRS("01 -02", "01-02", 2) + self._testRS(" 01-02", "01-02", 2) + self._testRS(" 01 -02", "01-02", 2) + self._testRS(" 01 - 02", "01-02", 2) + self._testRS(" 01 - 02 ", "01-02", 2) + self._testRS("01 - 02 ", "01-02", 2) + self._testRS("01- 02 ", "01-02", 2) + self._testRS("01-02 ", "01-02", 2) + + self._testRS("0-8/2", "0-8/2", 5) + self._testRS("1-7/2", "1-7/2", 4) + self._testRS("0-8 /2", "0-8/2", 5) + self._testRS("1-7 /2", "1-7/2", 4) + self._testRS("0-8/ 2", "0-8/2", 5) + self._testRS("1-7/ 2", "1-7/2", 4) + self._testRS("0-8 / 2", "0-8/2", 5) + self._testRS("1-7 / 2", "1-7/2", 4) + self._testRS("0 -8 / 2", "0-8/2", 5) + self._testRS("1 -7 / 2", "1-7/2", 4) + self._testRS("0 - 8 / 2", "0-8/2", 5) + self._testRS("1 - 7 / 2", "1-7/2", 4) + self._testRS("00-08/2", "00-08/2", 5) + self._testRS("01-07/2", "01-07/2", 4) + self._testRS("00-08 /2", "00-08/2", 5) + self._testRS("01-07 /2", "01-07/2", 4) + self._testRS("00-08/ 2", "00-08/2", 5) + self._testRS("01-07/ 2", "01-07/2", 4) + self._testRS("00-08 / 2", "00-08/2", 5) + self._testRS("01-07 / 2", "01-07/2", 4) + + # invalid patterns + self.assertRaises(RangeSetParseError, RangeSet, " 0 0") + self.assertRaises(RangeSetParseError, RangeSet, " 1 2") + self.assertRaises(RangeSetParseError, RangeSet, "0 1") + self.assertRaises(RangeSetParseError, RangeSet, "0 1 ") + self.assertRaises(RangeSetParseError, RangeSet, "1,5,39-42,10 0") + self.assertRaises(RangeSetParseError, RangeSet, "1,5,39-42,12 3,300") + self.assertRaises(RangeSetParseError, RangeSet, "1,5,") + self.assertRaises(RangeSetParseError, RangeSet, "1,5,") + self.assertRaises(RangeSetParseError, RangeSet, "1,5, ") + self.assertRaises(RangeSetParseError, RangeSet, "1,5,, ")