✨ Add Set/Range/Enum/etc methods to SequenceSet

The version of SequenceSet in net-imap prior to this commit was merely a
placeholder, needed in order to complete `tagged-ext` for #225.

This updates it with a full API, inspired by Set, Range, and Array.
This allows it to be more broadly useful, e.g. for storing and working
with mailbox state.

In addition to Integer, Range, and enumerables, any object with
`#to_sequence_set` can now be used to create a sequence set.  For
compatibility with MessageSet, `ThreadMember#to_sequence_set` collects
all child seqno into a SequenceSet.

Because mailbox state can be _very_ large, inputs are stored in an
internal sorted array of ranges.  These are stored as `[start, stop]`
tuples, not Range objects, for simpler manipulation.  A future
optimization could convert all tuples to a flat one-dimensional Array
(to reduce object allocations).  Storing the data in sorted range tuples
allows many of the important operations to be `O(lg n)`.

Although updates do use `Array#insert` and `Array#slice!`—which are
technically `O(n)`—they tend to be fast until the number of elements is
very large.  Count and index-based methods are also `O(n)`.  A future
optimization could cache the count and compose larger sets from a sorted
tree of smaller sets, to preserve `O(lg n)` for most operations.

SequenceSet can be used to replace MessageSet (which is used internally
to validate, format, and send certain command args).  Some notable
differences between the two:
* Most validation is done up-front, when initializing or adding values.
* A ThreadMember to `sequence-set` bug has been fixed.
* The generated string is sorted and adjacent ranges are combined.

TODO in future PRs:
* #index_lte => get the index of a number in the set, or if the number
  isn't in the set, the number before it.
* Replace or supplement the UID set implementation in UIDPlusData.
* fully replace MessageSet (probably not before v0.5.0)

Author: nevans

lib/net/imap/response_data.rb

@@ -763,6 +763,19 @@ class ThreadMember < Struct.new(:seqno, :children)
       #
       # An array of Net::IMAP::ThreadMember objects for mail items that are
       # children of this in the thread.
+
+      # Returns a SequenceSet containing #seqno and all #children's seqno,
+      # recursively.
+      def to_sequence_set
+        SequenceSet.new all_seqnos
+      end
+
+      protected
+
+      def all_seqnos(node = self)
+        [node.seqno].concat node.children.flat_map { _1.all_seqnos }
+      end
+
     end
 
     # Net::IMAP::BodyStructure is included by all of the structs that can be

lib/net/imap/response_parser.rb

@@ -464,7 +464,7 @@ def unescape_quoted(quoted)
       def sequence_set
         str = combine_adjacent(*SEQUENCE_SET_TOKENS)
         if Patterns::SEQUENCE_SET_STR.match?(str)
-          SequenceSet.new(str)
+          SequenceSet[str]
         else
           parse_error("unexpected atom %p, expected sequence-set", str)
         end

lib/net/imap/sequence_set.rb

@@ -51,7 +51,12 @@
           NUM: 1
           SEQ: !ruby/struct:Net::IMAP::ExtensionData
             data: !ruby/object:Net::IMAP::SequenceSet
-              atom: 1234:5,*:789654
+              string: 1234:5,*:789654
+              tuples:
+              - - 5
+                - 1234
+              - - 789654
+                - 4294967296
           COMP-EMPTY: !ruby/struct:Net::IMAP::ExtensionData
             data: []
           COMP-QUOTED: !ruby/struct:Net::IMAP::ExtensionData

test/net/imap/fixtures/response_parser/status_responses.yml

@@ -591,42 +591,60 @@ def test_send_invalid_number
       sock = server.accept
       begin
         sock.print("* OK test server\r\n")
-        sock.gets
+        sock.gets # Integer: 0
         sock.print("RUBY0001 OK TEST completed\r\n")
-        sock.gets
+        sock.gets # Integer: 2**32 - 1
         sock.print("RUBY0002 OK TEST completed\r\n")
-        sock.gets
+        sock.gets # MessageSet: 1
         sock.print("RUBY0003 OK TEST completed\r\n")
-        sock.gets
+        sock.gets # MessageSet: 2**32 - 1
         sock.print("RUBY0004 OK TEST completed\r\n")
-        sock.gets
+        sock.gets # SequenceSet: -1 => "*"
+        sock.print("RUBY0005 OK TEST completed\r\n")
+        sock.gets # SequenceSet: 1
+        sock.print("RUBY0006 OK TEST completed\r\n")
+        sock.gets # SequenceSet: 2**32 - 1
+        sock.print("RUBY0007 OK TEST completed\r\n")
+        sock.gets # LOGOUT
         sock.print("* BYE terminating connection\r\n")
-        sock.print("RUBY0005 OK LOGOUT completed\r\n")
+        sock.print("RUBY0008 OK LOGOUT completed\r\n")
       ensure
         sock.close
         server.close
       end
     end
     begin
+      # regular numbers may be any uint32
       imap = Net::IMAP.new(server_addr, :port => port)
       assert_raise(Net::IMAP::DataFormatError) do
         imap.__send__(:send_command, "TEST", -1)
       end
       imap.__send__(:send_command, "TEST", 0)
-      imap.__send__(:send_command, "TEST", 4294967295)
+      imap.__send__(:send_command, "TEST", 2**32 - 1)
       assert_raise(Net::IMAP::DataFormatError) do
-        imap.__send__(:send_command, "TEST", 4294967296)
+        imap.__send__(:send_command, "TEST", 2**32)
       end
+      # MessageSet numbers may be non-zero uint32
       assert_raise(Net::IMAP::DataFormatError) do
         imap.__send__(:send_command, "TEST", Net::IMAP::MessageSet.new(-1))
       end
       assert_raise(Net::IMAP::DataFormatError) do
         imap.__send__(:send_command, "TEST", Net::IMAP::MessageSet.new(0))
       end
       imap.__send__(:send_command, "TEST", Net::IMAP::MessageSet.new(1))
-      imap.__send__(:send_command, "TEST", Net::IMAP::MessageSet.new(4294967295))
+      imap.__send__(:send_command, "TEST", Net::IMAP::MessageSet.new(2**32 - 1))
+      assert_raise(Net::IMAP::DataFormatError) do
+        imap.__send__(:send_command, "TEST", Net::IMAP::MessageSet.new(2**32))
+      end
+      # SequenceSet numbers may be non-zero uint3, and -1 is translated to *
+      imap.__send__(:send_command, "TEST", Net::IMAP::SequenceSet.new(-1))
+      assert_raise(Net::IMAP::DataFormatError) do
+        imap.__send__(:send_command, "TEST", Net::IMAP::SequenceSet.new(0))
+      end
+      imap.__send__(:send_command, "TEST", Net::IMAP::SequenceSet.new(1))
+      imap.__send__(:send_command, "TEST", Net::IMAP::SequenceSet.new(2**32-1))
       assert_raise(Net::IMAP::DataFormatError) do
-        imap.__send__(:send_command, "TEST", Net::IMAP::MessageSet.new(4294967296))
+        imap.__send__(:send_command, "TEST", Net::IMAP::SequenceSet.new(2**32))
       end
       imap.logout
     ensure

test/net/imap/test_imap.rb

@@ -35,4 +35,23 @@ def test_uidplus_copyuid__uid_mapping
     )
   end
 
+  def test_thread_member_to_sequence_set
+    # copied from the fourth example in RFC5256: (3 6 (4 23)(44 7 96))
+    thmember = Net::IMAP::ThreadMember.method :new
+    thread = thmember.(3, [
+      thmember.(6, [
+        thmember.(4, [
+          thmember.(23, [])
+        ]),
+        thmember.(44, [
+          thmember.(7, [
+            thmember.(96, [])
+          ])
+        ])
+      ])
+    ])
+    expected = Net::IMAP::SequenceSet.new("3:4,6:7,23,44,96")
+    assert_equal(expected, thread.to_sequence_set)
+  end
+
 end