-
Notifications
You must be signed in to change notification settings - Fork 118
Support for concatenated objects #10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I like that, thanks. that's the idea for the API change:
This makes the API for the obvious case, obvious. The more complex API will impact only more demanding users. Thanks for the suggestion, |
This API would work. I do the opposite for return values (offset, object) and I return nil in the offset if the decoding failed (which I use for error checking) but that's a matter of style. In the context of Redis your ordering is simpler. |
Inspired by antirez#10, I added two new functions allowing you to limit multiple return from unpack. If you don't trust your msgpack input or want to manage flow control of returned object generation yourself, you can now limit the objects returned from unpack (but not with unpack() itself because it already returns an arbitrary number of objects). You can now unpack_one(msgpack) to get (next_offset, obj) returned; then call unpack_one(msgpack, next_offset) to get another offset and one more object> You can now also unpack_limit(msgpack, N) to get (next_offset, obj1, obj2, ..., objN) returned; then call unpack_limit(msgpack, K, next_offset) to get another offset and K more objects. This also refactors the copy/paste command addition in module bring-up. Tests added for new commands and all tests pass: TEST PASSED: 225 TEST FAILED: 0 TEST SKIPPED: 0 Closes antirez#10
Changes: - Improve table vs. array detection - Improve packing +-inf - Add multiple pack/unpack support - Add cmsgpack.safe module variant - Add local build infrastructure for easier testing - Add user-controlled unpack support limiting returned objects - Add Lua 5.3 compatibility - Remove an unnecessary malloc - Use Lua memory allocator instead of malloc for buffer creation Issues involved: - closes antirez#16 - allow multi pack/unpack by default - closes antirez#10 - unpack one/limit API added - closes antirez#13 and closes antirez#20 - use Lua allocator - closes antirez#15 - (included in antirez#16) - ignores antirez#22 because it's confusing - closes antirez#23 - fixed elsewhere - closes antirez#26 - extracted some useful parts from a difficult commit - closes antirez#28 - we started tagging versions again recently - closes antirez#27 - that failure case works for me now I merged commits with original author information where possible, but each commit required manual cleanup of one or more of: formatting fixes (no tabs, please), commit message fixes (more details please), extracting contents from a single 300 line commit with 5 different logical changes merged together, and general correctness checking after merging with newer code. As of this commit, all tests pass on Lua 5.1.5 and Lua 5.3-work2.
Changes: - Improve table vs. array detection - Improve packing +-inf - Add multiple pack/unpack support - Add cmsgpack.safe module variant - Add local build infrastructure for easier testing - Add user-controlled unpack support limiting returned objects - Add Lua 5.3 compatibility - Remove an unnecessary malloc - Use Lua memory allocator instead of malloc for buffer creation Issues involved: - closes antirez#16 - allow multi pack/unpack by default - closes antirez#10 - unpack one/limit API added - closes antirez#13 and closes antirez#20 - use Lua allocator - closes antirez#15 - (included in antirez#16) - ignores antirez#22 because it's confusing - closes antirez#23 - fixed elsewhere - closes antirez#26 - extracted some useful parts from a difficult commit - closes antirez#28 - we started tagging versions again recently - closes antirez#27 - that failure case works for me now I merged commits with original author information where possible, but each commit required manual cleanup of one or more of: formatting fixes (no tabs, please), commit message fixes (more details please), extracting contents from a single 300 line commit with 5 different logical changes merged together, and general correctness checking after merging with newer code. As of this commit, all tests pass on Lua 5.1.5 and Lua 5.3-work2.
Changes: - Improve table vs. array detection - Improve packing +-inf - Add multiple pack/unpack support - Add cmsgpack.safe module variant - Add local build infrastructure for easier testing - Add user-controlled unpack support limiting returned objects - Add Lua 5.3 compatibility - Remove an unnecessary malloc - Use Lua memory allocator instead of malloc for buffer creation Issues involved: - closes antirez#16 - allow multi pack/unpack by default - closes antirez#10 - unpack one/limit API added - closes antirez#13 and closes antirez#20 - use Lua allocator - closes antirez#15 - (included in antirez#16) - ignores antirez#22 because it's confusing - closes antirez#23 - fixed elsewhere - closes antirez#26 - extracted some useful parts from a difficult commit - closes antirez#28 - we started tagging versions again recently - closes antirez#27 - that failure case works for me now I merged commits with original author information where possible, but each commit required manual cleanup of one or more of: formatting fixes (no tabs, please), commit message fixes (more details please), extracting contents from a single 300 line commit with 5 different logical changes merged together, and general correctness checking after merging with newer code. As of this commit, all tests pass on Lua 5.1.5 and Lua 5.3-work2.
Changes: - Improve table vs. array detection - Improve packing +-inf - Add multiple pack/unpack support - Add cmsgpack.safe module variant - Add local build infrastructure for easier testing - Add user-controlled unpack support limiting returned objects - Add Lua 5.3 compatibility - Remove an unnecessary malloc - Use Lua memory allocator instead of malloc for buffer creation Issues involved: - closes antirez#16 - allow multi pack/unpack by default - closes antirez#10 - unpack one/limit API added - closes antirez#13 and closes antirez#20 - use Lua allocator - closes antirez#15 - (included in antirez#16) - ignores antirez#22 because it's confusing - closes antirez#23 - fixed elsewhere - closes antirez#26 - extracted some useful parts from a difficult commit - closes antirez#28 - we started tagging versions again recently - closes antirez#27 - that failure case works for me now I merged commits with original author information where possible, but each commit required manual cleanup of one or more of: formatting fixes (no tabs, please), commit message fixes (more details please), extracting contents from a single 300 line commit with 5 different logical changes merged together, and general correctness checking after merging with newer code. As of this commit, all tests pass on Lua 5.1.5 and Lua 5.3-work2.
Changes: - Improve table vs. array detection - Improve packing +-inf - Add multiple pack/unpack support - Add cmsgpack.safe module variant - Add local build infrastructure for easier testing - Add user-controlled unpack support limiting returned objects - Add Lua 5.3 compatibility - Remove an unnecessary malloc - Use Lua memory allocator instead of malloc for buffer creation Issues involved: - closes antirez#16 - allow multi pack/unpack by default - closes antirez#10 - unpack one/limit API added - closes antirez#13 and closes antirez#20 - use Lua allocator - closes antirez#15 - (included in antirez#16) - ignores antirez#22 because it's confusing - closes antirez#23 - fixed elsewhere - closes antirez#26 - extracted some useful parts from a difficult commit - closes antirez#28 - we started tagging versions again recently - closes antirez#27 - that failure case works for me now I merged commits with original author information where possible, but each commit required manual cleanup of one or more of: formatting fixes (no tabs, please), commit message fixes (more details please), extracting contents from a single 300 line commit with 5 different logical changes merged together, and general correctness checking after merging with newer code. As of this commit, all tests pass on Lua 5.1.5 and Lua 5.3-work2.
Changes: - Improve table vs. array detection - Improve packing +-inf - Add multiple pack/unpack support - Add cmsgpack.safe module variant - Add local build infrastructure for easier testing - Add user-controlled unpack support limiting returned objects - Add Lua 5.3 compatibility - Remove an unnecessary malloc - Use Lua memory allocator instead of malloc for buffer creation Issues involved: - closes antirez#16 - allow multi pack/unpack by default - closes antirez#10 - unpack one/limit API added - closes antirez#13 and closes antirez#20 - use Lua allocator - closes antirez#15 - (included in antirez#16) - ignores antirez#22 because it's confusing - closes antirez#23 - fixed elsewhere - closes antirez#26 - extracted some useful parts from a difficult commit - closes antirez#28 - we started tagging versions again recently - closes antirez#27 - that failure case works for me now I merged commits with original author information where possible, but each commit required manual cleanup of one or more of: formatting fixes (no tabs, please), commit message fixes (more details please), extracting contents from a single 300 line commit with 5 different logical changes merged together, and general correctness checking after merging with newer code. As of this commit, all tests pass on Lua 5.1.5 and Lua 5.3-work2.
Changes: - Improve table vs. array detection - Improve packing +-inf - Add multiple pack/unpack support - Add cmsgpack.safe module variant - Add local build infrastructure for easier testing - Add user-controlled unpack support limiting returned objects - Add Lua 5.3 compatibility - Remove an unnecessary malloc - Use Lua memory allocator instead of malloc for buffer creation Issues involved: - closes antirez#16 - allow multi pack/unpack by default - closes antirez#10 - unpack one/limit API added - closes antirez#13 and closes antirez#20 - use Lua allocator - closes antirez#15 - (included in antirez#16) - ignores antirez#22 because it's confusing - closes antirez#23 - fixed elsewhere - closes antirez#26 - extracted some useful parts from a difficult commit - closes antirez#28 - we started tagging versions again recently - closes antirez#27 - that failure case works for me now I merged commits with original author information where possible, but each commit required manual cleanup of one or more of: formatting fixes (no tabs, please), commit message fixes (more details please), extracting contents from a single 300 line commit with 5 different logical changes merged together, and general correctness checking after merging with newer code. As of this commit, all tests pass on Lua 5.1.5 and Lua 5.3-work2.
Changes: - Improve table vs. array detection - Improve packing +-inf - Add multiple pack/unpack support - Add cmsgpack.safe module variant - Add local build infrastructure for easier testing - Add user-controlled unpack support limiting returned objects - Add Lua 5.3 compatibility - Remove an unnecessary malloc - Use Lua memory allocator instead of malloc for buffer creation Issues involved: - closes antirez#16 - allow multi pack/unpack by default - closes antirez#10 - unpack one/limit API added - closes antirez#13 and closes antirez#20 - use Lua allocator - closes antirez#15 - (included in antirez#16) - ignores antirez#22 because it's confusing - closes antirez#23 - fixed elsewhere - closes antirez#26 - extracted some useful parts from a difficult commit - closes antirez#28 - we started tagging versions again recently - closes antirez#27 - that failure case works for me now I merged commits with original author information where possible, but each commit required manual cleanup of one or more of: formatting fixes (no tabs, please), commit message fixes (more details please), extracting contents from a single 300 line commit with 5 different logical changes merged together, and general correctness checking after merging with newer code. As of this commit, all tests pass on Lua 5.1.5 and Lua 5.3-work2.
Changes: - Improve table vs. array detection - Improve packing +-inf - Add multiple pack/unpack support - Add cmsgpack.safe module variant - Add local build infrastructure for easier testing - Add user-controlled unpack support limiting returned objects - Add Lua 5.3 compatibility - Remove an unnecessary malloc - Use Lua memory allocator instead of malloc for buffer creation Issues involved: - closes antirez#16 - allow multi pack/unpack by default - closes antirez#10 - unpack one/limit API added - closes antirez#13 and closes antirez#20 - use Lua allocator - closes antirez#15 - (included in antirez#16) - ignores antirez#22 because it's confusing - closes antirez#23 - fixed elsewhere - closes antirez#26 - extracted some useful parts from a difficult commit - closes antirez#28 - we started tagging versions again recently - closes antirez#27 - that failure case works for me now - closes antirez#31 - fix comment typos I merged commits with original author information where possible, but each commit required manual cleanup of one or more of: formatting fixes (no tabs, please), commit message fixes (more details please), extracting contents from a single 300 line commit with 5 different logical changes merged together, and general correctness checking after merging with newer code. As of this commit, all tests pass on Lua 5.1.5 and Lua 5.3-work2.
Changes: - Improve table vs. array detection - Improve packing +-inf - Add multiple pack/unpack support - Add cmsgpack.safe module variant - Add local build infrastructure for easier testing - Add user-controlled unpack support limiting returned objects - Add Lua 5.3 compatibility - Remove an unnecessary malloc - Use Lua memory allocator instead of malloc for buffer creation Issues involved: - closes antirez#16 - allow multi pack/unpack by default - closes antirez#10 - unpack one/limit API added - closes antirez#13 and closes antirez#20 - use Lua allocator - closes antirez#15 - (included in antirez#16) - ignores antirez#22 because it's confusing - closes antirez#23 - fixed elsewhere - closes antirez#26 - extracted some useful parts from a difficult commit - closes antirez#28 - we started tagging versions again recently - closes antirez#27 - that failure case works for me now - closes antirez#31 - fix comment typos I merged commits with original author information where possible, but each commit required manual cleanup of one or more of: formatting fixes (no tabs, please), commit message fixes (more details please), extracting contents from a single 300 line commit with 5 different logical changes merged together, and general correctness checking after merging with newer code. As of this commit, all tests pass on Lua 5.1.5 and Lua 5.3-work2.
MessagePack allows binary blobs composed of several valid MessagePack objects concatenated. This is the case of the official MessagePack test data for instance (https://github.com/msgpack/msgpack/tree/master/test).
To support this kind of data you need an interface that:
An advantage of implementing this is that it would make it possible to use the "official" test data to test this module.
The text was updated successfully, but these errors were encountered: