Review general protocol implementations

[aturon]

Collect info on Rust implementations of protocols like http, http/2, grpc, thrift, and so on. What is the status of each?

[aajtodd]

I know of at least 3 gRPC implementations:

1. https://github.com/stepancheg/grpc-rust
   • Pure rust
   • Built on protobuf and httpbis
   • Futures based
2. https://github.com/pingcap/grpc-rs
   • Rust wrapper around gRPC C core
   • Futures based
3. https://github.com/tower-rs/tower-grpc/
   • Pure Rust
   • Built on prost and h2
   • Futures based

I know some people are fairly involved already in tower-grpc.

On the one hand the work pingcap did wrapping the official gRPC C core makes a lot of sense and provides a decent way to stay up to date with gRPC. On the other it is not a showcase for Rust really (not sure if that is a goal or not).

There are many challenges to gRPC outside of the networking part including protubuf support for Rust, code generation for service stubs, etc.

In my opinion the goal for a gRPC library in Rust should be to gain official support by the gRPC project and be incorporated into that ecosystem. Most languages are wrappers around the C core but there are exceptions (grpc-java and grpc-go are completely written in their respective languages). Being officially recognized as part of the gRPC project would go a long way to breaking down barriers to adopting Rust by companies looking to use Rust for network services/gRPC. I know at one time the gRPC maintainers indicated they would be willing to pull a high enough quality implementation into the main repo, not sure if their stance is the same or not.

[yoshuawuyts]

HTTP/2

Known Rust Implementations

A list of known implementations include:

• carllerche/h2 (9 commits/month - pulse)
• mlalic/solicit (1 commit/month - pulse)
• stepancheg/rust-http2 (published as httpbis) (0 commits/month - pulse)

Out of all the HTTP/2 implementations h2 seems to currently be getting the most traction.

Other Noteworthy Implementations

The nghttp2 C library is used in both cURL and Node.js.

As of January 2018 it seemed that while nginx supports HTTP/2 for egress, it did not support it for the connections it was proxying (e.g. it could not speak HTTP/2 to the servers "behind nginx").

A complete list of known HTTP2 implementations can be found on the http2-spec wiki.

HTTP/2 Push

The HTTP/2 spec adds support for PUSH frames. This allows multiple responses to be sent in response to 1 or fewer requests. This is particularly useful to reduce latency when serving web pages that contain multiple assets from the same origin (e.g. a mix of multiple images, fonts, scripts, etc.).

While h2 seems to support PUSH PROMISE frames (spec), at the time of writing implementing full HTTP/2 push support does not seem to be a priority. However external help appears to be welcomed.

HTTP/2 push support in h2 is also a dependency for landing http/2 push support in the Hyper HTTP library.

Author's note: after reading the HTTP/2 spec I was under the impression that PUSH PROMISE support was equivalent to HTTP/2 push support. Apparently h2 is able to handle PUSH PROMISE frames without completely handling HTTP/2 push.

Useful Links

• http2 spec (RFC 7540)
• http2 extension RFCs
• http2/http2-spec (github)
• list of known HTTP/2 implementations

Author's note: HTTP/2 is still actively worked on. I'm personally most excited for the cache-digest and websockets extensions.

Conclusion

I hope this provides a rough overview of the available HTTP/2 libraries in Rust, links to the current HTTP/2 specifications, and an overview on the state of HTTP/2 push support in Rust.

I did not dive deeper into specifics of how Rust's HTTP/2 story stacks up against other languages, did notengage further on the steps required to improve Rust's HTTP/2 support

I'm hoping this turns out to be somewhat useful. I for one am very excited to have complete HTTP/2 support in Rust! Thanks!

[polachok]

Kafka

I know of 3 implementations:

• kafka
  • pure rust
  • synchronous
  • only older protocol supported (0.8.2)
  • doesn't fully support consumer groups & rebalancing
• rdkafka
  • bindings to rdkafka C/C++ library
  • sync and async (with futures)
  • all features supported
• tokio-kafka
  • not ready, not published, unstable
  • async, uses tokio
  • older protocols supported

Personal experience

kafka-rs is ok, but doesn't support newer protocol versions which are much more efficient (better compression, etc).
tokio-kafka is promising, but not very usable at the moment. I think with community support it could be made into a production class library.
rdkafka is what we use in production, it's ok, but api looks a bit weird to me (subjective).