Update library docs and readme

Author: fede1024

README.md

@@ -8,24 +8,29 @@
 Kafka client library for Rust based on [librdkafka].
 
 ## The library
-This library aims to provide a safe interface to librdkafka.
-It currently exports some of the funcionalities provided by the producer and consumer
-of librdkafka 0.9.2.
+`rust-rdkafka` provides a safe Rust interface to librdkafka.
+It currently exports a subset of the funcionalities provided by librdkafka 0.9.2.
 
-Producers and consumers can be accessed and polled directly, or alternatively
-a [futures]-based interface can be used:
+`rust-rdkafka` provides low level and high level consumers and producers. Low level:
 
-* A consumer will return a [`stream`] of messages, as they are received from Kafka.
-* A producer will return a [`future`] that will eventually contain the delivery
-status of the message.
+* `BaseConsumer`: simple wrapper around the librdkafka consumer. It requires to be
+  periodically `poll()`ed in order to execute callbacks, rebalance and receive messages.
+* `BaseProducer`: simple wrapper around the librdkafka producer. As in the consumer case,
+  the user must call `poll()` periodically to execute delivery callbacks.
+
+High level:
+
+ * `StreamConsumer`: it returns a [`stream`] of messages and takes care of polling the consumer
+ internally.
+ * `FutureProducer`: it returns a [`future`] that will be completed once the message is
+ delivered to Kafka (or fails).
 
 [librdkafka]: https://github.com/edenhill/librdkafka
 [futures]: https://github.com/alexcrichton/futures-rs
 [`future`]: https://docs.rs/futures/0.1.3/futures/trait.Future.html
 [`stream`]: https://docs.rs/futures/0.1.3/futures/stream/trait.Stream.html
 
-*Warning*: this library is still at an early development stage, the API is very likely
-to change and it shouldn't be considered production ready.
+*Warning*: the library is under active development and the APIs are likely to change.
 
 ## Installation
 
@@ -36,7 +41,7 @@ Add this to your `Cargo.toml`:
 rdkafka = "^0.2.0"
 ```
 
-This crate will compile librdkafka from sources and link it statically in your
+This crate will compile librdkafka from sources and link it statically to your
 executable. To compile librdkafka you'll need:
 
 * the GNU toolchain

generate_readme.py

@@ -0,0 +1,54 @@
+#!/usr/bin/python3
+
+from collections import namedtuple
+import sys
+
+
+INCLUDE_MARKER = "__INCLUDE_RUST_DOC__"
+
+Template = namedtuple("Template", ["header", "footer", "doc_path", "start"])
+
+
+def read_rust_doc_lines(path):
+    with open(path, "r") as rust_doc:
+        for line in rust_doc:
+            if line.startswith('//! '):
+                yield line[4:]
+            elif line.startswith('//!'):
+                yield line[3:]
+            else:
+                break
+
+
+def parse_template_file(path):
+    content = [line for line in open(path, "r")]
+    try:
+        marker_position = [n for (n, line) in enumerate(content)
+                           if line.startswith(INCLUDE_MARKER)][0]
+    except IndexError:
+        raise Exception("Missing include marker")
+    include_info = content[marker_position].split('$')
+    return Template(
+        header=content[0:marker_position], footer=content[marker_position+1:],
+        doc_path=include_info[1], start=include_info[2],
+    )
+
+
+template = parse_template_file("readme_template")
+doc = read_rust_doc_lines(template.doc_path)
+
+output = sys.stdout
+
+for line in template.header:
+    output.write(line)
+
+for line in doc:
+    if line.startswith(template.start):
+        output.write(line)
+        break
+
+for line in doc:
+    output.write(line)
+
+for line in template.footer:
+    output.write(line)

readme_template

@@ -0,0 +1,24 @@
+# rust-rdkafka
+
+[![crates.io](https://img.shields.io/crates/v/rdkafka.svg)](https://crates.io/crates/rdkafka)
+[![docs.rs](https://docs.rs/rdkafka/badge.svg)](https://docs.rs/rdkafka/)
+[![Build Status](https://travis-ci.org/fede1024/rust-rdkafka.svg?branch=master)](https://travis-ci.org/fede1024/rust-rdkafka)
+[![Join the chat at https://gitter.im/rust-rdkafka/Lobby](https://badges.gitter.im/rust-rdkafka/Lobby.svg)](https://gitter.im/rust-rdkafka/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
+
+Kafka client library for Rust based on [librdkafka].
+
+__INCLUDE_RUST_DOC__$src/lib.rs$## The library
+## Documentation
+
+Documentation is available on [docs.rs](https://docs.rs/rdkafka/).
+
+## Contributors
+
+Thanks to:
+* Thijs Cadier - [thijsc](https://github.com/thijsc)
+
+## Alternatives
+
+* [kafka-rust]: a pure Rust implementation of the Kafka client.
+
+[kafka-rust]: https://github.com/spicavigo/kafka-rust

src/lib.rs

@@ -2,24 +2,29 @@
 //! Kafka client library for Rust based on [librdkafka].
 //!
 //! ## The library
-//! This library aims to provide a safe interface to librdkafka.
-//! It currently exports some of the funcionalities provided by the producer and consumer
-//! of librdkafka 0.9.2.
+//! `rust-rdkafka` provides a safe Rust interface to librdkafka.
+//! It currently exports a subset of the funcionalities provided by librdkafka 0.9.2.
 //!
-//! Producers and consumers can be accessed and polled directly, or alternatively
-//! a [futures]-based interface can be used:
+//! `rust-rdkafka` provides low level and high level consumers and producers. Low level:
 //!
-//! * A consumer will return a [`stream`] of messages, as they are received from Kafka.
-//! * A producer will return a [`future`] that will eventually contain the delivery
-//! status of the message.
+//! * `BaseConsumer`: simple wrapper around the librdkafka consumer. It requires to be
+//!   periodically `poll()`ed in order to execute callbacks, rebalance and receive messages.
+//! * `BaseProducer`: simple wrapper around the librdkafka producer. As in the consumer case,
+//!   the user must call `poll()` periodically to execute delivery callbacks.
+//!
+//! High level:
+//!
+//!  * `StreamConsumer`: it returns a [`stream`] of messages and takes care of polling the consumer
+//!  internally.
+//!  * `FutureProducer`: it returns a [`future`] that will be completed once the message is
+//!  delivered to Kafka (or fails).
 //!
 //! [librdkafka]: https://github.com/edenhill/librdkafka
 //! [futures]: https://github.com/alexcrichton/futures-rs
 //! [`future`]: https://docs.rs/futures/0.1.3/futures/trait.Future.html
 //! [`stream`]: https://docs.rs/futures/0.1.3/futures/stream/trait.Stream.html
 //!
-//! *Warning*: this library is still at an early development stage, the API is very likely
-//! to change and it shouldn't be considered production ready.
+//! *Warning*: the library is under active development and the APIs are likely to change.
 //!
 //! ## Installation
 //!
@@ -30,7 +35,7 @@
 //! rdkafka = "^0.2.0"
 //! ```
 //!
-//! This crate will compile librdkafka from sources and link it statically in your
+//! This crate will compile librdkafka from sources and link it statically to your
 //! executable. To compile librdkafka you'll need:
 //!
 //! * the GNU toolchain