Create AdminClient

[jeffwidman]

This is to track creating an AdminClient to mirror the Java AdminClient mentioned in KIP-4

What are your plans on this front?

Are you waiting til the Java crew releases their AdminClient (as mentioned in KIP-4)? Or ready to start this sooner and then cleanup the implementation to spec once they get to it?

I ask because we could use this relatively soon, right now we have a python script calling subprocess to call shell commands and it's pretty janky. But I suspect the Java AdminClient won't actually be released for a bit yet...

[tvoinarovskyi]

Just to remind, there was a ticket #857 for KIP-4 and Create/Delete API.

[jeffwidman]

Oh, I'd missed that, thanks!

[jeffwidman]

Reopening to track the AdminClient creation per #857 (comment)

As part of this, it'd be nice if there was a simple API for returning topic metadata (without having to consume those topics) such as broker offsets, consumer group offsets, and probably the calculated consumer lag.

[tvoinarovskyi]

What about the actual interface for AdminClient? Should we use something like Scala's AdminClient as a reference?

[jeffwidman]

I haven't looked at the Scala admin client, but there has been a fair bit of chatter recently on the Kafka dev list re: what the new Java AdminClient interface should look like. I think, but haven't confirmed that the Java one and the Scala one are different.

[tvoinarovskyi]

By the way, seems like apache/kafka#2472 was merged into Java Client. It's basically what the ticket is about, right?

[jeffwidman]

Kafka 0.11 is live, including the new Java AdminClient. Now just need to port it to kafka-python

[pvanderlinden]

I was wondering if someone has made a start on this? I started looking at it myself, but I'm not that familiar with the kafka-python code yet, to start it decently from scratch.

For people who are interrested in just creating topics, made a quick and dirty script so far: https://gist.github.com/pvanderlinden/3c9b8fc58e07afe2a205d906ba86d9d0
It is missing a lot of error handling though.