Update - As I started playing around with Kafka Manager it became apparent to me that this tool was only intended to be used as a UI for administration tasks, not for monitoring health (such as lag) of existing production topics. So soon enough I will also be looking at web-console.
Previously we had built our own monitoring solution with bastardized angular and bootstrap, but it was painfully slow and took many developer hours to maintain it (the slowness could be down to the JMX reporters and thousands of topics).
Step 1 - Download and extract the package
If you followed my previous post, I set up a node for zookeeper and I'm going to install Kafka manager on here also. So, you will need to first SSH onto the Zookeeper Node and download the package from GitHub, unzip and then rename the dir.
If you're like me and you don't have scala installed you will need to grab it (it doesn't seem to be as easy as "apt-get install sbt"). Below are the commands to run to install scala manually.
Cd into your new kafka-manager directory then run the build command.
Step 4 - Copy that Zip file to a suitable location and unzip
Yes you would expect a conf file to point the manager to a Zookeeper host(s) of which there is a such a file, but for me at least it didn't work, I tried the dns, the ip and since I'm running this app on the same host as the Zookeeper node I also tried localhost...none of them worked. In case you have better luck than me below are the entries that need to be made:
The conf file is located at: "conf/application.conf" and alter the zkhosts entry
Below is the command I ran to over-ride the conf and it worked just fine.
Go to Cluster -> Setup a new cluster.
You can now see it's picked up my cluster and the existing 2 topics I created. If you have an existing topic in your cluster, you can click on it and explore from there.
In the top menu, click on topic and then create. You are presented with a bunch of parameters that you might be accustomed to from using the stand CLI. Once done there click on topic view.
We created our topic, now we can see it in all it's glory! If you're old hat to Kafka... you'll be pleased to know that the topic "delete" function.. works!.
Clicking on the broker menu item, provides you with a summary of your brokers.
This bit was a little buggy for me, I created a test4 topic and deleted it (out of this example) and it's still showing up as an option to reassign. When I selected test4 it then presented an error that it didn't exist.
This is a very very simple application, for some reason I assumed because it was made by Yahoo it would provide more features.
When you should use it:
I think it would be useful on an enterprise level where system engineers don't have to always go through the CLI to complete their work, where engineers complete 5,000 tasks a day and this is just one of them. It enables you to digest the information pretty quickly.
I guess I expected more? Maybe I'm being greedy? I would love to see some metrics of cluster health around topic consumer lag etc. I'm assuming that since Yahoo is large scale they use standardized health metrics systems across their applications and they have something already feeding into that. Who knows? Kafka is pretty young in the terms of things and we could see this project grow. (Also it screams bootstrap).. confirmed src it's bootstrapped.
Keep posted as I dive into web-console.