kfkwho, an authorizer for Apache Kafka® that counts
Who talks
to what.
A drop-in replacement for the broker's StandardAuthorizer that publishes JMX
metrics per principal, client id, topic, operation and request type. This is how, in
fourteen slides and one small model.
Scroll, or use the arrow keys.
1 · the question
Who produces to this topic?
The producers of a topic are what you need to know before you change, move or delete it. Consumers come right after: which clients read this topic, and when did they last show up?
Admins are the same question about other operations.
2 · the catch
The broker knows, and never says
A broker knows, on every request, who sent it (principal, client id, address) and what it touches (topic, group, operation). It never puts the two together in a metric.
- Quota metrics are per client, with no topic.
BrokerTopicMetricsare per topic, with no client.- Client telemetry (KIP-714) needs new clients and a reporter plugin.
So the question is answered by grepping request logs.
3 · the idea
Count at the authorizer
The authorizer is the one hook that sees both sides of every request, for clients of any version: the principal and the client id, the resource and the operation. kfkwho extends the standard one. The verdict stays the broker's; the observation becomes a metric.
4 · the verdict
It observes. It does not decide.
MeteredStandardAuthorizer is StandardAuthorizer plus metrics.
authorize() calls the parent first and returns exactly what the parent returns.
ACLs, super.users and allow.everyone.if.no.acl.found work as
before.
- It extends the standard authorizer; it does not wrap it. The broker feeds
ACLs only to a
ClusterMetadataAuthorizer, and a plain wrapper would deny everything, silently. - Only brokers record. A dedicated controller passes the parent's verdict through and publishes nothing.
5 · a series
One counter per who and what
An access series is one MBean per principal, client id, resource, operation, request type and verdict. The tag order is fixed, so an exporter rule is one line.
kfkwho:type=access,principal=…,client-id=…,resource-type=TOPIC,resource=…,operation=…,api=…,result=ALLOWED
kfkwho:type=client,principal=…,client-id=…,listener=…,security-protocol=…
It is counted once per action the broker authorizes: a Produce to three topics is three.
The client series is one per principal and client id, counted once per
authorize call: who is connected, without the resource.
6 · reading it
Producers first
The api tag is the request type. It tells a client that really produced from
one that only fetched metadata about the topic.
| Client | Operation | Series |
|---|---|---|
| Producer | WRITE | TOPIC, WRITE/PRODUCE |
| Only fetched metadata | DESCRIBE | TOPIC, DESCRIBE/METADATA: it has not produced |
| Consumer | READ | TOPIC, READ/FETCH, plus GROUP, READ with OFFSET_COMMIT, JOIN_GROUP, HEARTBEAT |
| Admin | CREATE | CREATE/CREATE_TOPICS, counted the same way |
For a GROUP series the resource is the group id, which links client ids to
consumer groups. Denied requests are counted too, with result=DENIED.
7 · the hot path
One lookup per action
authorize() runs for every topic of every Fetch and Produce, so what the
metrics add to it is measured, not guessed. A series seen before costs one lookup per action
and allocates nothing.
On first sight of a key the request thread only leaves a pending series. A
background thread, kfkwho-series, creates it, at about 40 µs each;
requests that arrive meanwhile are counted into it once it exists.
Measured with JMH on a cloud VM with 4 vCPU: the parent alone took 338 ns per call and the metered authorizer 525 ns, 187 ns added; later runs, with the client series counted as well, added 213 to 332 ns. First sight adds about 1.1 µs on the request thread. No allocation per call. The budget is under 1 µs and under 3 µs. The runs are in Performance.
8 · bounded
Idle series expire. A cap folds the rest.
Clients come and go, and every new key is a series. Two bounds keep their number finite.
- TTL: a series without a request for
kfkwho.series.ttl.seconds(600) is removed. Seen again, it starts from zero, which Prometheus reads as a counter reset. - Cap: at most
kfkwho.max.series(10000) series of each type. Past it, a new key is recorded into that type's__other__series, and counted inaccess-series-overflow-total.
Series that already exist keep counting, and a series freed by the TTL makes room for the next new key. A non-zero overflow means some clients are not told apart.
9 · names
One application, one series
Consumer and Streams clients put a per-instance suffix in their client id; left as is, every restart is a new series. Client id rules rewrite the id before it becomes a tag.
| Client id, as sent | Tag |
|---|---|
consumer-orders-app-3-6f1c2a4e-1b2c-4d5e-8f90-abcdef123456 | consumer-orders-app |
consumer-orders-app-3, billing-1 | consumer-orders-app, billing |
The rules are kfkwho.client.id.rules, each pattern=>replacement
in Java regex syntax; the first whose pattern matches the whole id applies. Internal topics
(__.*) are not recorded at all: that is kfkwho.resource.exclude.
10 · try it
Start and stop clients
access-series-count 0 of 4
access-series-evicted-total 0
access-series-overflow-total 0
kfkwho.series.ttl.seconds 20
kfkwho.max.series 4
| principal | client-id | resource | operation / api | request-total | idle |
|---|---|---|---|---|---|
| no series | |||||
A model of the bounds, not the real code: one series per client, created at
once, every verdict ALLOWED. A real client leaves several series, and the
defaults are 600 s and 10000 series. Start billing-2: it counts into the series
of billing-1. Start everything: the fifth key goes to __other__.
Stop a client and count to twenty.
11 · out of the broker
JMX, then Prometheus
The MBeans are in the kfkwho domain, published through the broker's own metrics
library and a JmxReporter, the way quota metrics are. The repository ships
rules for the Prometheus JMX Exporter, jmx-exporter/kfkwho.yml. Each tag
becomes a label of the same name, with - written as _.
Who produces to orders, per principal and client id:
sum by (principal, client_id) (
rate(kfkwho_access_requests_total{resource_type="TOPIC", resource="orders",
operation="WRITE", api="PRODUCE", result="ALLOWED"}[5m]))
12 · limits
What it does not do
- It never changes a verdict, and it keeps no ACLs: they are in the cluster metadata.
- It does not leave an open cluster open. A cluster that ran without an
authorizer starts enforcing ACLs the moment it gets one. To observe without
enforcing, set
allow.everyone.if.no.acl.found=trueand put the principals the brokers and controllers use between themselves insuper.users. - It counts actions the broker authorizes. Bytes and records are not in these metrics.
- Past the cap it does not tell clients apart: who they are is lost for as long as the cap stays reached.
- The exporter rules are a first, minimal version: they export
request-totalof the access series and nothing else. A Grafana dashboard has not landed yet. - Nothing has run on a production cluster yet, and a setting or a metric may still be renamed; every such change is listed in the changelog.
- Apache Kafka 4.1 and later, KRaft, is the target. Earlier releases are not tested.
13 · why believe any of this
How it is tested
- Unit tests drive the real
StandardAuthorizerwithout a broker. A test exercises the class that ships, or it is deleted. - An integration test runs the built jar as the authorizer of an
apache/kafkabroker, with the JMX Exporter agent and the shipped rules, and checks the exporter's output. CI runs it on every supported broker release. - The docs are tested: a test fails when the configuration table, the metrics tables or the list of public types disagree with the code.
- Benchmarks measure the hot path; every change to
authorize()quotes them against the budget. - Every new metric dimension needs a cap, a TTL or an allowlist, and a test that proves the bound.
14 · go
Four things on every node
- the kfkwho jar on the broker's classpath;
authorizer.class.name=dev.kfkit.kfkwho.MeteredStandardAuthorizer;- the Prometheus JMX Exporter agent with the rules in
jmx-exporter/kfkwho.yml; - a rolling restart, one node at a time.
In the apache/kafka image, with the jar, the agent and the rules under
/opt/kfkwho, add to the environment the broker already has:
CLASSPATH=/opt/kfkwho/kfkwho.jar
KAFKA_AUTHORIZER_CLASS_NAME=dev.kfkit.kfkwho.MeteredStandardAuthorizer
KAFKA_SUPER_USERS=User:admin
KAFKA_OPTS=-javaagent:/opt/kfkwho/jmx_prometheus_javaagent.jar=9404:/opt/kfkwho/kfkwho.yml
The jar is on the releases page, with its SHA-256 and a cosign signature, and the same
jar is on Maven Central as dev.kfkit:kfkwho. A cluster that
ran without an authorizer starts enforcing ACLs: read Installation first.
- Source on GitHub
- Installation: the image, a tarball, Strimzi
- Configuration
- Metrics: MBeans, tags, Prometheus series
- Embedding: the same metrics from another authorizer
- Releases, Maven Central
- kfkit
With Claude Code, the repository carries a skill for adopting kfkwho: what to find out first, the order of the steps, what not to do.
/plugin marketplace add kfkit/kfkwho
/plugin install kfkwho@kfkwho