A practitioner's reference for rpk, Redpanda's single-binary CLI, with copy-paste commands for topics, produce/consume, consumer groups, ACLs, and a side-by-side mapping to the Apache Kafka shell scripts.
If you have spent any time operating Apache Kafka, you know the drill: kafka-topics.sh lives in one place, kafka-console-consumer.sh in another, and half your muscle memory is just remembering which shell script wraps which broker API. Redpanda collapses all of that into one tool. This post is a working reference for that tool, with each command mapped back to the Kafka script it replaces so you can translate what you already know.
rpk (Redpanda Keeper) is a single-binary command-line client for managing Kafka-API-compatible clusters. It handles topics, produce and consume operations, consumer groups, ACLs, schema registry, and cluster administration through one executable, instead of the dozen-plus separate kafka-*.sh scripts that ship with Apache Kafka. Because it speaks the standard Kafka protocol, the data-plane commands (rpk topic, rpk group, rpk acl) work against any Kafka-compatible broker, not only Redpanda. The cluster-tuning and Redpanda-specific admin features are the parts that assume a Redpanda backend.
If you are still weighing streaming platforms before you commit to a CLI, our comparison of Kinesis and Kafka covers the broader trade-offs.
Connecting: profiles instead of --bootstrap-server everywhere
Every Kafka shell script makes you repeat --bootstrap-server host:9092 on every invocation. rpk replaces that with a profile: a named, saved connection that bundles brokers, TLS, and SASL credentials. You set it once and forget it.
# Create a profile that points at a self-managed cluster
rpk profile create prod \
--set kafka_api.brokers=broker1:9092,broker2:9092 \
--set tls.enabled=true
# Create a profile straight from a Redpanda Cloud cluster (interactive picker)
rpk profile create cloud-prod --from-cloud=prompt
# List, switch, inspect
rpk profile list
rpk profile use prod
rpk profile print
After rpk profile use prod, every later command targets that cluster with no connection flags. For a one-off override you can still pass -X brokers=... or set RPK_PROFILE. There is no Apache Kafka equivalent here; with the shell scripts you either retype the bootstrap string or maintain a client.properties file and pass --command-config to each script.
Cluster operations
Day-one questions on any cluster: is it healthy, who are the brokers, and what is the live configuration. rpk answers all three.
# Is the cluster healthy? (leaderless/under-replicated partitions, nodes down)
rpk cluster health
# Broker list, controller, cluster ID
rpk cluster info
# View and change cluster-wide config (Redpanda-specific tuning)
rpk cluster config get log_segment_size
rpk cluster config set log_segment_size 134217728
rpk cluster health and rpk cluster config have no clean one-to-one Kafka script. On Apache Kafka you reach for kafka-broker-api-versions.sh, kafka-metadata-quorum.sh, or kafka-configs.sh --entity-type brokers to assemble the same picture. This is one of the places rpk genuinely consolidates several tools into one.
| Task | rpk | Apache Kafka |
|---|---|---|
| Cluster health / status | rpk cluster health |
kafka-metadata-quorum.sh ... describe (partial) |
| List brokers | rpk cluster info |
kafka-broker-api-versions.sh --bootstrap-server ... |
| Get/set cluster config | rpk cluster config get/set |
kafka-configs.sh --entity-type brokers --alter |
Topic operations
This is where most operators live. The rpk topic command creates, lists, describes, alters, and deletes topics, and it accepts repeatable -c key=value flags for topic configs at creation time.
# Create: 6 partitions, replication factor 3, compacted
rpk topic create orders -p 6 -r 3 -c cleanup.policy=compact
# Create several topics in one shot
rpk topic create orders payments shipments -p 6 -r 3
# List and describe
rpk topic list
rpk topic describe orders
# Change a config after creation (incremental alter)
rpk topic alter-config orders --set retention.ms=604800000
# Add partitions (cannot be undone)
rpk topic add-partitions orders --num 6
# Delete
rpk topic delete orders
The flag shapes line up closely with the Kafka script you already know, which makes translation almost mechanical.
| Task | rpk | Apache Kafka |
|---|---|---|
| Create topic | rpk topic create t -p 6 -r 3 |
kafka-topics.sh --create --topic t --partitions 6 --replication-factor 3 |
| List topics | rpk topic list |
kafka-topics.sh --list |
| Describe topic | rpk topic describe t |
kafka-topics.sh --describe --topic t |
| Alter topic config | rpk topic alter-config t --set k=v |
kafka-configs.sh --entity-type topics --entity-name t --alter --add-config k=v |
| Add partitions | rpk topic add-partitions t --num 6 |
kafka-topics.sh --alter --topic t --partitions 12 |
| Delete topic | rpk topic delete t |
kafka-topics.sh --delete --topic t |
One translation trap worth flagging: rpk topic add-partitions --num 6 adds six partitions, whereas kafka-topics.sh --alter --partitions 12 sets the new total. Same outcome, different mental model. If you want a refresher on what a topic actually is under the hood, see what is a Kafka topic.
Producing and consuming
rpk topic produce and rpk topic consume cover quick data-plane work without a separate console-producer and console-consumer binary. The consume command defaults to JSON output and supports rich format strings, offsets like start, end, +2, or -3, and consumer-group mode.
# Produce: type records on stdin; key:value with -k separator
echo "order-42:created" | rpk topic produce orders -k order-42
# Consume the 10 newest-from-start records, custom format, then exit
rpk topic consume orders -o start -n 10 -f '%k => %v\n'
# Consume as part of a group (commits offsets like a real consumer)
rpk topic consume orders -g audit-group
# Consume from specific partitions only
rpk topic consume orders -p 0,1 -o end
For deep, ad hoc debugging (filtering, time-based seeks, raw byte inspection across a fleet of partitions) many operators still keep kcat in the toolbox. rpk covers the common path; kcat covers the awkward edge cases. Both are fine to use against a Redpanda cluster.
| Task | rpk | Apache Kafka | kcat |
|---|---|---|---|
| Produce | rpk topic produce t |
kafka-console-producer.sh --topic t |
kcat -P -t t |
| Consume from start | rpk topic consume t -o start |
kafka-console-consumer.sh --topic t --from-beginning |
kcat -C -t t -o beginning |
| Consume N then exit | rpk topic consume t -n 10 |
kafka-console-consumer.sh --topic t --max-messages 10 |
kcat -C -t t -c 10 |
| Consume in a group | rpk topic consume t -g g |
kafka-console-consumer.sh --topic t --group g |
kcat -C -t t -G g t |
Consumer groups and offset management
Lag investigation and offset surgery are the operations you reach for when something is stuck. rpk group describe shows members and per-partition lag; rpk group seek is the rewind/fast-forward tool. The seek syntax is more expressive than the Kafka script: you can seek to a timestamp, to start/end, or even to the committed offsets of a different group.
# List groups and inspect lag
rpk group list
rpk group describe audit-group
# Reset to the beginning of the log (only when the group is idle)
rpk group seek audit-group --to start
# Reset to a Unix timestamp (seconds, ms, or ns)
rpk group seek audit-group --to 1622505600 --topics orders
# Copy another group's committed offsets onto this one
rpk group seek audit-group --to-group replay-group --topics orders
# Remove a group entirely
rpk group delete audit-group
As with Kafka's --reset-offsets, a seek is rejected while any group member is active, so stop your consumers first. The dry-run habit from Kafka carries over conceptually: inspect with rpk group describe before you seek.
| Task | rpk | Apache Kafka |
|---|---|---|
| List groups | rpk group list |
kafka-consumer-groups.sh --list |
| Describe lag | rpk group describe g |
kafka-consumer-groups.sh --describe --group g |
| Reset to earliest | rpk group seek g --to start |
kafka-consumer-groups.sh --group g --reset-offsets --to-earliest --execute |
| Reset to timestamp | rpk group seek g --to <unix> |
kafka-consumer-groups.sh --group g --reset-offsets --to-datetime ... --execute |
| Delete group | rpk group delete g |
kafka-consumer-groups.sh --delete --group g |
ACLs and security
Authorization commands now live under rpk security acl (the older rpk acl still works as an alias). The model maps directly to Kafka ACLs: a principal, an operation, and a resource (topic, group, or transactional ID). A typical producer/consumer needs three grants.
# Producer + consumer access to a topic
rpk security acl create --allow-principal User:app \
--operation write,read,describe --topic orders
# The consumer group it commits to
rpk security acl create --allow-principal User:app \
--operation describe,read --group audit-group
# Transactional producer (exactly-once)
rpk security acl create --allow-principal User:app \
--operation describe,write --transactional-id app-txn
# Review what is granted
rpk security acl list --allow-principal User:app
| Task | rpk | Apache Kafka |
|---|---|---|
| Grant topic ACL | rpk security acl create --allow-principal User:app --operation read,write --topic t |
kafka-acls.sh --add --allow-principal User:app --operation Read --operation Write --topic t |
| List ACLs | rpk security acl list |
kafka-acls.sh --list |
| Create SASL user | rpk security user create app -p '<pass>' |
kafka-configs.sh --alter --add-config 'SCRAM-SHA-256=...' --entity-type users --entity-name app |
When you operate across clusters (replication, DR, migration), the same ACL and offset concepts apply, but the failure modes change. Our writeup on MirrorMaker 2 deployment gotchas covers what breaks when offsets and topic configs cross a cluster boundary.
When to reach for rpk, and when not to
rpk is the obvious default on a Redpanda cluster: one binary, profile-based connections, and consolidated cluster tuning. Against a plain Apache Kafka cluster, the data-plane commands work fine because they ride the standard protocol, but the cluster-tuning and self-test features (rpk redpanda tune, rpk cluster config) assume a Redpanda backend and will not apply. For low-level byte-and-timestamp debugging, kcat is still the sharper instrument. And on multi-broker Apache Kafka operations where your runbooks, automation, and team knowledge are already built around kafka-*.sh, switching the operational tooling buys you little.
Key takeaways:
- rpk is one binary that replaces the entire
kafka-*.shscript collection for topics, produce/consume, groups, ACLs, and admin. - Profiles remove the
--bootstrap-server/--command-configrepetition from every command. - Data-plane commands are protocol-portable and run against any Kafka-compatible broker; cluster-tuning commands are Redpanda-only.
rpk group seekis more expressive than Kafka's--reset-offsets, including seek-to-another-group.- ACL commands moved under
rpk security acl, mapping cleanly tokafka-acls.shsemantics. - Keep
kcatfor the awkward debugging that rpk does not cover.
If you are designing the pipeline these topics feed into, our Kafka, Flink, and ClickHouse blueprint shows where the CLI work ends and the streaming architecture begins. For deeper help with Redpanda or Kafka in production, talk to our streaming team.
