Apache Kafka
Apache Kafka is available as a project service for durable event streaming. SailOps runs Kafka 4 in KRaft mode — no ZooKeeper — with one or three nodes that each act as broker and controller. Trial organizations cannot provision it.
Create a cluster
Open Services → Create Service → Apache Kafka. The form has two sections:
-
Basic Configuration — cluster name, Kafka version and number of nodes.
- 1 node is meant for development: every topic has a single replica, so a node restart interrupts traffic.
- 3 nodes run on three different workers. Topics default to three replicas and writes require two in-sync replicas, so the cluster keeps serving while one node restarts.
The number of nodes is fixed at creation.
-
Resources — compute class (
eco,standardorproduction), flavor (vCPU and memory per node, at least 2 GiB), storage size per node (at least 10 GiB) and storage class (standardorperformance). The storage class is fixed at creation. The compute class can be changed later; the flavor and storage size can only grow. Changes are applied by restarting the nodes one at a time.
Connect an application
The cluster is private to its project. Clients bootstrap from <cluster-name>-kafka:9092 and authenticate with SASL/SCRAM:
bootstrap.servers=<cluster-name>-kafka:9092
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="<username>" password="<password>";
Anonymous clients are rejected. Traffic stays inside the project network, which is isolated from other projects; public access is not available.
The simplest way to connect a container or a job is a connection: select Apache Kafka as the target and choose an access level. SailOps creates a dedicated user for the connection and injects KAFKA_BOOTSTRAP_SERVERS, KAFKA_SECURITY_PROTOCOL, KAFKA_SASL_MECHANISM, KAFKA_USERNAME and KAFKA_PASSWORD. Deleting the connection deletes its user.
Use a client that supports SCRAM-SHA-512, such as the Java client, franz-go, KafkaJS or librdkafka (confluent-kafka, kcat). A new user or password can take a few seconds to be accepted by every broker.
Users
The Users tab manages SCRAM users for clients that do not use a connection. Each user has an access level that applies to every topic and consumer group of the cluster:
| Access level | Topics | Consumer groups |
|---|---|---|
readonly | read, describe | read, describe |
readwrite | read, write, create, describe, describe configs | read, describe |
full | all operations | all operations |
Clients authenticate with the user's SASL username, shown in the tab next to the name you chose — it is derived from that name and the cluster (kf-…), not the name itself. The password is shown once, when the user is created or its password is regenerated. Store it immediately.
Topics
The Topics tab declares topics: name, partitions, replication factor, retention and cleanup policy (delete or compact). Retention defaults to seven days and the cleanup policy to delete. Partitions can only grow; the replication factor cannot exceed the number of nodes and is fixed at creation. Names starting with __ are reserved for Kafka, and a name that differs from an existing topic only by . versus _ is refused, as Kafka does.
A topic with fewer replicas than three keeps accepting writes with acks=all while one broker is down: SailOps sets its minimum in-sync replicas to one less than its replication factor.
Automatic topic creation on first produce is disabled. A user with the full access level can still create topics through the Kafka Admin API — Kafka Streams internal topics, for example — but those topics are not listed in the tab.
Kafka UI
Kafka UI is an optional web interface for the cluster, based on the open-source kafbat/kafka-ui project. It browses brokers, topics, messages and consumer groups, and connects to the cluster on its own: there is nothing to configure inside it.
To enable it, edit the cluster (or continue through creation), turn on Kafka UI and save. Two settings apply:
- Read-only, on by default. Kafka UI can then browse topics, messages and consumer groups but not produce messages or change topics and offsets — the Kafka user it connects with is denied those operations too. Turn it off for write actions permitted by that user's Kafka ACLs.
- Allowed CIDR Blocks restricts which client addresses can reach the interface. Left empty, any address reaches the login page.
Once it is ready, the cluster overview shows a Kafka UI card with the HTTPS link, served on the platform's shared gateway with a random suffix (for example events-kafka-ui-k3x9q2ab.gw.sailops.dev). Opening the link prompts for a username and password, which accounts with the Developer role or above can reveal, copy and regenerate from the card. Regenerating them signs out open sessions; Kafka users and their passwords are not affected.
Kafka UI runs as one more pod of the cluster and is billed for its CPU and memory at the cluster's compute class. It stops while the project is suspended and is deleted with the cluster; turning it off deletes it.
Kafka UI is unavailable on installations without a shared gateway. If your installation doesn't have one, the toggle is disabled.
Monitoring and costs
The overview shows the cluster status, its bootstrap endpoint and its CPU, memory and storage usage. Statuses are Running, Updating (a rolling change is in progress), Pending (provisioning), Warning, Stopped (project suspended) and Deleting (the cluster is being deleted). The Observability tab charts historical usage, the Logs tab shows broker logs and the Alerts tab lists the cluster's alerts.
A cluster is billed for each node according to its flavor, plus its provisioned storage and, when enabled, Kafka UI. Storage remains billable while a project is suspended.
Kafka clusters have no snapshots. Three replicas protect against the loss of a node, not against a deleted topic or a faulty producer. Keep an external copy of any data you cannot replay.