Managing Access Control

1. Understanding ACL Model

Component	Description	Notes
Principal	Identity that can perform operations (user, group, service account)	Can be defined in SASL/SCRAM or LDAP
Resource	Object that access is controlled over (topic, group, cluster)	Resource types are defined in Kafka
Operation	Action that can be performed (Read, Write, Describe, etc.)	Operations vary by resource type
Permission	Allow or Deny access to a resource for a principal	Permissions can be inherited or explicitly set

Example: ACL Configuration

kafka-acls.sh --bootstrap-server localhost:9092 \
  --add --allow-principal User:alice \
  --operation Read --resource-type Topic --resource-name my-topic

Note: ACLs are enforced at the broker level and apply to all clients connecting to that broker.

Key ACL Concepts

Principal identifies the user or service
Resource is the target object (topic, group, etc.)
Operation defines what actions are allowed
Permission determines if access is granted or denied

2. Enabling Authorization

Example: Enable ACL authorization in Kafka

# Set the following properties in server configuration
authorizer.class.name=kafka.security.authorizer.AclAuthorizer
sasl.enabled.mechanisms=PLAIN
security.protocol=SASL_PLAINTEXT

Property	Values	Description
authorizer.class.name	`kafka.security.authorizer.AclAuthorizer`	Enables ACL-based authorization
sasl.enabled.mechanisms	`PLAIN`	Enables SASL/PLAIN authentication
security.protocol	`SASL_PLAINTEXT`	Uses SASL over plaintext for communication

Note: ACLs are enforced at the broker level and require SASL authentication to be enabled.

Warning: Authorization is disabled by default. Ensure it's enabled in production environments.

3. Creating ACLs

Resource Type	Name	Operation	Permission
Topic	`orders`	`READ`	`ALLOW`
Group	`consumer-group-1`	`READ`	`ALLOW`
Cluster	`kafka-cluster`	`DESCRIBE_CONFIGS`	`ALLOW`

Example: Create ACL for Topic Read Access

kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 \
  --add --allow-principal User:alice \
  --operation READ --topic orders

Example: Create ACL for Consumer Group Access

kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 \
  --add --allow-principal Group:consumer-group-1 \
  --operation READ --group consumer-group-1

Note: ACLs are evaluated in order, and the first matching rule determines the outcome. Ensure that more specific rules come before general ones.

4. Listing ACLs

Command	Description	Options	Example
kafka-acls.sh	List ACLs in Kafka	`--bootstrap-server`, `--list`, `--topic`, `--user`	`kafka-acls.sh --bootstrap-server localhost:9092 --list`

Example: Listing ACLs for a Specific Topic

kafka-acls.sh --bootstrap-server localhost:9092 --list --topic my-topic

Example: Listing ACLs for a Specific User

kafka-acls.sh --bootstrap-server localhost:9092 --list --user alice

5. Removing ACLs

ACLs in Kafka can be removed using the kafka-acls.sh script. The removal process requires specifying the correct principal and resource details to ensure that only the intended ACLs are deleted.

Parameter	Description	Example Value
--principal	Principal for which the ACL is to be removed	`User:ANONYMOUS`
--topic	Topic name for which the ACL is to be removed	`my-topic`
--operation	Operation type to remove (e.g., `READ`, `WRITE`)	`READ`
--cluster	Indicates that the ACL applies to the cluster	`true`

Example: Remove ACL for a Topic

kafka-acls.sh --bootstrap-server localhost:9092 \
  --remove \
  --principal User:ANONYMOUS \
  --topic my-topic \
  --operation READ

Note: Ensure that the user running the command has sufficient permissions to remove ACLs in Kafka.

Warning: Removing an ACL can affect access to topics or resources. Always verify the impact before executing removal commands.

6. Setting Principal Format

Property	Values	Description
sasl.kerberos.principal.format	`USER@REALM`, `USER`	Defines the format of the Kerberos principal name. Default is `USER@REALM`.
sasl.plain.username	`string`	Username for PLAIN SASL mechanism. Used in principal format when `sasl.kerberos.principal.format` is set to `USER`.
sasl.kerberos.principal.to.local.rules	`string`	Rules to map Kerberos principals to local user names. Used in principal format when `sasl.kerberos.principal.format` is set to `USER@REALM`.

Example: Configuring Principal Format

{<br>  "sasl.kerberos.principal.format": "USER@REALM",<br>  "sasl.plain.username": "admin"<br>}

Note: The sasl.kerberos.principal.format setting affects how Kafka interprets and validates Kerberos principals. Ensure that the format matches your Kerberos server configuration.

7. Configuring Super Users

Example: Define super users in Kafka

# kafka-server-start.sh config/server.properties

# Define super users
super.users=User:admin;User:monitoring

Property	Values	Description
super.users	`User:admin;User:monitoring`	Comma-separated list of super users in the format `User:<principal>`

Note: Super users bypass ACL checks and have full access to all resources.

Warning: Super users should be used sparingly and only for administrative tasks.

8. Understanding ACL Operations

Operation	Description	Permissions Required	Notes
Allow	Grants permission to perform an action	Admin or ACL management rights	Used to grant access to topics, groups, clusters, etc.
Deny	Explicitly blocks a permission	Admin or ACL management rights	Can override allow rules in some cases
Describe	Reads metadata about a resource	Read or Describe permissions	Used for checking topic existence, consumer group status, etc.
Read	Allows reading messages from a topic	Read permission on topic	Required for consuming messages from a topic
Write	Allows producing messages to a topic	Write permission on topic	Required for publishing messages to a topic
Delete	Deletes a topic or consumer group	Delete permission on resource	Used for removing topics or consumer groups
Alter	Modifies configuration of a resource	Alter permission on resource	Allows changing topic or consumer group settings
DescribeConfigs	Reads configuration of a resource	DescribeConfigs permission on resource	Used to inspect topic or consumer group configurations

Example: Creating an ACL to allow reading from a topic

kafka-acls.sh --bootstrap-server localhost:9092 \
  --add \
  --allow-principal User:alice \
  --operation Read \
  --topic my-topic

Note: ACLs are evaluated in order, with the most specific matching rule taking precedence.

Warning: Misconfigured ACLs can lead to unintended access or denial of access.

9. Setting Resource Patterns

Resource Type	Pattern Type	Description	Example
Topic	`LITERAL`	Matches a specific topic name exactly	`my-topic`
Topic	`PREFIXED`	Matches topics with a common prefix	`user-*`
Topic	`ANY`	Matches any topic	`*`
Group	`LITERAL`	Matches a specific consumer group ID exactly	`my-group`
Group	`PREFIXED`	Matches consumer groups with a common prefix	`app-*`
Group	`ANY`	Matches any consumer group	`*`
Cluster	`LITERAL`	Matches the cluster resource	`kafka-cluster`
Cluster	`ANY`	Matches any cluster resource	`*`
Transactional ID	`LITERAL`	Matches a specific transactional ID exactly	`tx-id-123`
Transactional ID	`PREFIXED`	Matches transactional IDs with a common prefix	`tx-*`
Transactional ID	`ANY`	Matches any transactional ID	`*`

Example: Configure Resource Pattern for Topics

kafka-acls.sh --bootstrap-server localhost:9092 \
  --add \
  --allow-principal User:alice \
  --operation Read \
  --topic user-*

Example: Configure Resource Pattern for Consumer Groups

kafka-acls.sh --bootstrap-server localhost:9092 \
  --add \
  --allow-principal User:alice \
  --operation Read \
  --group app-*

Note: Resource patterns are used in conjunction with ACLs to define access control rules for Kafka resources.

10. Using Wildcard ACLs

Wildcard ACLs in Kafka allow you to define access control rules using wildcards for resource names, which can simplify the management of permissions across many similar resources.

Resource Type	Wildcard Pattern	Description
Topic	`topic-*`	Matches all topics starting with "topic-"
Group	`group-*`	Matches all consumer groups starting with "group-"
Cluster	`*`	Matches all clusters (use with caution)

Example: Define ACLs with Wildcards

kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 \
  --add --allow-principal User:* \
  --operation Read --resource-type Topic --resource-name topic-* \
  --host localhost

Note: Wildcard patterns are supported in resource names, but not in principal names or operation types.

11. Configuring ACL Binding

Property	Description	Default Value	Notes
kafka.acl.binding.type	Defines the type of ACL binding to configure	`simple`	Valid values: `simple`, `complex`
kafka.acl.binding.resource.type	Resource type for which the ACL is defined	`topic`	Valid values: `topic`, `group`, `cluster`, `transactional.id`
kafka.acl.binding.resource.name	Name of the resource to bind ACL to	`null`	Required for `topic`, `group`, and `transactional.id`
kafka.acl.binding.operation	Operation that is allowed or denied	`read`	Valid values: `read`, `write`, `delete`, `describe`, `alter`
kafka.acl.binding.principal	Principal (user or group) to which the ACL applies	`null`	Required field, must follow the format `PrincipalType:PrincipalName`
kafka.acl.binding.host	Host from which the principal can access the resource	`*`	Can be a specific IP or `*` for any host

Example: Configuring ACL Binding for a Topic

{
  "kafka.acl.binding.type": "simple",
  "kafka.acl.binding.resource.type": "topic",
  "kafka.acl.binding.resource.name": "user-events",
  "kafka.acl.binding.operation": "read",
  "kafka.acl.binding.principal": "User:alice",
  "kafka.acl.binding.host": "*"
}

Example: Configuring ACL Binding for a Consumer Group

{
  "kafka.acl.binding.type": "simple",
  "kafka.acl.binding.resource.type": "group",
  "kafka.acl.binding.resource.name": "analytics-group",
  "kafka.acl.binding.operation": "read",
  "kafka.acl.binding.principal": "User:analytics-service",
  "kafka.acl.binding.host": "192.168.1.10"
}

Note: ACLs are evaluated in order, and the first matching rule applies. Ensure that more specific rules come before general ones.