Do you know what the 'main cluster' and 'max cluster' properties refer to in Kafka, and where they are configured?

In Kafka, the term 'main cluster' usually refers to the primary cluster configuration that a client connects to, defined by the bootstrap.servers property. This property lists the broker addresses that the client uses to discover the cluster. The 'max cluster' property is not a standard Kafka broker setting; it is often a custom configuration used by higher‑level frameworks (e.g., Kafka Connect or Spark) to limit the number of consumer instances or partitions that can be assigned to a consumer group. In Kafka Connect, for example, you can set 'max.tasks' to control the maximum number of parallel tasks, which indirectly limits the cluster size. In Spark Structured Streaming, you can set 'spark.streaming.kafka.maxRatePerPartition' to cap the ingestion rate per partition. These properties are typically set in the client configuration file (consumer.properties or producer.properties) or passed programmatically when creating the consumer or producer. Understanding where each property lives—broker, client, or framework—is essential for tuning cluster behavior and ensuring reliable data ingestion.