How do I start a BAW process from a Kafka topic or publish process events to Kafka?

Question

Our enterprise integration moved from MQ to Kafka. Can BAW consume Kafka messages to start processes or trigger UCAs, and can it publish events to Kafka without custom code?

Answer 1

Since BAW 21.0.x the workflow engine has Kafka integration in both directions, configured rather than coded:

• Inbound - an undercover agent can be triggered by messages of a Kafka topic (the inbound Kafka event integration in Process Designer / Studio): define the topic and the message format (JSON), map fields of the message to the UCA parameters; the engine's Kafka consumer (configured once per server: bootstrap servers, security, consumer group) delivers each message to the UCA, which starts a process or correlates to a waiting instance - exactly like JMS eventmsg but with plain JSON payloads.
• Outbound - the business event emitter publishes lifecycle and tracking events to Kafka (the BAI pipeline); for custom messages a service flow uses the outbound Kafka integration of your level, or a small Java integration with the Kafka client library where the built-in producer is missing.

# server configuration (traditional: 100Custom.xml <kafka> section; CP4BA: kafka settings in the workflow CR / shared configuration)
<kafka merge="mergeChildren">
  <bootstrap-servers merge="replace">kafka1.example.com:9093,kafka2.example.com:9093</bootstrap-servers>
  <security-protocol merge="replace">SASL_SSL</security-protocol>
  <sasl-mechanism merge="replace">SCRAM-SHA-512</sasl-mechanism>
  <sasl-jaas-config merge="replace">org.apache.kafka.common.security.scram.ScramLoginModule required username="baw" password="...";</sasl-jaas-config>
  <consumer-group merge="replace">baw-orders</consumer-group>
</kafka>
# message on topic "orders.created" -> UCA "OrderCreated" (JSON mapping: $.orderId -> orderId, $.customer.id -> customerId)
{ "orderId": "ORD-42", "customer": { "id": "c1" }, "amount": 1250 }

// producer from a service flow when the built-in producer is not available (Java integration with kafka-clients)
Properties p = new Properties(); p.put("bootstrap.servers", servers); p.put("key.serializer", StringSerializer.class.getName()); p.put("value.serializer", StringSerializer.class.getName());
try (KafkaProducer<String,String> producer = new KafkaProducer<>(p)) { producer.send(new ProducerRecord<>("orders.approved", orderId, json)).get(); }

Design notes: one consumer group per BAW environment (test and production must not share); idempotent UCAs (Kafka delivers at least once - de-duplicate on a business key); keep the message small and reference documents by id; use a schema (JSON schema / Avro via a gateway) so that field mapping stays stable; on CP4BA the Kafka used by BAI can double as the enterprise Kafka or you point the integration at your own cluster. Older releases (20.x and below) have no Kafka support - bridge with App Connect / a Kafka-to-REST connector that calls sendMessage.

References

• BAW documentation - Kafka event integration (inbound events, event emitter)
• CP4BA - Kafka configuration for Business Automation Workflow