0 votes
17 views
ago by (30.6k points)
We have basic CPU/memory monitoring but incidents still surprise us (Event Manager backlog, stuck timers, database growth). What should a BAW monitoring setup watch, and how do we get the numbers on WAS and on CP4BA?

1 Answer

0 votes
ago by (30.6k points)

Monitor three layers: JVM / platform, the BAW engine, and the business flow. What matters and where to get it:

SignalWhyWASCP4BA
Heap after GC, GC pauseleaks, undersized JVMPMI JVM runtime / verbose GCLiberty monitor feature metrics scraped by Prometheus (jvm_*)
Thread pool usagesaturationPMI WebContainer poolthreadpool_* metrics
JDBC pool wait time / free connectionsthe classic bottleneckPMI JDBC providerconnectionpool_* metrics
Event Manager backlog (due tasks not executed)timers / UCAs lateProcess Admin Event Manager monitorsame tables, Operations REST
Failed instances / tasks in errorintegration outagesREST search (instanceStatus=Failed)Operations REST / BAI
BPMDB size and growthretentiondatabase monitoringdatabase monitoring
Response time of task list / coachuser experiencesynthetic probesynthetic probe
# CP4BA: Workflow pods expose Liberty metrics; a ServiceMonitor picks them up (OpenShift user workload monitoring must be enabled)
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata: { name: baw-metrics, namespace: cp4ba }
spec:
  selector: { matchLabels: { app.kubernetes.io/name: bawfin } }   # label of your Workflow service
  endpoints: [ { port: https, scheme: https, path: /metrics, tlsConfig: { insecureSkipVerify: true }, basicAuth: { username: {name: baw-metrics-auth, key: user}, password: {name: baw-metrics-auth, key: password} } } ]
# example alert (PromQL) - heap after GC above 85 % for 10 minutes
- alert: BawHeapHigh
  expr: base_memory_usedHeap_bytes / base_memory_maxHeap_bytes > 0.85
  for: 10m
# release-independent engine probes through the REST API (a monitoring user with read access)
PUT /rest/bpm/wle/v1/search/query?organization=byInstance&run=true&size=1&condition=instanceStatus%7CFailed     -> data.totalCount = failed instances
GET /rest/bpm/wle/v1/system/details                                                                       -> version, build, node - a cheap liveness probe

Alerts that matter (in order of how often they precede an incident): JDBC pool wait time above zero for more than a minute; Event Manager backlog growing (timers late); failed instance count rising; heap above 85 % after GC; BPMDB growing faster than the retention job deletes; certificate expiry of Server definitions (external calls start failing at midnight); and the synthetic task-list probe slower than 3 s. On CP4BA add the operator's readiness of the CR (oc get icp4acluster -o jsonpath of the conditions) and pod restarts. Business monitoring (cycle times, SLA breaches) belongs in BAI / the PDW dashboards, not in the infrastructure alerts.

References

Related questions

723 questions

807 answers

98 comments

4.8k users

Join BPM Community Discord Channel

Welcome to BPM Tips Q&A, Community wiki/forum where you can ask questions and receive answers from other IBM BPM experts and members of the community. Users with 2000 points will automatically be promoted to expert level.
Created by Dosvak LLC
Our Youtube Channel
...