How do I call watsonx.ai (or another LLM API) directly from a BAW service flow with REST, for releases without the generative AI activity?

Question

We are on BAW 22 / 23 (traditional and containers) without the built-in generative AI task. Can we still use foundation models from a process, and what does a robust implementation look like (authentication, prompt, parsing, timeouts)?

Answer 1

Yes - the model is an HTTPS API. Two service flows do the job: one gets an IAM token from the API key (cached in a shared variable or an environment variable with an expiry), the other calls the text generation endpoint. On traditional BAW use the REST integration step or a script with HttpURLConnection; the script version below works on every release:

// 1. IAM token (service flow "watsonx token", cache result for ~50 min in tw.env / a shared BO)
var url = new Packages.java.net.URL("https://iam.cloud.ibm.com/identity/token");
var c = url.openConnection(); c.setRequestMethod("POST"); c.setDoOutput(true); c.setConnectTimeout(10000); c.setReadTimeout(20000);
c.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
var body = "grant_type=urn:ibm:params:oauth:grant-type:apikey&apikey=" + encodeURIComponent(tw.env.watsonxApiKey);
var os = c.getOutputStream(); os.write(new java.lang.String(body).getBytes("UTF-8")); os.close();
var text = String(new java.util.Scanner(c.getInputStream(), "UTF-8").useDelimiter("\\A").next());
tw.local.token = JSON.parse(text).access_token;

// 2. text generation (service flow "watsonx generate": inputs prompt, token; output text)
var url = new Packages.java.net.URL(tw.env.watsonxUrl + "/ml/v1/text/generation?version=2024-05-01");
var c = url.openConnection(); c.setRequestMethod("POST"); c.setDoOutput(true); c.setConnectTimeout(10000); c.setReadTimeout(60000);
c.setRequestProperty("Content-Type", "application/json"); c.setRequestProperty("Accept", "application/json");
c.setRequestProperty("Authorization", "Bearer " + tw.local.token);
var payload = JSON.stringify({
  model_id: tw.env.watsonxModel,                       // e.g. "ibm/granite-13b-instruct-v2" or a llama / mistral model available in the region
  project_id: tw.env.watsonxProject,
  input: tw.local.prompt,
  parameters: { decoding_method: "greedy", max_new_tokens: 300, stop_sequences: ["}"] }
});
var os = c.getOutputStream(); os.write(new java.lang.String(payload).getBytes("UTF-8")); os.close();
var status = c.getResponseCode();
var stream = status < 400 ? c.getInputStream() : c.getErrorStream();
var resp = String(new java.util.Scanner(stream, "UTF-8").useDelimiter("\\A").next());
if (status >= 400) throw new Error("watsonx " + status + ": " + resp.substring(0, 300));
tw.local.text = JSON.parse(resp).results[0].generated_text;

Making it robust: keep API key and project id in environment variables / secrets (never in the script); wrap the call in the retry pattern for 429 / 503 (question on retries); cap max_new_tokens; ask for JSON with a stop sequence and parse defensively; store the model id and a hash of the prompt in the instance for audit; and put the calls behind one reusable service flow ("Ask model": prompt in, text out) so that prompts live in EPVs or a toolkit and can change without redeploying every app. The same two steps work for any OpenAI-compatible endpoint by changing the URL, the auth header and the payload; on CP4BA with restricted egress, ask for the NetworkPolicy to the provider.

References

• watsonx.ai API reference - text generation, IAM token
• BAW documentation - REST integration and server scripts