Question

SSL handshake fails for new web services/REST services over https, how to create trust in websphere?

Answer 1

An SSL handshake failure for a new HTTPS endpoint almost always means the server's certificate chain is not in the cell truststore that WebSphere / BAW uses for outbound connections (CellDefaultTrustStore, or the node's NodeDefaultTrustStore in older cells). Fix it with the signer retrieval:

# WebSphere admin console
Security > SSL certificate and key management > Key stores and certificates > CellDefaultTrustStore > Signer certificates
  > Retrieve from port : host = api.partner.com, port = 443, alias = partner-api  -> Retrieve signer information -> OK -> Save
# or scripted (wsadmin Jython)
AdminTask.retrieveSignerFromPort('[-keyStoreName CellDefaultTrustStore -keyStoreScope (cell):Cell01 -host api.partner.com -port 443 -certificateAlias partner-api]')
AdminConfig.save()
# or the command-line tool of the profile
retrieveSigners.sh CellDefaultTrustStore ClientDefaultTrustStore -host api.partner.com -port 443

Then restart the members (or at least re-read the SSL config: Dynamically update the run time when SSL configuration changes occur must be enabled). Other causes, in the order to check:

• Protocol / cipher mismatch: the endpoint requires TLS 1.2/1.3 and the SSL configuration is on SSL_TLS - set Protocol = TLSv1.2 (or TLSv1.3 on WebSphere 8.5.5.18+) in SSL configurations > CellDefaultSSLSettings > Quality of protection.
• TLS defaults of the IBM JDK: callers that use HttpsURLConnection from scripts or Java integrations follow the JVM defaults - set the JVM custom property com.ibm.jsse2.overrideDefaultTLS=true so that they use TLS 1.2 as well.
• Client certificate required (mutual TLS): import your client cert into the CellDefaultKeyStore and point the outbound SSL config (dynamic outbound endpoint SSL configuration for that host:port) at it.
• Intermediate CA missing on the server side: openssl s_client -connect host:443 -showcerts shows whether the endpoint sends the full chain; retrieve the root and intermediate signers if not.

Enable javax.net.debug=ssl:handshake on a test member to see the exact failure (unable to find valid certification path = truststore, handshake_failure = protocol / cipher). On CP4BA add the certificate to the trusted certificate list in the custom resource (trusted_certificate_list) instead of a truststore.

References

• BAW documentation - configuring SSL for outbound connections (retrieving signers)