A java app encountered a weird SSL Connection Reset issue while upgrading to use java11. it talked to a legacy service, it works very well while running with java8. somehow 40% percent requests will failure in SSLException after java11.

  java.net.SocketException: Connection reset
  	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
  	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
  	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:110)
  	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1408)
  	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1314)
  	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)
  	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:411)

Cause

The legacy server is running on Oracle WebLogic Server with java8. the server will reset the connection sometimes due to unknown reason.

Analysis

We are using HttpClients custom the default request config.

CloseableHttpClient httpClient = HttpClients.custom().setDefaultRequestConfig(config)      .setSSLSocketFactory(sslCsf).setConnectionManager(cm).build();

I thought it might be cipher suite not match or socket timeout issue. It turns out the not migrated app also will receive the connect reset from the legacy server after enable the logs. The difference from the logs are:

java.net.SocketException: Connection reset // the existed app with java8 runtime
javax.net.ssl.SSLException: Connection reset // the new migrated app with java 11 runtime

I also observed the retry log from the existed app, but not see them in the new one.
org.apache.http.impl.execchain.RetryExec : Retrying request to {s}

Which makes sense, the existed app will retry the requests and got successfully response finally when connection reset happened. but the new app just throw the exception.

Why requested failed with SSLException not retry, but SocketException retried?

The default http request retry handler will not retry for the below 4 exceptions.

Solution

Write a custom http client retry handler and retry the request even encountered javax.net.ssl.SSLException: Connection reset.

You can implement your own HttpRequestRetryHandler class and set it as the retry handler for your http client.

I just copy the same implementation of DefaultHttpRequestRetryHanlder and removed SSLException.class

CloseableHttpClient httpClient = HttpClients.custom().setDefaultRequestConfig(config)
					.setSSLSocketFactory(sslCsf).setConnectionManager(cm)
					.setRetryHandler(new HttpRequestRetryHandler())

Debugging SSL/TLS Connections

use the following command-line option on the java command:

java -Djavax.net.debug=all

The following example shows potential logging settings in application.properties:

logging:
  level:
    org:
      apache:
        http: DEBUG

References:

https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html
https://docs.spring.io/spring-boot/docs/2.1.18.RELEASE/reference/html/boot-features-logging.html