Hi Team,
We are running TRC version 9.1.2.00.20 with standalone WAS and remote DB2, we have RC on Linux server.
Now when we try to run our RC web portal to we getting DB connection error and sometime even webpage doesn’t starts. After checking RC server log we can see below error :
25-Aug-2018,08:14:37 [WebContainer : 41] [ERROR] - Threw SQLException whilst getting a connection from a datasource :J2CA1010E: Connection not available; timed out waiting for 180,000 seconds.
25-Aug-2018,08:14:37 [WebContainer : 41] [ERROR] -
An unexpected application error occurred. Details are shown below. There may be additional information in the log files.
Status: 500 - (Internal server error) The server encountered an error and can’t fulfill the request.
Exception occurred: class java.lang.NullPointerException
Message: java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
at com.ibm.uk.greenock.db.PromptedQuery.(Unknown Source)
at com.ibm.uk.greenock.asset.RCAsset.hwKeyMatch(Unknown Source)
at com.ibm.uk.greenock.asset.RCAsset.getHWKey(Unknown Source)
at com.ibm.uk.greenock.callHome.CallHomeDAO.processCallHome(Unknown Source)
at com.ibm.uk.greenock.servlet.CallHomeExec.handleRequest(Unknown Source)
at com.ibm.uk.greenock.servlet.CallHomeExec.doPost(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1233)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:782)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:481)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)
at com.ibm.uk.greenock.filter.SecureRequestFilter.doFilter(Unknown Source)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.uk.greenock.filter.UTF8Filter.doFilter(Unknown Source)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:967)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1107)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:949)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1817)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:463)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:530)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:316)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:287)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1187)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink$MyReadCompletedCallback.complete(SSLConnectionLink.java:694)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1833)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
25-Aug-2018,08:20:49 [WebContainer : 44] [INFO ] - Attempting match with the old target data
25-Aug-2018,08:20:56 [WebContainer : 42] [ERROR] - Threw SQLException whilst getting a connection from a datasource :J2CA1010E: Connection not available; timed out waiting for 180,000 seconds.
25-Aug-2018,08:20:56 [WebContainer : 42] [INFO ] - Database connection error
25-Aug-2018,08:21:08 [WebContainer : 43] [ERROR] - Threw SQLException whilst getting a connection from a datasource :J2CA1010E: Connection not available; timed out waiting for 180,000 seconds.
25-Aug-2018,08:21:08 [WebContainer : 43] [ERROR] -
An unexpected application error occurred. Details are shown below. There may be additional information in the log files.
Status: 500 - (Internal server error) The server encountered an error and can’t fulfill the request.
Exception occurred: class java.lang.NullPointerException
Message: java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
at com.ibm.uk.greenock.db.PromptedQuery.(Unknown Source)
at com.ibm.uk.greenock.asset.RCAsset.hwKeyMatch(Unknown Source)
at com.ibm.uk.greenock.asset.RCAsset.getHWKey(Unknown Source)
at com.ibm.uk.greenock.callHome.CallHomeDAO.processCallHome(Unknown Source)
at com.ibm.uk.greenock.servlet.CallHomeExec.handleRequest(Unknown Source)
at com.ibm.uk.greenock.servlet.CallHomeExec.doPost(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1233)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:782)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:481)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)
at com.ibm.uk.greenock.filter.SecureRequestFilter.doFilter(Unknown Source)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.uk.greenock.filter.UTF8Filter.doFilter(Unknown Source)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:967)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1107)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:949)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1817)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:463)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:530)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:316)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:287)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1187)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink$MyReadCompletedCallback.complete(SSLConnectionLink.java:694)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1833)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:816)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1892)
25-Aug-2018,08:21:41 [WebContainer : 14] [ERROR] - Threw SQLException whilst getting a connection from a datasource :J2CA1010E: Connection not available; timed out waiting for 180,000 seconds.
25-Aug-2018,08:21:41 [WebContainer : 14] [ERROR] -
An unexpected application error occurred. Details are shown below. There may be additional information in the log files.
Status: 500 - (Internal server error) The server encountered an error and can’t fulfill the request.
Exception occurred: class java.lang.NullPointerException
Message: java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
at com.ibm.uk.greenock.db.PromptedQuery.(Unknown Source)
at com.ibm.uk.greenock.asset.RCAsset.perfectMatch(Unknown Source)
at com.ibm.uk.greenock.asset.RCAsset.getHWKey(Unknown Source)
at com.ibm.uk.greenock.callHome.CallHomeDAO.processCallHome(Unknown Source)
at com.ibm.uk.greenock.servlet.CallHomeExec.handleRequest(Unknown Source)
at com.ibm.uk.greenock.servlet.CallHomeExec.doPost(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1233)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:782)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:481)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)
at com.ibm.uk.greenock.filter.SecureRequestFilter.doFilter(Unknown Source)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.uk.greenock.filter.UTF8Filter.doFilter(Unknown Source)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:967)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1107)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:949)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1817)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:463)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:530)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:316)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:287)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1187)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink$MyReadCompletedCallback.complete(SSLConnectionLink.java:694)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1833)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:816)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1892)
25-Aug-2018,08:21:41 [WebContainer : 14] [ERROR] -
An unexpected application error occurred. Details are shown below. There may be additional information in the log files.
Status: 500 - (Internal server error) The server encountered an error and can’t fulfill the request.
Exception occurred: class java.lang.NullPointerException
Message: java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
at com.ibm.uk.greenock.db.PromptedQuery.(Unknown Source)
at com.ibm.uk.greenock.asset.RCAsset.perfectMatch(Unknown Source)
at com.ibm.uk.greenock.asset.RCAsset.getHWKey(Unknown Source)
at com.ibm.uk.greenock.callHome.CallHomeDAO.processCallHome(Unknown Source)
at com.ibm.uk.greenock.servlet.CallHomeExec.handleRequest(Unknown Source)
at com.ibm.uk.greenock.servlet.CallHomeExec.doPost(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1233)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:782)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:481)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)
at com.ibm.uk.greenock.filter.SecureRequestFilter.doFilter(Unknown Source)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.uk.greenock.filter.UTF8Filter.doFilter(Unknown Source)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebApp
29-Aug-2018,09:17:05 [WebContainer : 25] [ERROR] - Threw SQLException whilst get ting a connection from a datasource :J2CA1010E: Connection not available; timed out waiting for 180,000 seconds.
29-Aug-2018,09:17:05 [WebContainer : 25] [INFO ] - Database connection error
29-Aug-2018,09:17:26 [WebContainer : 48] [ERROR] - Threw SQLException whilst get ting a connection from a datasource :J2CA1010E: Connection not available; timed out waiting for 180,001 seconds.
29-Aug-2018,09:17:26 [WebContainer : 48] [INFO ] - Database connection error
After that if we reboot our WAS and DB instance will be able to connect the webpage again.
Please help us to resolve this issue.