We moved the database locally and no change. What we landed on with support was the WebReports server which was co-located was spawning excessive (2,250+) TCP connections on 127.0.0.1 That was ultimately filling up the connections and making the Root Server services non-responsive. We disabled Web Reports and you can watch the TCP connections drop. Keep in mind Web Reports was not in use at all. If you want to see what I am talking about try running netstat -abn | find /c on a server where WebReports and the Root Server are on the same server. Then disable the Web Reports service and you can see that number drop dramatically. Ultimately, we are moving Web Reports off and HCL is looking at the reason for this.
3 Likes