WAN Network overload because actionsite downloads over and over again

1

(imported topic written by LeoCiociano)

Hi,

I had a problem yesterday and it persists today.

I have about 300 of Relays connected by slow WAN links to the TEM server, every relay with its own WAN connection.

Suddenly, yesterday in the afternoon, the actionsite (Master Action Site) started to download to relays over and over again, provoking the WAN connections to get significantly slow. The only way I had to stop this traffic was stopping the TEM server services, and when I startup it again, it continues trasmitting the actionsite again. I don´t know what to do. I opened a PMR at IBM, but I didn´t get a response yet.

This is an extract from a logfile on one of the relays…

At 03:36:48

-0300 -

GatherActionMV command received. Version difference, gathering action site.

At 03:36:50

-0300 - actionsite (http://TEMSERVER:52311/cgi-bin/bfgather.exe/actionsite)

Download

‘http://127.0.0.1:52311/bfmirror/bfsites/manydirlists_1/__diffsite_6b3313afc64456d48c2d0c7caea90483b1fca312_to_bc03ca57a0af7d8958ba395a1a92e8907f39ebd3’ as ‘__TempUpdateFilename’

At 03:37:00

-0300 - actionsite (http://TEMSERVER:52311/cgi-bin/bfgather.exe/actionsite)

Successful Synchronization

with FixSite (version 8038,379) - ‘http://127.0.0.1:52311/cgi-bin/bfenterprise/BESGatherMirror.exe?url=http://TEMSERVER:52311/cgi-bin/bfgather.exe/actionsite&Time=19Sep03:36:49&rand=1a5d304f&ManyVersionSha1=6b3313afc64456d48c2d0c7caea90483b1fca312’

At 03:41:52

-0300 -

GatherActionMV command received. Version difference, gathering action site.

At 03:41:53

-0300 - actionsite (http://TEMSERVER:52311/cgi-bin/bfgather.exe/actionsite)

Download

‘http://127.0.0.1:52311/bfmirror/bfsites/manydirlists_1/__diffsite_43add567b06463744a28c0326be5ffbcb668c0ef_to_600d4cdc5c347cbb3596ef71169aa847094a053e’ as ‘__TempUpdateFilename’

At 03:42:03

-0300 - actionsite (http://TEMSERVER:52311/cgi-bin/bfgather.exe/actionsite)

Successful Synchronization

with FixSite (version 8040,381) - ‘http://127.0.0.1:52311/cgi-bin/bfenterprise/BESGatherMirror.exe?url=http://TEMSERVER:52311/cgi-bin/bfgather.exe/actionsite&Time=19Sep03:41:52&rand=6f1ec67f&ManyVersionSha1=43add567b06463744a28c0326be5ffbcb668c0ef’

At 03:44:01

-0300 -

Report posted successfully.

At 03:46:14

-0300 -

GatherActionMV command received. Version difference, gathering action site.

At 03:46:16

-0300 - actionsite (http://TEMSERVER:52311/cgi-bin/bfgather.exe/actionsite)

Download

‘http://127.0.0.1:52311/bfmirror/bfsites/manydirlists_1/__diffsite_600d4cdc5c347cbb3596ef71169aa847094a053e_to_6944a193ee3292820ef0d5ef95b0cdf51c23dcaa’ as ‘__TempUpdateFilename’

At 03:46:25

-0300 - actionsite (http://TEMSERVER:52311/cgi-bin/bfgather.exe/actionsite)

Successful Synchronization

with FixSite (version 8040,383) - ‘http://127.0.0.1:52311/cgi-bin/bfenterprise/BESGatherMirror.exe?url=http://TEMSERVER:52311/cgi-bin/bfgather.exe/actionsite&Time=19Sep03:46:15&rand=f24af827&ManyVersionSha1=600d4cdc5c347cbb3596ef71169aa847094a053e’

Anyone knows what it might be happening?

Thanks in advance.

Leo

2

(imported comment written by SystemAdmin)

If you haven’t yet you will want to contact support on this.

3

(imported comment written by LeoCiociano)

Hi Jimmie,

I have already solved it with help of the support team.

Thanks,

Leo

4

(imported comment written by MarlonRojas)

Leo can you tell us how you resolve this problem? any flag or something like that?

5

(imported comment written by LeoCiociano)

Hi Marlon,

This was the problem. We have a DSA Environment, with two replicated Databases. We used Computer Remover Tool to delete old and duplicated computers. This tool had a bug deleting inactive relays on DSA environment, this caused databases start replicating lots of useless data. This made every computer on the environment to download all this data too, all the time.

Diagnostic: Support team found a bad relay registry on the database modified by the Computer Remover Tool. This was generating the useless replication, and generating lots of traffic.

Solution: I manually edited this registry, as Support team told me, and bad replication traffic stopped.

After this incident, a new version of Computer Remover Tool (
3.0.1.124)
was released fixing this issue.

3.0.1.123 ~~~–> 3.0.1.124: Changed the tool so that it will no longer set the LastReportNumber column in the COMPUTERS table to 0. This prevented the row from beng replicated to the other DSA severs, and the servers would not know that the computer habeen deleted.

Last version of the tool is
3.0.1.128.

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Endpoint%20Manager/page/TEM%20Computer%20Remover

Hope this was helpfull.

Regards,

Leo

6

(imported comment written by MarlonRojas)

Leo, thanks a lot !