Saw this post and wanted to provide another probable solution to this if you still have issues and notice enhanced security is NOT enabled and minimum client relay setting looks correct.
Many app aware security devices that are set to block http/https on non-standard ports can cause this error to occur. In this case, bigfix client is sending HTTP/HTTPS data over TCP 52311 and a app-aware security device or software in the middle of the communication chain could see that and block as it expects HTTP/HTTPS to only pass over 80/433.
An indication that this is the case is to try these steps:
Run a port check from a client system having issue. For example:
(In Powershell admin prompt):
Test-NetConnection -ComputerName relayname_or_IP -Port 52311 -InformationLevel Detailed
(or any command line test that tests if port 52311 is open)
In this case, with these errors people are reporting above related to masthead setting, this should work and show port is open (this is because the port test doesn’t actually use HTTP/HTTPS so it succeeds).
Note if you have a normal firewall blocking TCP 52311, you’ll get WINSOCK errors in client…not the errors you see above in the thread relating to masthead.
However if you try a CURL command from client system having issue, you may see it fail
curl -k http://relay.domain:52311/cgi-bin/bfenterprise/clientregister.exe?RequestType=Version
This is because the app aware security device detects actual HTTP/HTTPS over 52311 (instead of 80/443) and drops the connection.
Overall, this is a big clue that this is occurring. (Port tests succeed but actual connections using http/https over TCP 52311 fail.)
Solution is is to find the security device in question that is intercepting the client traffic and add/modify rule to allow http or https over 52311.
For example, in something like a Palo Alto, you don’t “enable tcp 52311”, you “enable https on tcp 52311” and “enable http on tcp 52311”.
The layer-4 firewall, when blocking HTTP/HTTPS over non-standard ports tend to allow every TCP SYN / ACK / SYN-ACK handshake to go through and then reset the connection after they detect the protocol that’s running on it.