Migrating service to new hardware, WebUI won't behave

straffin · November 7, 2023, 10:23pm

Plan and progress so far: Moving BigFix from old server a.domain.com to new server b.domain.com, masthead is for old-bigfix.domain.com which is currently a cname for a.domain.com. new-bigfix.domain.com is a cname that points at b.domain.com and will be used for WebUI/Web Reports. At migration time, old-bigfix.domain.com will be re-pointed to b.domain.com. For migration testing, b.domain.com currently has a hosts file re-pointing old-bigfix.domain.com to itself and a few clients have been successfully tested using the same method. SSL Certificate has been generated with b.domain.com as the subject name and both old-bigfix.domain.com and new-bigfix.domain.com as alternative names.

Problem: WebUI won’t deal with any certificates or names, stopping the service.

We’d actually just uninstalled WebUI, manually deleting all pertinent files and BES Client Settings, then reinstalled WebUI. First, the WebUI failed with its self-signed cert:
Tue, 07 Nov 2023 16:36:37 -0500 -- [WebUI] WebUISiteDownloader error: HTTP Error 60: SSL peer certificate or SSH remote key was not OK: SSL: no alternative certificate subject name matches target host name 'old-bigfix.domain.com'
(old-bigfix.domain.com being the name in the masthead, not the name of any server. The self-signed certificate understandably has a subject name of B – no domain.com.)

We swapped in the new SSL cert/key into the appropriate Client Settings (_WebUIAppEnv_WEB_CERT_FILE, _WebUIAppEnv_WEB_key_FILE) after successfully testing the files with Web Reports and having them work just fine. We also changed the _WebUIAppEnv_PLATFORM_HOST Client Setting to both new-bigfix,domain.com and b.domain.com, but get the same error as above.

Gonna submit to HCL Support as well, but wanted to see if anyone here could answer (or at least discuss) before they do.

JasonWalker · November 8, 2023, 2:36am

I’d be interested in hearing what Support has to say on your ticket, but the way I read the message is that WebUI doesn’t like the certificate that the BESRootServer is presenting.

From the WebUI host, try a browser connection to https://old-bigfix.domain.com:52311 and to https://old-bigfix.domain.com:52315 and ensure that the name resolution is correct (WebUI’s HOSTS file sending you to the new server) and that the certificate the server presents is the new certificate you expect, with the SubjectAltName entries

vk.khurava · November 8, 2023, 4:12am

I completed a similar BigFix platform migration from Win2012 to Win2022 (SQL 2019) last week, using two CNAMEs: a.domain.com (as per the masthead) and another that we changed to b.domain.com on the new master. We assigned the old CNAME to a fake root to ensure that connectivity would not be affected, but we had to replace all automation and API queries that referenced the old CNAME with b.domain.com.

I was also receiving the exact same error message for WebUI. Thinking that there might be a problem with WebUI, I uninstalled it and then reinstalled it. This is when I realized that the problem was caused by my implemented SSL certificates, so I removed the SSL certs’ reg keys and allowed WebUI to operate by default. Later, I worked on the certificate part and fixed that as well.

Sharing my conf file below which I used for SSL cert generation for WebUI -

[ req ]
default_bits = 4096
default_keyfile = keyfile.pem
distinguished_name = req_distinguished_name
attributes = req_attributes
req_extensions     = req_ext
prompt = no
output_password = mysecret

[ req_distinguished_name ]
C = US
ST = Washington
L = East Wenatchee
O = BFX
OU = PR
CN = hostname.domain.com
emailAddress = Support@domain.com
[ req_ext ]
subjectAltName = @alt_names
		[alt_names]
		DNS.1 = hostname.domain.com
		DNS.2 = Hostname
		DNS.3 = 11.11.11.11
		DNS.4 = localhost
		DNS.5 = b.domain.com
		DNS.6 = b

[ req_attributes ]
challengePassword = mysecret

vk.khurava · November 8, 2023, 4:23am

I’m sharing the conf file I used for the RESTAPI/Weberport SSL certs below after discovering that the SSL certs and conf file I made for the master RESTAPI/Webreport would not function with WebUI.

The only difference I could find in the two files was the CN. RESTAPI/Webreport functions properly with CN set to b.domain.com, but WebUI refers to the old CNAME from the masthead, so changing the CNAME in the conf file to hostname solved the problem and allowed WebUI to function.

I’m not sure of the underlying reasoning, but once something functions, I usually don’t waste time trying to figure it out further.

[ req ]
default_bits = 4096
default_keyfile = keyfile.pem
distinguished_name = req_distinguished_name
attributes = req_attributes
req_extensions     = req_ext
prompt = no
output_password = mysecret

[ req_distinguished_name ]
C = US
ST = Washington
L = East Wenatchee
O = BFX
OU = PR
CN = b.domain.com
emailAddress = Support@domain.com
[ req_ext ]
subjectAltName = @alt_names
		[alt_names]
		DNS.1 = hostname.domain.com
		DNS.2 = hostname
		DNS.3 = 11.11.11.11
		DNS.4 = localhost
		DNS.5 = b.domain.com
		DNS.6 = b

[ req_attributes ]
challengePassword = mysecret

atlauren · November 8, 2023, 4:54am

This might be too late, but I’d have prepopulated all the certs, on all new/old systems, with all the new/old names as SAN entires. At least then, once the DNS records are sorted, each participant will look at its names and decided “yeah that works”.

Also, each system is really picky about the exact ordering of its cert chain. I’ve bulked up my notes over the years with these.

straffin · November 8, 2023, 3:13pm

As far as I know, I did this much. There’s no point in adding the older actual “server” names as they were never really used, but the masthead “service” name is in there.

That’s what I’m finding odd… the cert and key work fine with Web Reports, but apparently not with WebUI. I’d have thought that they’d use the same logic/rules for cert processing.

straffin · November 8, 2023, 3:39pm

Hrm… it’s not. Pointing the browser on b.domain.com–which has a hosts file pointing old-bigfix.domain.com to itself by IP address–at https://old-bigfix.domain.com:52311 is coming up with a response from a.domain.com, not b.domain.com. I’ve flushed the DNS cache, but even my browser appears to be ignoring the hosts file.

PowerShell’s Test-NetConnection still respects the hosts file entry.

Hrm…

vk.khurava · November 8, 2023, 4:22pm

It’s not; as I indicated above, WebReport and WebUI both agree on a separate common name in the conf file that is used to process SSL certificates.

You can check the following WebUI setting, which is always your masthead server name (I tried changing it based on b.domain.com but it didn’t work). I also had a host file entry, but WebUI is disregarding it based on masthead server name.

_WebUIAppEnv_PLATFORM_HOST

Only after I changed the CN name in my openssl conf file (mentioned before) did WebUI begin to function with SSL certificates and associated settings enabled.

JasonWalker · November 8, 2023, 4:27pm

Maybe the browser is going through a proxy server? Perhaps try with ‘curl’ on the command line, since curl would ignore the OS/browser-configured proxy?

JasonWalker · November 8, 2023, 4:40pm

To be clear here, there are still plenty of pieces in this interaction that I don’t fully understand myself, but hopefully this may still be helpful.

When WebUI connects to the root server, it uses a different port - instead of connecting on the default 52311, by default it connects to a separate dedicated port 52315, which is used only by WebUI. The HTTPS listener on 52315 uses a different self-signed certificate; the custom cert you’re adding on the Root server does not change the certificate we use on the 52315 listener.

Checking mine, the certificate we generate for the port-52315 listener has a Subject matching the masthead name, and Subject Alt Names matching the masthead name, IPv4 addresses, IPv6 addresses, ‘localhost’, and the loopback IP addresses (my root server’s real hostname doesn’t appear anywhere in the list). I don’t know whether it’s possible to add more SubjectAltNames to that certificate.

I’m not a huge fan of this workaround, but I expect the simplest thing to start moving is to add a HOSTS file entry on the WebUI server, directing the masthead name ‘bigfix.domain.com’ to the new root server’s IP address, and configure the _WebUIAppEnv_PLATFORM_HOST setting to match the masthead name bigfix.domain.com (and let the HOSTS file entry direct it to whichever IP address is correct).

straffin · November 8, 2023, 4:48pm

THIS is exactly what we have now that’s not working.

straffin · November 8, 2023, 4:49pm

I’m gonna look into this. Thanks!

JasonWalker · November 8, 2023, 5:05pm

Hmm. I think the first part is to make sure the name resolution is working - pinging the masthead name, does it go to the IP of the new server? I could see the browser getting to the wrong place via proxy, but at least ‘ping’ needs to get to the right server first.

That said, I don’t see why we’d get that particular message even if we ended up connecting to the old server instead of the new - the old server should also have a self-signed cert with the SubjectAltName matching the masthead name. (If WebUI connected to the old server, there should be a different error in that WebUI’s client cert should not be authorized on the old server but I’m not seeing that in your log)

If there are any Proxy configs on Windows or in the BES Settings, be sure you’ve added proxy bypass exclusions for all of these hostname aliases involved?

I feel like I’m missing something here but it’s a little difficult to visualize how everything’s configured just from the descriptions. I think Support has a better chance reviewing your configuration in a live session

JasonWalker · November 8, 2023, 5:08pm

Are you changing masthead names as part of this migration as well? I’m a bit unclear on that point… at first I thought you were keeping the same masthead name and just moving the services to new hosts but on re-reading, I’m thinking maybe “old-bigfix.domain.com” is one masthead and “new-bigfix.domain.com” is a new masthead?

Or are “old-bigfix” and “new-bigfix” just aliases for users to hit WebUI and Web Reports from their browsers?

straffin · November 8, 2023, 5:26pm

No… keeping the old-bigfix.domain.com masthead the same from a.domain.com to b.domain.com, but the desire is to have folks access the server on b.domain.com via new-bigfix.domain.com for WebUI access. Using new-bigfix.dmoain.com should just be a browser, cert, and SAML auth issue, though, and not affect the actual operation of the WebUI. Or so I thought.

straffin · November 8, 2023, 5:27pm

Heh… yeah. It’s also really hard to keep translating to these sanitized names instead of our actual servers, but using the actual names in a forum seems like a bad idea.

atlauren · November 8, 2023, 5:33pm

If memory serves[*], the HOSTS file is only observed by certain (legacy) APIs; many “modern” ones in fact have an entirely different code path through TCP/IP.

Am I remembering legacy/modern Windows? macOS? That part, in truth, I cannot recall.

straffin · November 8, 2023, 5:42pm

I’ve no idea myself. Networking is my one true weak point. I can say, however, that PowerShell’s Test-NetConnection (aka PowerShell ping) respects the hosts file, so there’s that…

straffin · November 8, 2023, 5:42pm

Requesting a new cert with old-bigfix.domain.com as the CN with everything else as an SAN…

straffin · November 9, 2023, 12:05am

Also, at the request of HCL Support, I’ve uninstalled the WebUI that we’d installed via downloaded installer and re-installed via fixlet. Even before getting to applying new SSL certs, the service-wrapper.log file shows the following:

Wed, 08 Nov 2023 18:20:23 -0500 -- WebUI service version 11.0.0.175 starting
Wed, 08 Nov 2023 18:20:23 -0500 -- OpenSSL Initialized (Non-FIPS Mode)
Wed, 08 Nov 2023 18:20:23 -0500 -- Using OpenSSL crypto library OpenSSL 3.1.1 30 May 2023
Wed, 08 Nov 2023 18:20:23 -0500 -- [WebUI] WebUISiteDownloader error: HTTP Error 60: SSL peer certificate or SSH remote key was not OK: SSL: no alternative certificate subject name matches target host name 'masthead.server.name'

So… yeah…