Fun times with TLS 1.2

Just thought I would let everyone know my past few days experience. We enabled Enhanced Security/SHA256 downloads last Spring with no issues. This past week it was determined that the connection from root server to the remote SQL server was still using TLS 1.0.

Wednesday night we ran a reg file that disabled SSL 2.0/3.0 and TLS 1.0/1.1 on our remote SQL server. Came in the next morning and no ODBC connection. I updated the Native SQL client from 2011.110.55058.00 to 2011.110.7001.00 Link on the root server which enabled ODBC again. Got error on BESRelay log that the database was not initialized. Ran BESAdmin, no change. Opened a Sev1 PMR and worked with Dave L, Mark L, and a few SQL guys. We tried to delete and recreate the encryption keys but were not successful.

In the end (8 hours later), we re-enabled TLS 1.0/1.1 and restored the DB to prior to the change. Then we created a new set of encryption keys via BESAdmin and a script for WebUI, rebuilt ODBC (again) and finally got comms restored.

They are going to test a few solutions for me and hopefully get some guidance on how to enable TLS 1.2 going forward from root to remote db.

Both servers are Win2016 vms. SQL 2016

Has anyone else had any issues using a remote database? At the minimum make sure to update your SQL Native Client on the root server prior to disabling 1.0/1.1. I did both x86 and x64.

4 Likes

I’m sorry for your horrible experience (I’m sure there were some post adult beverage). Thank you for sharing. Our SQL server, also SQL 2016, is local on the BES server. I’m wondering, and hoping, that we would not be at risk should be ever be forced/required to mess with TLS.

1 Like

Thanks. Hoping to get some better guidance from HCL on how to move forward. I plan to wait until after the March patch cycle at least to try again.

The below sequence worked fine for me, so there must be some inner aspects that make the difference. Official Support is required I think to help identify the root cause of the issue.

  1. Freshly installed BigFix 9.5.5 on WIN2K16 system, with remote SQL Server 2016 SP2 running on another WIN2K16 system (Note: BigFix 9.5.5 installs SQL Native Client 2011.110.5058.00)

  2. Successfully enabled “Enhanced Security/SHA256 downloads” on RootServer

  3. Disabled SSL 2.0/3.0 and TLS 1.0/1.1 on the remote SQL Server system

  4. At this point, RootServer started logging this message repeatedly:

Mon, 11 Mar 2019 11:42:58 +0100 - Main Thread (4996) - Startup failed: error testing database connection: Database Error: [Microsoft][SQL Server Native Client 11.0]TCP Provider: An existing connection was forcibly closed by the remote host.
_ (08001: 10,054)_
[Microsoft][SQL Server Native Client 11.0]Client unable to establish connection (08001: 10,054)

  1. On the RootServer system, manually updated SQL Native Client from 2011.110.5058.00 to 2011.110.7001.00 as per BigFix 9.5 Patch 6 is now available (post #8)

  2. After the update of SQL Native Client, all BigFix server processes could successfully connect to the remote database, and the BigFix deployment resumed full function mode

  3. Successfully upgraded all BigFix server components to 9.5.11

3 Likes

Would you be able to post the content of the .reg file you used to disable TLS 1.1, 1.0, and SSL v3? Also the versions of Windows and SQL on the remote host?

I’m curious to see whether the cipher suites are also affected in the .reg file, and certain combinations of Windows and SQL server required updates on the database server to support TLS 1.2.

Were there any other database clients besides bigfix that use this server, and were they successful when you disabled the older protocols?

On my test system, running C:\Windows\SysWOW64\odbcad32.exe, Drivers tab, SQL Server Native Client 11.0 shows version 2011.110.6518.00. Our BES is W2K12R2 with a local SQL 2016 install.

This version is newer than your fresh install. Do we know the minimum version required as so not to have this issue arise in the future?

1 Like

My steps were similar to aginestr, except we already had 9.5.11 in place. Mine failed at roughly #6 with the following error:
Wed, 06 Mar 2019 08:58:37 -0800 – 5580 – Unable to connect to database: Database Error: The database is not initialized.
Wed, 06 Mar 2019 08:58:47 -0800 – 5580 – Unable to connect to database: Database Error: The database is not initialized.

The only way to recover was to restore the db and create new keys.

Also Note: There is an IBM support page (https://www-01.ibm.com/support/docview.wss?uid=swg21506136) for that error. Do not follow it. Running BESAdmin on the sql server is a no-no apparently.

Also, one factor on my side could be that, due to connection issues, we were running SQL authentication instead of Windows authentication for our ODBC connections, and the services running as local system on the root server. I plan on working with our DBs to get that corrected using our BF service account.

As far as I know, minimum version of SQL Native Client supporting TLS 1.2 should be 2011.110.6020.0, so you should be OK.

1 Like

I’ve also tried with step 7 after step 2, and still successful, so it must be something else.

In similar fashion, if your database isn’t serving TLSv1.2, then BigFix Inventory (formerly SUA) and Compliance (formerly SCA) will also both fail to start when NIST SP 800-131 mode is set to “strict”. That setting causes the web server to require a TLSv1.2 DB connection.

I am on 9.5.11 on Server 2016 and SQL 2016. I have separate servers for Core, SQL, Web Reports, Inventory, and Compliance.

We updated the SQL native client to 11.4.7001.0 on servers with an older version. This was definitely needed on BigFix infrastructure servers. The older version installed by default did not support TLS 1.2. I believe that I originally installed 9.5.8 in this new environment. No older than 9.5.6 for certain.

I ran the following weak cipher baseline. Note that many of these were not enabled, but it is a baseline for all servers, not just BigFix, so there is some legacy stuff. These did not impact anything in BigFix.

Backup cipher and protocol config
Multi-Protocol Unified Hello Client
Multi-Protocol Unified Hello Server
Disable SSL 2.0 Protocol Server
Disable SSL 2.0 Protocol Client
Disable PCT 1.0 Protocol Server
Disable PCT 1.0 Protocol Client
Disable DES 56/56 Cipher
Disable NULL Cipher
Disable RC2 128/128 Cipher
Disable RC2 40/128 Cipher
Disable RC2 56/128 Cipher
Disable RC4 128/128 Cipher
Disable RC4 40/128 Cipher
Disable RC4 56/128 Cipher
Disable RC4 64/128 Cipher
Disable Triple DES 168 Cipher
Enable AES 128/128 Cipher
Enable AES 256/256 Cipher

I then disabled TLS 1.0 and 1.1. TLS 1.2 was previously enabled, and I had told apps to use 1.2 where applicable. This reg key exists for both client and server for 1.0, 1.1, and 1.2.Note that the key “disabled by default” does not actually disable anything. This one does.

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Server]
“Enabled”=dword:00000000

After this, everything works except for the Security and Compliance application server. It lost its database connection. I re-enabled TLS 1.0 on it and the SQL server to get it working again just this morning. I have not dug deeper to see what I missed yet.

Then I got my weekly email that had this thread as the lead, so I thought I’d chime in…

If no SQL Native Client 11.x is already present on the system, BigFix 9.5.6 installs SQL Native Client 11.3.6538.0, which is supposed to support TLS 1.2. Any 11.x version of SQL Native Client older than 11.3.6538.0 must have been installed by another party, and then used also by BigFix 9.5.6.