Security of BigFix Relay exposed to public Internet

(imported topic written by rmnetops91)

Does anyone have any feedback or information on what kind of exposure to expect having a BigFix relay accessible to the public Internet (so laptops can communicate to the relay while roaming).

I’ve read in other posts not to store passwords in fixlets (which makes sense), but it got me thinking about why? We may not store passwords in fixlets, but there may be other information in fixlets we wouldn’t want the public to access such as what software we might use internally or how we deploy/install/configure software.

How hard would it be for someone to access the fixlet data off of a relay, by either snooping around the 52311/TCP port via a browser/scanner, etc. or by specifically targeting BigFix over this port coming from a machine that is not part of the masthead?

Any feedback would be appreciated.

(imported comment written by BenKus)

Hello,

You shouldn’t store passwords in Fixlets even if you don’t have an Internet-facing Relay because anyone inside your network could snoop it…

But to answer you general question, any data that would be available to an agent would be available to someone pretending to be an agent connecting to your relay. The idea with the Internet-facing Relay is that you do expose your BigFix Fixlets/action to a potential snooper, but that data itself is not particularly useful (provided that you don’t do things like put passwords in Fixlets) and that the increased risk (which most people believe to be very minimal) is definitely offset by the very-large security decreased risk in managing your laptops all over the Internet.

Ben

(imported comment written by SystemAdmin)

Beyond passwords, there still may be, to rmnetops point, security issues with a DMZ-facing relay.

If a given Bigfix implementation is in a relatively vanilla form, just used for patching and deployment of basic scripts, then I agree with Ben that there is little risk relative to the security benefits.

On the other hand, there are some categories that probably should not be made publicly available. Consider that some custom packages may contain installers for various expensive and proprietary software packages. Lets say you distribute the installers via Bigfix. If you had a relay in the DMZ, anyone in the world could download the package and extract the contents. Now they have the latest version of XYZ software that your company just paid a bazillion bucks to purchase. If software that is keyed to your company starts showing up on warez sites on the web, who is responsible then? How do you explain to IBM, Microsoft, Oracle, SAP, et al. how it is that software specifically keyed and licensed to your company is showing up in Romania, Siberia, or Hong Kong?

Here is another circumstance to ponder. A considerable amount of time and resources goes into developing hundreds of custom fixlets and baselines that are used to secure, automate, and control the company’s line of business. This custom content has become a competitive advantage to the company and contains proprietary processes and intellectual property optimized for a specific industry. A competitor would LOVE to tap into this gold mine!

We have not yet, despite being a long-term Bigfix customer, implemented a DMZ relay. We’d love to, but given that we fit into both of the scenarios above, we can’t until better controls are available.

Implementing options for authentication and encryption on a relay would resolve these issues and allow content to be published confidently. Since the relay is a web server, SSL is the logical choice for encryption. I have seen some home-brew attempts to make this work, but none supported officially by Bigfix (as far as I’m aware).

For authentication, I suggest using certificate-based authentication similar to the capabilities of IIS. The certificate advantage is that it could still be seamless to the end user and get the benefits of authentication without a logon box to the client. Since Bigfix is built on a PKI platform, it should be relatively straight-forward to do certificate validation. This would simply extend the concept of encrypting client post data to the next level. Alternatively, either public certificates or a private enterprise certificate infrastructure could provide certificates to do the validation. For example, we already run Microsoft Enterprise CAs that issue various machine and user certificates to our organization.

If it isn’t already on the product roadmap, please add relay SSL encryption and certificate authentication to the feature request list.

(imported comment written by SY57_Jim_Montgomery)

A co-worker and I were talking about exactly how vulnerable a DMZ relay could be, and we didn’t come up with much.

  • An attacker would need to spoof the requests of the Bigfix client.

  • An attacker would need to have the URL of your content he’s trying to download (which uses the sha and URL of your main BES)

  • To get the URL’s he could gather the site, but that’s assuming he knows the site name the content is in

  • An attacker would need to have the current masthead of your install. Without that you can’t gather

  • I suppose an attacker could sniff a packet from a client report to grab what he needs

  • If you use MLE then even that is encrypted.

I don’t see any attack vector on the Relay. I don’t see how a DMZ relay is any risk, if you use MLE, and even then I’m not sure sniffed traffic would get what you need.

-Jim

(imported comment written by rmnetops91)

All very good and interesting points. I’m with JonL in wanting a feature that requires the client agents to authenticate to the server before they can get to fixlet downloads/content. I think using agent client certs is a great idea.

(imported comment written by BenKus)

Hey guys,

Good discussion… I think the points you guys make stand:

Relays expose some info about your policies / packages / custom Fixlets / actions, but Relays generally do not expose “critical info”.

Note that there is another aspect here that an attacker could try to do DoS attacks on your BigFix Server through the BigFix Relay (for instance, if the attacker pretended to be 1,000,000 fake agents, it would make your consoles slow).

But again, I will note that one of the biggest risks that many companies run is that many of the most critical laptops with lots of important info used by important people are out in the open Internet where they are getting attacked constantly… It is very nice to have the ability to update their patches, virus definitions, see their status, etc.

To address the point about authentication, we actually need to implement Agent authentication (rather than server authentication or relay authentication, which are much easier). The reason for this is that in the scenario of the DMZ relay, we are trying to protect against spoofed agents…

To make a scheme like this work, we will need some sort of mechanism to declare and agent as “trusted” and have an ability to do a key exchange so we know that no one is spoofing the agent… but the question for you guys is how would you like to go about the process of “trusting” agents considering so many computers are thrown away, are reimaged, or are new purchases each month… If we are too harsh and require something like a special password to install, then agent deployment gets too hard… but if we are too lenient and allow agents to declare themselves trusted, then we haven’t really accomplished anything because an attacker could spoof the self-trusting process…

We are working on schemes for this for a future version, but do you guys have any suggestions about how this might work from the standpoint of the deployment process to allow a “trusted” agent? Do you mind if the key is widely distributed?

Ben

(imported comment written by SystemAdmin)

Ben,

The best way may be to build a framework that allows for one or more options rather than a one-size-fits-all approach. Consider popular security products and standards in their approach to options.

Lets use the IP security standard as an example. A standards-based approach is adopted by the IT industry. It is optional to even apply the standard. Many mainstream vendors provide out-of-the-box support for the IP security standard. When one chooses to implement IP security, there are many choices that range from weak to DOD strength configurations. The needs of the organization drive the decisions of whether to even implement the standard. If they do, they chose the strength and complexity wanted/needed to secure their particular business.

So applying that to Bigfix, I’d like to see agent authentication be optional and standards-based. Assuming a customer would chose agent authentication, there should be several levels and combinations of options. This would facilitate the range of customer needs anywhere from keeping out the casually curious to industrial strength bullet-proof security.

Those options, at a conceptual level, should address the following:

  1. Data-independent: Protection could be applied either selectively, by group, site, and/or by relevance clause to:

a) Fixlets

b) Tasks

c) Analysis

d) Baselines

e) Actions

f) Report/Post data

  1. Location-independent: It shouldn’t matter where the data is at. There should be options to apply it based on location:

a) At rest within the SQL database (encryption options)

b) At rest on the Root server, relay, or agent (encryption options)

c) In transit over the wire (TLS, SSL, etc.)

  1. Standards-based: Open standards that are well tested, peer reviewd, and respected.

  2. Interoperable with other major security products and protocols

So lets drill down a level and discuss the mechanisms available to achieve these options.

  1. Cryptography plays a huge role here. Bigfix obviously already has a PKI embedded in it. Lets expand on this inherent strength:

a) Use the builtin PKI to control not only the creation and execution of fixlet content, but who (at the console level) can view but not execute (auditor or managment) or execute but not view (delegate responsibilities to lower levels such as help desk without revealing sensitive configurations). At the relay level, optionally encrypt the action script and/or attachments of fixlets in addition to the post/upload encyption currently supported. At an agent level, not only verify the action signature, but also optionally decrypt the action script and/or attachment using either native Bigfix PKI or another specified PKI.

b) Refine delegation controls allowing more granularity.

c) Continue to sign custom content by default, but also offer various levels of encryption of the action script and/or attachments. (Similar to RMS/IRM where the content creator can choose the degree, if any, of protection.)

d) Support enabling trust via either public or private certificate authorities. Microsoft Enterprise Certification Authority integration or interoperability would be useful.

e) Consider “Health” certificate support. Many times Bigfix is used to remediate machines for NAC/NAP/etc. to bring them into compliance. What about expanding on that concept to allow the receipt and qualification of health certificates in additional to remediating agents into a healthy state? In this scenario, any agent, healthy or not could evaluate and run AV, firewall, and security patch sites, but not necessarily other custom sites content. Evaluation and execution of custom site content could be optionally predicated on one or more current health certificates either issued by Bigfix for passing remediation preconditions, but also common ones from Symantec, McAfee, Microsoft SCCM, Cisco, and others. It would be a tiered model such that “premium” content or actions would have higher requirements from the agent.

f) If there is a current user, provide an easy way to qualify user certificates and/or tokens.

  1. Active Directory / LDAP Directory: Directory support could provide an alternate means of authentication. They probably should not be the primary means that an agent would rely upon, but would work well to provide a second or third means of verification.

a) Computer account: A valid computer account in AD could be a way to authenticate.

b) Users and Groups: Obviously these provide many possiblities.

c) Certificate validation (OCSP) in conjunction with AD user, group, and/or computer verification is a powerful combination.

  1. Radius Server support

  2. Custom Relevance Expression: A custom expression could tie together a number of unique aspects for a given organization.

The question you pose, Ben, “Do you mind if the key is widely distributed?” is a challenging one. How does one establish trust when distributing the security key effectively leaves the “key-under-the-doormat”? This is a struggle with many vendors - even security vendors - who have a “secure” agent or communications. Many times it is simply security-by-obscurity.

While I have yet to see a perfect solution, there are definitely some approaches better than others. Offering multiple and diverse options allows customers to optionally apply as few or many options as their business demands. The key is multiple factors. It may not be too difficult to obtain a single factor, but to assemble all the right pieces out of a permutation of options, algorithmns, and strengths becomes a virtually impossible undertaking. For example, if someone knew that everything was protected by a single Bigfix key or certificate on the agent, they could focus on finding or cracking that one key or cert. If they knew that everything was protected by a custom list of attributes uniquely selected by each customer that may include such diverse criteria as health certificate status, computer membership in AD, a current Kerberos group membership LDAP token, etc. in addition to the Bigfix certificate, the bad guys give up since they are facing an impossible number of permutations.

Rather than Bigfix taking a shot in the dark with their one best guess at how customers want to secure their environments, provide a flexible framework into which none, one, or multiple options can be selected by the customer.

(imported comment written by SY57_Jim_Montgomery)

I’ve spent most of today looking at the client traffic to and from my relay with a sniffer. I had thought the license or masthead may be required in order to get an actionsite (and therefore the available downloadables in the fixlets), but I don’t see it at all, even when the client registers.

In my network captures, the only tricky piece of information that the client sends is the last SHA of the site that it knew about. If an attacker knew the sha (which she could get from sniff), I’d guess she could probably get the entire site with no problem, and in turn access any downloadable data. Okay, I agree now that there is a vulnerability. (Sorry, I need to see stuff before I believe it… no I’m not from Missouri.)

Good post JonL.

The problem with a flexible framework with multiple options is A) it takes longer and more resources for BigFix to develop that and B) more options for users doesn’t always mean better deployments. Don’t get me wrong, I think it’s an awesome long term goal.

I, however, would like to have a secure relay that I can deploy to the DMZ TODAY. —> What if clients could generate a public/private key pair, and BES (and certain relays) keep all client public keys. Then you just have to have an admin approve the devices sending in new keys and any key changes. Then all your traffic can be encrypted both ways.

My problem is I don’t think BigFix as a company wants (or really could) implement a PKI infrastructure to cover all the other needs. 3rd party integration is key there (no pun intended).

–Jim

(imported comment written by rmnetops91)

For starters I would suggest simple PKI client certificate verification by the server (i.e. include a client certificate in the BES Agent installer that is customer specific).

#1 A client cert is generated one time during the deployment of the BigFix server components.

#2 This client cert is unique to each BigFix deployment, and is automatically built into the BES Agent Installer along with the masthead, and any other customer specific files that are included with the installers.

#3 When a BES Server receives a connection attempt by a BES agent, it requires client certificate authorization, which the BES agent provides.

#4 If it’s a trusted cert for that BigFix installation, it allows the BES agent to download/access fixlets, etc. If it’s not a valid client cert, it’s denied.

There may be additional questions to consider like (how do you prevent theft of the BES Agent Installer). I can’t answer that within the amount of time I have to post this, but the above would be better than what we have now, and would at least help prevent casual access attempts from unauthorized machines.

(imported comment written by BenKus)

Hey rmnetops,

Thanks for the feedback… We have considered this scheme and it is certainly doable (in fact, the masthead contains a certificate that we could potentially use)… But we definitely have a fear that if we spend the time and resources to implement this solution, we might get an immediate follow-up request that says that the “shared certificate” scheme is not adequate because it is too easy to get your hands on the shared cert… So it might be better than what we have today, but we don’t want to spend a lot of time implementing a big scheme that most people don’t think is sufficient.

Ben

(imported comment written by rmnetops91)

That’s true. My suggestion may only be acceptable to all customers if the cert can be well protected in some way.

(imported comment written by SystemAdmin)

A Bigfix certificate would be a good start and “it is certainly doable”. That is certainly better than nothing, to rmnettops point.

Even if Bigfix only offered its own certificate at the outset, but built a foundation or framework with the potential to hook in additional options in the future would be valuable. Perhaps a future release could add options for a second or third authentication choice. A valid AD/LDAP computer account, user, or group may be a popular second factor.

If a relay could be configured with SSL/TLS encryption and Bigfix certificate-based authentication, we would strongly consider implementing a DMZ relay. Add Active Directory lookup (computer|user|group) for a second factor of authentication and the security guys might actually smile.

(imported comment written by MattBoyd)

After dealing with SCCM a bit, I’m a bit cautious about certificates. For SCCM to work in native mode, you need to run a full blown CA infrastructure and issue a unique client certificate to each computer which is, quite frankly, a PITA.

But as long as there were an easy way in BigFix to get unique certificates to each client during the client installation/subscription process, I don’t think it would be an issue.

(imported comment written by SystemAdmin)

It sounds, from Ben’s post, that any initial authentication option would involve the “shared certificate” inherent in the Bigfix architecture. The clients already have that cert courtesy of the masthead, so there wouldn’t be anything else to deploy. While not ideal, I’d much rather have something than nothing.

As Bigfix progresses down the road of security options for a DMZ relay, I would welcome and vigorously encourage either support of a native PKI with a unique machine certificate or supporting an industry-standard PKI such as Microsoft’s Enterprise Certificate Authorities, PGP, etc. I’ve run both for years. While designing and deploying CAs can be challenging for those not familiar, they offer substantial security benefits in many aspects of an organization. We run a CA infrastructure supplying unique user and computer certificates. If the design and initial deployment are sound, there is little work to maintain them going forward.

(imported comment written by MattBoyd)

JonL, I’m sure you have more experience with Microsoft CAs than me, but one of my concerns is how to distribute the certificates to clients that aren’t in Active Directory. Maybe you know of an automated way to do this, but I was under the impression that not having all clients connected to the same AD forest (or any AD at all) makes it much harder to get a unique certificate to every client and can lead to some support issues. Of course, I could be wrong.

(imported comment written by SystemAdmin)

Boyd,

It is definitely more challenging to get unique certs on non-AD clients, no doubt about it. Simple Certificate Enrollment Protocol (SCEP) was created to address these types of issues. It is an open cross-platform standard for certificate enrollment.

SCEP specification: http://www.ietf.org/id/draft-nourse-scep-20.txt

Microsoft SCEP whitepaper: http://www.microsoft.com/downloads/details.aspx?familyid=E11780DE-819F-40D7-8B8E-10845BC8D446&displaylang=en

SCEP is natively supported in Windows 2008. There is an add-on for Windows 2003: http://www.microsoft.com/downloads/details.aspx?familyid=9F306763-D036-41D8-8860-1636411B2D01&displaylang=en

OpenSCEP: http://openscep.othello.ch/

A Java version of a SCEP client: http://www.urut.ch/scep/

A UNIX SCEP client: http://www.klake.org/~jt/sscep/

Cisco SCEP: http://newsroom.cisco.com/dlls/fspnisapi0b37.html

For non-domain Windows clients, a bit of Certutil scripting can handle cert enrollment.

On a related note, Online Certificate Status Protocol (OCSP) is the corresponding mechanism for certificate verification and validation that is handy for non-AD integrated or non-Windows machines. This is a natively supported role in Windows 2008.

I’m in the process of implementing both SCEP and OCSP to expand to non-Windows devices.

To tie this back to the DMZ relay discussion, if Bigfix does do a secure relay, it would probably make sense to start with the readily available solution (shared Bigfix cert), then support of mainstream options that would benefit a large portion of the customer base (PGP/GPG, Microsoft Enterprise CAs, LDAP/Kerberos), and finally more niche categories could be addressed via SCEP, etc.

(imported comment written by MattBoyd)

Thanks JonL, I just learned something new. Very cool stuff.

(imported comment written by SystemAdmin)

There’s certainly some good ideas here and the key point being that communication needs to be encrypted and the clients need to be validated/approved somehow. It may be useful to look at how RIM does it. All communications are encrypted from the device to the Blackberry server as well, to add a new device without it being directly connected to the network an “activation” password has to be configured on the server and then entered on the device in order to register with the server. I think this method would work very well. If the client is on the local lan and not coming through the gateway it’s approved to connected. If it’s going to connect for the first time via the DMZ/proxy/gateway it will need to be configured with an “activation” password.

Just a thought…

(imported comment written by BenKus)

Hey j2johnson,

I think that makes sense… We just have been reluctant to require that admins or users type in activation passwords to deploy BigFix… but we can work on that idea to make it easier for you guys…

Note that the data coming from the agent is indeed encrypted (as long as you enable encryption in BES Admin).

Ben

(imported comment written by SY57_Jim_Montgomery)

Not all of the data coming from the agent is encrypted. We have MLE turned on in our environment, and I sat with a network sniffer for most of one day.

All the Report data from the client going back up to the relay is encrypted, you know, the results from fixlet evaluations, etc.

However:

initial registration, not encrypted

requests for sites, not encrypted

requests for downloads, not encrypted

So my problem has been that if we put up a DMZ (internet facing) relay, then anyone watching bigfix traffic go by that knows what to look for

could

see what our sitenames are, and in turn gather a site themselves, and have access to all the content.

–Jim