Relay Mechanism & Data Transfer Process

Usage and Config
1

(imported topic written by sgrewer91)

Hi,

I am in the process of reviewing BF as a potential solution and had a few questions related to the way BF managed data transfers (flow, resilience,bandwidth optimization,cache management). Hopefully someone could shed somelight on the below:

• Does the Software Distribution process support the push of packages to BES Relays?

• Does the OSD process support the push of content/WIM images to BES Relays?

• Assuming both/either of the above are supported:

o What happens if an in progress download between Server->Client; Server->Relay; Relay->Client is interrupted?

o How is package versioning handled i.e. is all content sent or only the difference

o Is content automatically pushed to all relays or are they selected?

• When Clients build Relay list

o Is it only based on closest client or does it consider link saturation levels?

o By default how many Relays can be contained in a list?

o How hard do they try connecting to Relays before they abort and move on?

o If connecting to all Relays in the list fail is an attempt made to recycle through list?

• How is cache managed on the Clients, Server and Relay?

o Is there a cleanout mechanism? How does this work – last used/accessed content…?

• Can Relays support clients which are on different platforms e.g. Relay on Unix – Clients on Windows?

• Dynamic Bandwidth Throttling

o Does it monitor the end-to-end link or just the NIC?

o Is the bandwidth adjusted during flight dependent on bandwidth usage or is this just done once upfront?

• Typical BES Relay architecture – are clients setup to manually or automatically detect relays?

Appreciate any assistance.

Regards,

Shelly G

2

(imported comment written by BenKus)

You should check with your sales engineer to get more information and discussions on these topics. Here are the short answers:

Does the Software Distribution process support the push of packages to BES Relays?

Yes. The relays cache the swdist packages as they are needed by the agents.

• Does the OSD process support the push of content/WIM images to BES Relays?

Yes. The images are stored on the relays that you choose.

• Assuming both/either of the above are supported:

o What happens if an in progress download between Server->Client; Server->Relay; Relay->Client is interrupted?

The download will be resumed from where it left-off when connectivity is restored (the download checkpoint/restart is automatic)

o How is package versioning handled i.e. is all content sent or only the difference

Yes.

o Is content automatically pushed to all relays or are they selected?

Relays are driven on-demand so as soon as an agent requests the download, the relay caches it. It doesn’t bother to cache things that none of its children ask for.

• When Clients build Relay list

o Is it only based on closest client or does it consider link saturation levels?

Relay autoselection is based on the relays closest to the agent on the network. We don’t attempt to measure bandwidth in relay autoselection (because bandwidth is extremely hard to measure accurately and even if you can measure it, bandwidth usage it is a temporary state).

o By default how many Relays can be contained in a list?

No specific limit. The biggest customer that I know of has 15,000 relays. If you have more than 1,000 relays, then typically you want to work with a deployment engineer to make sure you have your settings set to properly optimize your large environment.

o How hard do they try connecting to Relays before they abort and move on?

Agent will attempt an http connection to a relay twice waiting a minute or so between each try… if it fails, it moves on… (I will need to double-check this behavior to make sure I am accurate, but in any case, this behavior is configurable).

o If connecting to all Relays in the list fail is an attempt made to recycle through list?

Yes. According to the relay retry settings.

• How is cache managed on the Clients, Server and Relay?

It is automatically managed with a LRU scheme (we don’t want customers to worry about manual management) - http://support.bigfix.com/cgi-bin/kbdirect.pl?id=232 .

o Is there a cleanout mechanism? How does this work – last used/accessed content…?

Automatic LRU cache replacement policy.

• Can Relays support clients which are on different platforms e.g. Relay on Unix – Clients on Windows?

Yes. Relays don’t care about the OS of their children.

• Dynamic Bandwidth Throttling

o Does it monitor the end-to-end link or just the NIC?

End-to-end.

o Is the bandwidth adjusted during flight dependent on bandwidth usage or is this just done once upfront?

Adjusted based on the bandwidth used ongoing.

• Typical BES Relay architecture – are clients setup to manually or automatically detect relays?

Automatic is recommended whenever possible to save you time and effort: http://support.bigfix.com/bes/misc/besrelays.html . My best guess is that most agents in the world are using automatic relay selection.

Ben

3

(imported comment written by sgrewer91)

Thanks for clarifying things Ben, very much appreciated !

4

(imported comment written by sgrewer91)

One more thing i was interested in finding out about. BigFix has a great reputation for Patch Management. I do however see SW Dist and OSD included but there has not been much mention of this in compelling reasons to purchase - is this becuase these x2 things were added recentely ? I did notice SW Dist in a trial version from last year but this only had a wizard which pushed from root server to clients and bypassed Relay mechanism.

Appreciate your feedback.

5

(imported comment written by NoahSalzman)

Both SW Dist and OSD have been greatly enhanced in the last year… and the marketing materials and technical docs haven’t reached “full bloom” as the patch info has. However…

You can find up-to-date OSD docs here: http://support.bigfix.com/resources.html

You can find an up-to-date SW Dist thread (with links to docs) here: http://forum.bigfix.com/viewtopic.php?pid=29460

6

(imported comment written by sgrewer91)

Hi Ben,

Just looking for some additional detail around x2 items you answered above.

  1. If connection is broken during download you mentioned that it would commence from where it stopped once relay becomes available again. What happens if the inital download relay never recovers ? does the client go to the next relay in list and commence from checkpoint or would it start from the begining again ? how exactly is this controlled ?

2)Cache management - i have seen this as a configurable setting on the server and relay but not client. How is the clients cache managed ? Is it all LRU again with configurable settings ?

Thanks,

Shelly

7

(imported comment written by BenKus)

Hi Shelly,

  1. Yes. I believe the agent can restart its download from another relay. The interaction works by the agent requesting the data (you can think of it like the agent will ask the relay for the file and say “I already have 5 MB of the file so start there”).

  2. Yes. You can control the settings to control the agent cache. The default is that the agent keeps 20 MB LRU cache and it is configurable with settings shown here: http://support.bigfix.com/bes/misc/besconfigsettings.html (see _BESClient_Download_DownloadsCacheLimitMB as an example)

Ben

8

(imported comment written by sgrewer91)

Thanks Ben for responding !

The more I evaluate the more questions keep coming to mind :frowning: The documentation on hand is very very limited in the area of how relays function and the questions i am asking are truly fundemental when one makes an evaluation between products.

Relays:

  1. You mentioned that the client builds a list of relays which it contacts for specific content(patch…) - is this list built first and how does it know when to stop building e.g. i may have 1500 relays and don’t want the entire list to be run through - is this controlled by TTL/Hop settings ? or is there another way of only looking at a area/subset

2)Is there a limit on the number of requests a Relay can handle ? How would one control this if it did exist ?

3)If a client finds it’s closest relay and this does not contain the package it is looking for but lets say the very next relay in the list does has this package will the client then go to the relay with the package or will the closest relay simply pull the package down from another Relay or Root server … ?

4)I would like to know something more around how the best relay is selected, all docs. say it’s the closest relay whcih is always selected but this does not make sense to me becuase so many other things like which has the content should surely apply ?

5)What is the best practice for rolling out relays ? i guess one could roll out as many as you like but are there any implications like load on the network. You mentioned if a client needed content and 2 realys were same distance away it would then just decide but based off what ? relays in this situation could surely cause unecessary issues with multiple pulls from source location ? Is there a rule which says not more than so many in a subnet or something ?

  1. Million dollar question - how does a relay know where to go to get it’s content if there are a few levels present e.g. root > relay > relay > client. ? Again if this works on closest relay are there any other conditions besides HOP ? How does it know to hop up the tree :slight_smile:

7)You mentioned only changes are transfered between two versions is this file level differences or binary diffs. ?

8)Do you need to purchase SWD and OSD as additional modules - anything around how the purchasing may work. Just high level no pricing. Is the product sold in modules essentially ?

9)I have BF version 8.0.584.0 but this does not contain SW Distribution or OSD can someone tell me how i can obtain this so i can play around with it ?

Thanks,

Shelly

9

(imported comment written by JackCoates91)

Hi,

  1. hop count controls this.

  2. the underlying OS is the real limit. For instance, a relay on XP isn’t going to handle much load, while one on Red Hat will handle quite a lot. Presuming that the OS can handle the load, we don’t recommend more than a 1,000 nodes per relay.

  3. Once the client asks a relay, it’s that relay’s responsibility to try upstream for the file – that way clients don’t have to hunt around.

  4. hop count and ping time (if ICMP is open).

  5. it depends on your network infrastructure – I tend to prefer auto-election of systems that are already in place. For instance, let’s imagine a retail environment, and we might promote the back-of-store system as an initial relay, some hub file servers as transfer points, and a few extras in business offices, the DMZ, and the data centers.

  6. each relay looks upstream for another relay or the server, just as an agent does.

  7. inventory and command channel content is binary diff, files are file-level (undiffed).

  8. It’s in suites – TEM-LM (Tivoli Endpoint Manager Lifecycle Management) would set you up with SWD, OSD, Patch, remote control, and inventory.

  9. the 30/30 download has a subset of functions that cross lots of different areas – it’s probably best to contact sales for a real eval license.

10

(imported comment written by sgrewer91)

Q1: So is a list of relays built up at all ? and if so at what stage is this done ?

Q7: What is channel command content exactly ?

Q9: What is 30/30 download you refer to ? is there a link to this or something similar you can share ?

Thanks

11

(imported comment written by sgrewer91)

Q2: How does one control how many connections a relay will manage/allow/permit ?