BigFix (apt) removed ssh server for Ubuntus

DBalazs 2022-02-22 18:06:16 UTC #1

Hi!

We’ve ran into a pretty disturbing incident that during BigFix patching on several of our Ubuntu servers the openssh server package got simply removed, remote ssh access became impossible. According to the apt history log it seems with the update of the openssh-client package, the openssh-server package got removed and not updated as it should have been. And then the server was left like this. This looks really bad to be honest.

dpkg lists the server package as removed:
rc openssh-server 1:7.6p1-4ubuntu0.3 amd64 secure shell (SSH) server,

/var/log/apt/history.log shows the cause:
Start-Date: 2022-02-19 03:00:43
Commandline: apt-get -qqy install openssh-client=1:7.6p1-4ubuntu0.5
Upgrade: openssh-client:amd64 (1:7.6p1-4ubuntu0.3, 1:7.6p1-4ubuntu0.5)
Remove: openssh-sftp-server:amd64 (1:7.6p1-4ubuntu0.3), openssh-server:amd64 (1:7.6p1-4ubuntu0.3)
End-Date: 2022-02-19 03:00:48

Did anyone else experience such issue, is there any way to prevent such removals instead of upgrades from happening again?

Thanks,
Balazs

JasonWalker 2022-02-22 18:56:40 UTC #2

I haven’t seen that specifically…do you know which Fixlet you ran that may have caused this?
I see some similar complaints in the Mint Linux forums, which are also Debian-based. It may have to do with the openssh-client and openssh-server being required to stay at equal versions, I see the openssh-server that was removed is an older version than the openssh-client that was installed.

DBalazs 2022-02-22 19:10:42 UTC #3

Basically all security fixlets relevant at least for 1 server are put into a Baseline at the beginning of each month and it gets distributed on all servers. So in the baseline there was the newer version of openssh-client but also the new openssh-server fixlet. And then according to the apt history the older version of the openssh-server has been removed during the install of the newer openssh-client package, and then the new version of the server was not re-installed anymore. But then this could happen to a lot of other different version-dependent packages which is scary and we did not experience such behavior so far. Well it’s true that we have mostly RedHat based servers.

JasonWalker 2022-02-22 19:20:07 UTC #4

If you don’t have one open already, I’d suggest opening a support incident on this. I think our Ubuntu content team may need to get some more details on what you’re seeing, and I’m afraid this is a little out of my area of knowledge.

DBalazs 2022-02-23 07:20:18 UTC #5

Yes thanks I’ll do that.