Antivirus scan results with Rancher

As of March 2024, antivirus (AV) scanners and cybersecurity solutions have been flagging SUSE Rancher container images for possibly containing viruses or other malware. SUSE Rancher has investigated the situation and found that internally-built and upstream binaries and images are being flagged. The most commonly flagged binaries and images are the BusyBox binary and the Prometheus container image, which also embeds BusyBox.
 

So far, the Rancher Security team has not found a known virus or malware in our images. Our  assessment of the malware detections being recently flagged by external AV solutions is that the detections are all false positives.
 

This situation is aggravated by the fact that some cybersecurity tools, such as VirusTotal, aggregate the scans of multiple AV solutions, which increases the number of detections.
 

The following GitHub issues address these concerns:
 

• Executables from k3s get flagged as malware by Antivirus F-Secure and on virustotal.com: https://github.com/k3s-io/k3s/issues/9738
• Windows Defender alert on harvester-v1.2.0-patch1-amd64.iso: https://github.com/harvester/harvester/issues/5260
• VirusTotal flagging trojan on alertmanager image: https://github.com/prometheus/alertmanager/issues/3733

We suggest that concerned customers use their support channels to directly contact AV vendors and request that vendors verify the safety of the flagged files. Customers should also request that vendors provide them details about which binaries are being flagged and which strings and signatures trigger the detections.
 

After exhausting direct avenues with  AV vendors, customers should  open support tickets with SUSE and report possible malware detections, providing  the reassessment  from  the AV vendor and  associated details about the flagged binaries and signatures. Without such information, we cannot properly assess the virus reports. Since AV vendors do not usually share such details with non-customers, that information must be provided by the customer in their tickets to SUSE.
 

The Rancher Security team runs daily internal AV scans in all images that we use and deliver in our products to our customers and community users. Our automation uses ClamAV, a well-known open-source AV solution.
 

We are continuously reaching out to AV vendors to help increase the accuracy of their scans, asking them to reassess their virus reports for false positives related to the reported binaries and images. Some of these vendors have responded by replying to SUSE, re-evaluating the flagged binaries and images and updating their signatures, such that the spuriously flagged files are no longer reported as possible malware. A few AV vendors have not yet responded to our requests and might still be evaluating their signatures and detections.

From our analysis, a common source of false positives is the known and trusted open-source BusyBox binary, which is being spuriously flagged as malware. This might be due to malicious actors embedding the BusyBox binary inside actual malware or using it during attack campaigns, completely unrelated to the binary’s presence inside SUSE Rancher container images. It might also be due to faulty AV signature matching.
 

Misuse of BusyBox by malicious actors does not necessarily mean that BusyBox is malware or  that it has security issues. Malicious actors are simply using or targeting BusyBox, in the same way that known commercial products use other open-source projects as development libraries. The malicious misuse of BusyBox leads to AV scanners flagging any BusyBox binaries as possible malware, because the scanners previously detected the same binary hash signatures during known attacks.
 

Golang binaries built for Windows are also a known source of false positives, as highlighted on the Golang official documentation.

For more details on the relation between the famous Mirai botnet and BusyBox, and why BusyBox is sometimes flagged as  malware, see https://cybersecurity.att.com/blogs/security-essentials/the-mirai-botnet-tip-of-the-iot-iceberg and https://www.fortinet.com/blog/threat-research/omg--mirai-based-bot-turns-iot-devices-into-proxy-servers. Some additional information about BusyBox is available at https://github.com/docker-library/busybox/issues/29 and https://twitter.com/AmitaiCo/status/1759195098271658123.
 

The Golang project has a virus FAQ due to Golang compiled binaries being frequently flagged as possible malware, see https://go.dev/doc/faq#virus.