Tuesday, 19 June 2018

Launching VirusTotal Monitor, a service to mitigate false positives

One of VirusTotal’s core missions is to empower our antivirus partners. By building better tools to detect and study malware, VirusTotal gets to make a dent in the security of billions of users (all those that use the products of our partners). Until now we have focused on helping the antivirus industry flag malicious files, and now we also want to help it fix mistaken detections of legit files, i.e. false positives. At the same time, we want to fix an endemic problem for software developers.

False positives impact antivirus vendors, software developers and end-users. For example, let us imagine a popular streaming service app that allows in-app digital content purchases. We will call it Filmorrific.

Filmorrific happens to be so popular that when an antivirus vendor mistakenly flags it as malware, the AV vendor gets terrible press as major online news sites, computer magazines and blogs report on the issue. This leads to big reputation damage for the AV vendor.

The detection of Filmorrific leads to the software being quarantined and blocked from running on end-user machines. End-users are now unable to access their favourite streaming service, and they are also confused, thinking that Filmorrific has trojanized their machines.

For Filmorrific, the software publisher, this immediately translates to blacking out in the entire user base of the detecting AV vendor. Suddenly, they not only lose revenue coming from the installed base, but also trust from less technical users that do not really understand what is going on, and they get overloaded with support tickets accusing them of infecting user machines. Filmorrific in turn decides to sue the detecting antivirus company for the damage, and we have now come full circle.

Note that in this context, a software developer is not only a company creating an app or program distributed to thousands of machines and including some kind of monetisation strategy. Today, almost every organization builds internal tools that their finance, accounts payable, HR, etc. teams use. All of these tools are prone to false positives, and while this might not have a revenue impact, it certainly has a cost in terms of productivity hours lost because the workforce can’t access a given app.

What if we could kill these three birds with the same stone? Enter VirusTotal Monitor. VirusTotal already runs a multi-antivirus service that aggregates the verdicts of over 70 antivirus engines to give users a second opinion about the maliciousness of the files that they check. Why not take advantage of this setup not only to try to detect badness, but also to flag mistaken detections of legit software?

VirusTotal Monitor is a new service that allows software developers to upload their creations to a private cloud store in VirusTotal. Files in this private bucket are scanned with all 70+ antivirus vendors in VirusTotal on a daily basis, using the latest detection signature sets. Files also remain absolutely private, not shared with third-parties. It is only in the event of a detection that the file will be shared with the antivirus vendor producing the alert. As soon as the file is detected, both the software developer and the antivirus vendor are notified, the antivirus vendor then has access to the file and its metadata (company behind the file, software developer contact information, etc.) so that it can act on the detection and remediate it if it is indeed considered a false positive. The entire process is automatic.

For antivirus vendors this is a big win, as they can now have context about a file: who is the company behind it? when was it released? in which software suites is it found? What are the main file names with which it is distributed? For software developers it is an equally big win, as they can upload their creations to Monitor at pre-publish stage, to ensure a release without issues. They can also keep their files in the system, to automate notification of false positives to antivirus vendors in the future. Software developers no longer have to interact with 70 different vendors, each having its own interface and strenuous process to communicate issues.

In particular, software vendors use a Google-drive like interface where they can upload their software collections and provide details about the files:

Upon upload, the files are immediately scanned with the 70+ antivirus engines in VirusTotal, and then once a day thereafter. At any point in time you can refer to the Analyses view in order to see the health of your collection with respect to false positives:

All of this scanning activity is summarized in the dashboard where users land on subsequent accesses to the platform:

Developers are not forced to use this web interface, as the platform allows email notifications and offers a full REST API that is very useful when automating software release pipelines:

On their end, antivirus vendors also see something similar. They get access to a platform with all items that the particular engine detects and they can integrate with it programmatically via a different API endpoint. This is how certain vendors are able to quickly react and get over 200 false positives from our test bed fixed within minutes:

As previously stated, all files in this flow are private; they are not distributed to third-parties, only to antivirus vendors producing detections. This said, if one of the files in a Monitor collection happens to be uploaded to the standard public VirusTotal service, we will highlight that the file belongs to an organization in Monitor and will display the pertinent detections in orange rather than red:

VirusTotal Monitor is not a free pass to get any file whitelisted, sometimes vendors will indeed decide to keep detections for certain software, however, by having contextual information about the author behind a given file, they can prioritize work and take better decisions, hopefully leading to a world with less false positives.  The idea is to have a collection of known source software, then each antivirus can decide what kind of trust-based relationship they have with each software publisher.

As Marc Andreessen once said, “software is eating the world”, however, there is not much it can eat unless it can actually execute -- let’s make sure that legit software can run.

No comments:

Post a Comment