Compliant, easy and actionable integration of VirusTotal in 3rd-party products - Welcome VT Augment

TL;DR: We are releasing an official, compliant and recommended method for displaying VirusTotal context in 3rd-party products and services, so that end-users can enjoy a single pane of glass experience when working with their tools of choice. Read the docs / See the demo (click on the VirusTotal icon next to each observable).



Security analysts world-wide are demanding a single pane of glass experience from their products. Corporate cybersecurity stacks are increasingly complex: too many tools and services, information scattered across numerous databases, arduous stitching together of disparate sources, etc. Incident response and threat hunting have become a time consuming quest across multiple browser tabs. The experience is poor.

If you develop some kind of security product, you will probably know that a common request coming from users is to integrate VirusTotal threat context and reputation. CrowdStrike can speak to this popular demand, just recently we worked together to build a Falcon-VirusTotal integration for their CrowdStrike store. We will be speaking about this and other integrations with our antivirus partners in future posts.

Notwithstanding, at VirusTotal we have to make sure that our data is not misused to the detriment of the ecosystem, this is why we have a strict policy regarding scanning companies and use of our services. This is also why our premium service terms prohibit displaying raw data in 3rd-party products and interfaces, especially those exposed to end-customers.

This said, over time we have seen many legit use cases for integration, mostly revolving around enrichment (adding a layer of context) of alerts/detections that get generated through some means other than VirusTotal. Indeed, when Incident Responders and SOC analysts review alerts, they want to answer questions such as:

• Given a hash in an alert, is there any second stage payload that I should be searching for in my environment?
• What’s the C2 infrastructure tied to a given hash? Has it shown up in my network logs?
• Given a domain flagged by my IDS, is it a flagrant false positive based on its popularity and malicious observations recorded by VirusTotal?
• Given an IP address that matched my threat feeds, has it been seen serving malware? If so, which hashes? Have those been seen across my fleet of machines?
• Once my EDR reveals a compromise, is it a well known threat to the industry? i.e. is it widely detected? Is it rather a targeted attack?

To answer these and other questions many companies have implemented a bring-your-own-api-key model where their end-users plug their VirusTotal API key into their products. Sightings recorded in those products then get automatically enriched via such key. While this is theoretically OK, it has resulted in poor and weak integrations that:

• Are extremely basic and often just display detection ratios, which is not only on the verge of compliance but is pretty useless given today’s false positive and false negative rates.
• Fail to display the wealth of contextual information that VirusTotal records: C2s and network traffic, delivery mechanisms, relationships with other artifacts, submission and in-the-wild metadata, crowdsourced detections via {YARA, SIGMA, IDS} rules, etc.
• Do not evolve as VirusTotal itself improves. Whenever we incorporate new data points or release new features, these rarely show up in those integrations. Moreover, for them to show up the integrator must invest engineering resources to update the logic.
• Miss the opportunity to create a single product experience where common customers can easily pivot from the 3rd-party product into VirusTotal to conduct deeper investigations.

Not everything is lost. We are introducing VT AUGMENT, an HTML widget that greatly reduces the heavy lifting required to display VirusTotal context in 3rd-party products:

• It can enrich the most common threat observables: files/hashes, domains, IPs and URLs.
• You do not need to parse complex API response objects and build fancy templates, VirusTotal directly serves a report with all the context that we have for the observable.
• The report can be styled to match your interface.
• VirusTotal seamlessly adds new features and data points to the widget, without requiring engineering work on your side to update.
• It allows you to display all VirusTotal details, not just a subset of them. Moreover, it is not constrained to an analysis data dump, it also displays our threat graph for the given observable and any related IoCs.

All the details displayed in the report are pivotable, meaning that your users can search for similar files, jump to other files communicating with the same domain, discover other malware signed with the same authenticode certificate, etc. with a single click.



Most importantly, VT AUGMENT technically ensures compliance with our terms. Since we are not exposing parseable API fields, there is no room for backend black magic to drive detections, perform machine learning, copy signatures or other non-compliant use cases that would go against the VirusTotal ecosystem.

You can see VT AUGMENT in action in the following demo environment that simulates a SIEM alerts dashboard, click on the VirusTotal icon next to each observable to display the VT AUGMENT widget:
https://www.virustotal.com/ui/widget/demo/dedicated?full=1

Please note that VT AUGMENT still requires you to implement a bring-your-own-api-key model where your end-users plug their API key into your product. This said, we are also open to consider integrations driven by a single integrator key, always with prior consent from VirusTotal.



You can dive into the specifics surrounding the VT AUGMENT integration in our API reference:
https://developers.virustotal.com/v3.0/reference#widget-overview

A standard VirusTotal API key will be enough to test the flow, but remember that the final setup must make use of each of your users’ API keys, unless you have explicit permission from VirusTotal. Additionally, note that we have also published a handy javascript client library to further ease the task of displaying the widget report in your own interface:
https://github.com/VirusTotal/vt-augment

If you are interested in integrating VT AUGMENT and require more API quota to develop and test the setup, need some help with the technical implementation or want to discuss partnership opportunities involving this new widget please do not hesitate to contact us.

We will be creating a specific section in our main VirusTotal website to document and showcase these integrations. Similarly, we will be showing some gratitude to the first adopters by featuring their integrations in an upcoming blog post series on this topic, stay tuned!

Read More

Context is king (part I) - Crowdsourced Sigma rules

In our previous blog post we started discussing how important it is to have relevant context when doing any investigation and how at VirusTotal, we are working hard to provide as much context as possible. Indeed, there are many new features we have already implemented and that we want to share with all of you. Today we will discuss Crowdsourced Sigma rules.

What are Sigma rules? Probably at this point you are already familiar with YARA: in essence, a rule-based engine to detect certain patterns in files. YARA became a de-facto standard in Threat Intelligence sharing, widely used for static detection, attribution, monitoring and hunting.

With this same idea in mind, Sigma was developed as a “YARA for logs”, allowing the creation of generic rules that could be later used in most SIEMs. Given Sigma rules match against System event logs, one of the main differences with YARA is that rules will be behaviour-based instead of matching static patterns in files.

Now, at VirusTotal our sandboxes store all event logs during detonation, which are later used to match Crowdosourced Sigma rules. In particular, we are importing rules from the following public repositories (big thanks to all of them for their help):

•         https://github.com/SigmaHQ/sigma
•         https://github.com/joesecurity/sigma-rules
•         https://sigma.socprime.com/#!/

If you are curious, you can even check the full list of rules and the number of matches for each of them in our documentation.

Relevant additional context for file reports

Sigma matches help researchers and investigators get more context about a given file. It is also an additional and quick way of finding potentially related files based on the same behaviour. Similar to Crowdsourced YARA rules, VirusTotal Intelligence users will find the list of Sigma rules matching a given file in the Detection tab:

        

From there you can View the rule itself, check what events made this file to match this rule in particular, as well as finding other files matching this Sigma rule. Remember that for finding all the files matching a given Sigma rule you can always use the “sigma_rule” modifier followed by its ID as shown in the Sigma rules list documentation (or by simply clicking on them).

For example, the following search returns all matches for the “Milum malware detection” Sigma rule (based on WildPressure APT)”:

        sigma_rule:30fcf3924a898a9d1747e89068ab2990c77ca3914a94aa78466d28a9d9da32bb

When opening some of the search results in a Graph, we can see there are relationships among them:

Keep in mind that you can add more search modifiers to your Sigma rule search. For instance, we can add to our previous rule a filter to get the results that also matched any Crowdsourced YARA rule, probably this will help us to quickly identify the subset of results we are interested in as well as serve as a double check:  

        sigma_rule:30fcf3924a898a9d1747e89068ab2990c77ca3914a94aa78466d28a9d9da32bb have:crowdsourced_yara_rule

From here we can further pivot and select just the subset that additionally matches a particular set of YARA rules. All this context helps us to save time and adds a very relevant source of information when determining a file’s maliciousness.

Context for detection

Sigma matches can be used as a nice addition to AV detections. For instance, we can quickly find undetected files having Sigma matches:

        have:sigma_rule p:0

It is also possible to filter by rule severity by using the sigma_<severity> search operator. To search undetected files having no detections and matching high severity rules we can do:

        p:0 sigma_high:1+

Since the sigma_<severity> operator accepts integer values, it is possible to set a minimum severity for the matching rules. The aforementioned previous search query will find files matching Sigma rules with at least severity 1 or higher.

For instance, as a result of searching “p:0 sigma_high:5+”, the following file is returned:

Even though it has no AV detections, its bundled files have:

Fooled by context?

A final thought about adding more context.

We believe the more relevant information we have around any set of activities or IOC, the better. However, sometimes different pieces of information can contradict each other: what to do then? It really depends, the previous example is a good one in this direction as it shows how some malware was not detected given some technicality (was packing other malware inside) but when run in the sandbox the Sigma rule fired up. 

We do a great effort in trying to keep all the crowdsourced elements we add as relevant as possible, carefully selecting the sources. But there might always be clashes, especially when it comes to aspects such as attribution where different security companies might have different mappings for APT groups.

Still, we find all these additions of great value for Threat Intelligence practitioners. As we all know, Intelligence is not bought but produced: we keep working hard to provide you with all the relevant signals you can use to make your own informed decisions.

Read More