It's all about the structure! Creating YARA rules by clicking

Since we made our (extended) vt module available for LiveHunt YARA rules we understand it is not easy for analysts to keep in mind all the new potential possibilities - too many of them! Our goal is to make YARA rule creation as easy as possible while providing security experts everything they need to make even more powerful rules. Our recently published new YARA editor, which incorporates full syntax coloring and auto-complete while you develop your rule, is a first step.

However, we wanted to go further. We already discussed how you can use predefined templates (additionally you can check our Threat Hunting with VirusTotal - Episode 4 for further examples and ideas), but in this post we want to focus on a terrific new feature when creating rules using the “Structure” of any given object (file, URL, domain or IP).

“Structure” provides the full JSON containing all details VirusTotal knows for any given indicator. For instance, you can paste a file hash and you will get full details about its behaviour and metadata. What is better, you can simply click on any field you are interested in, and it will automatically included in a fresh new YARA rule in the editor - no need to remember how to get that particular field in the VT module anymore.

In case you are wondering, this also deals with all kinds of loops. If any of the selected fields needs to be iterated, the correct syntax will automatically be added to your rule.

Let’s check the different object types.

Files

For a file object you will find two different branches in the resulting JSON - behaviour and metadata.

The behaviour key is based on the sample execution in the sandbox. For example, you can create rules based on files written by the malware, files dropped, mutexes created, processes created, sigma results or ATT&CK MITRE results, among others.

Let’s suppose that we are interested in creating a new detection logic focused on some specific file written. In that case, we want to open the files_written section and then click on the file that we have observed as suspicious for our rule. Automatically, a new rule pops up with that condition (note that the loop condition was conveniently created for us too).

We can keep editing the rule to adapt it to our needs, like adding additional conditions to detect a specific string or path, another file name, etc.

If your security posture takes into account the ATT&CK MITRE matrix, maybe you want to create rules adding these fields in your logic, available under the key mitre_attack_techniques branch.

In addition to vt.behaviour, it is also possible to use vt.metadata to create a rule based on file metadata. Under the metadata key, we have a lot of interesting information that we can use to create our rule.

Probably one of the most interesting fields is "itw". Under this key, we can create rules based on ITW communications that we are interested in detecting whether related to IPs, domains or URLs.

For example, we may be interested in files that were downloaded ITW with response code 200, from the Discord CDN and that download binaries but more specifically DLLs.

Another interesting approach could be to hunt for files that are downloaded ITW, but with characteristics that could interest us in the whois of the domain from which it was downloaded. This could be interesting if we are monitoring certain domains that are being registered.

Metadata gives us multiple ways to play to create livehunt rules. From more complex rules using ITW applying filters related to domains, IPS or URLs to more basic things where we can include information from exiftool, submitters, fuzzy hashing, etc.

Combining the power of metadata and behaviour will result in a quality YARA rule!

URL

For URLs, under the “net” section in the VT module, you have the possibility to use the keys url, ip and domain as shown on the Netloc summary table. Any field available under these keys can be used to create your URL hunting rule.

Some of the features you can use to create your rule include URL response headers, downloaded and communicating files, URL path, domain whois, IP ASN, among others. Just by clicking on the fields you are interested in and adapting them to your needs, you can create a robust rule that helps you follow a campaign you are interested in investigating.

A use case could be that we were interested in discovering new URLs seen in VirusTotal, where the path meets a certain pattern, resolves to a certain network block and the domain registry is a registry known as commonly used to register malicious domains. Finally, to avoid noise we are interested just in new URLs.

Last rule can match for example an URL used by Gamaredon threat actor.

IP

The fields available for the IP entity can be found under the the ip key in VT.net. Here you can play with fields such as IP whois, communicating files, netblocks and others.

From here, we can add as much information as we are interested in to identify new ip addresses from suspicious campaigns. The following image is the result of a few clicks on fields containing a specific IP address.

Let’s suppose we want to identify new IP addresses that belong to a certain ASN (here we explain how to calculate a network range) and have some type of communication with PEEXE binaries.

This type of use case could even be used to monitor certain network ranges that may belong to our organization or customers to identify if a new IP address has any files that carry out communications.

Domain

Last but not least, we can also use the new Structure functionality with domains. In this case, domains include information about both the domain itself and the IP address it resolves.

And the same process that we have followed with the other entities that we have taken as an example, it would only be enough to click on the fields that interest us and shape our rule.

Within the information that we can find within the domains, there is an interesting field called categories. Within these categories we can identify if the domain could be linked to malware, phishing, spyware...

To create a use case with this field, let's say that we want to discover new domains that are related to phishing, and that the value of the not_before field of the HTTPS certificate is greater than a specific date that we want to search for information.

Another case that we can do also related to phishing is to monitor a specific favicon that is using our brand image. Subsequently, we are also interested in whether it includes a pattern in the domain name or in the alternative name in the certificate.

Wrapping up

At VirusTotal we continue trying to include the greatest number of functionalities that are useful for analysts for threat hunting. Our goal is to make work easier and spend time intelligently when using the platform.

The idea of this new feature is to continue to add new fields that can be consumed through VirusTotal intelligence to make livehunt rule creation more powerful. It is not easy to remember or know which fields are available within the files to create livehunt rules, so the new "Structure" functionality can help us.

We want livehunt rules to be a great tool to detect campaign patterns and to be able to track players more powerfully.

We would also like to announce that we have opened a GitHub where the community can publish their YARA rules and contribute! During the following weeks we will be posting new rules https://github.com/VirusTotal/vt-public-crowdsourced-yara.

We hope you liked this functionality. Happy hunting!

Read More

Threat hunting converting SIGMA to YARA

Malware threat hunting is the process of proactively searching for malicious activity. It is a critical part of any organization's security posture, as it can help to identify and mitigate threats that may have otherwise gone undetected.

Sigma rules and YARA rules are two powerful tools that can be used for detection and  malware threat hunting. Sigma rules are a type of open rule language that can be used to describe malicious activity.  

Many sigma rules can be converted into yara rules for use with the VT yara module to match data from our inhouse and external sandboxes and behavioral engines.  You can then use the VirusTotal IOC Stream ,  to view the YARA matches on new file analysis. 

Below are some examples of how to convert from SIGMA to YARA:

Example 1: Matching processes

Consider Sigma rule to detect base64 decode.

title: Decode Base64 Encoded Text -MacOs
id: 719c22d7-c11a-4f2c-93a6-2cfdd5412f68
status: test
description: Detects usage of base64 utility to decode arbitrary base64-encoded text
references:
    - https://github.com/redcanaryco/atomic-red-team/blob/f339e7da7d05f6057fdfcdd3742bfcf365fee2a9/atomics/T1027/T1027.md
author: Daniil Yugoslavskiy, oscd.community
date: 2020/10/19
modified: 2022/11/26
tags:
    - attack.defense_evasion
    - attack.t1027
logsource:
    category: process_creation
    product: macos
detection:
    selection:
        Image: '/usr/bin/base64'
        CommandLine|contains: '-d'
    condition: selection
falsepositives:
    - Legitimate activities
level: low

  

The sigma rule can be translated to a Yara rule similar to:

import "vt"

rule base64decode
{
  meta:
    sigma_source = "https://github.com/SigmaHQ/sigma/blob/master/rules/macos/process_creation/proc_creation_macos_base64_decode.yml"
    example1 = "f3e5c20b34731d6611e1a49def1c89d5c180db9bb465f8471ba84c1ad16b90e5"
    example2 = "ea502018cb3eeb56a930df29c7447857c6cca05d3431d2f575d2c62753bb81f1"
  condition:
    for any cmd in vt.behaviour.command_executions : (
        cmd icontains "base64 " and cmd icontains " -d"
    )
}  
  

Remember to test your rule to ensure it matches the desired samples.

Example 2: Matching DNS

In this example, we will generate YARA matches that produce similar results to the VirusTotal Intelligence query, with a search modifier.

• sigma_rule:"DNS Query To Remote Access Software".

Sigma rule from SigmaHQ to dectect common remote access domains:

title: DNS Query To Remote Access Software Domain
id: 4d07b1f4-cb00-4470-b9f8-b0191d48ff52
related:
- id: 71ba22cb-8a01-42e2-a6dd-5bf9b547498f
type: obsoletes
- id: 7c4cf8e0-1362-48b2-a512-b606d2065d7d
type: obsoletes
- id: ed785237-70fa-46f3-83b6-d264d1dc6eb4
type: obsoletes
status: experimental
description: |
An adversary may use legitimate desktop support and remote access software, such as Team Viewer, Go2Assist, LogMein, AmmyyAdmin, etc, to establish an interactive command and control channel to target systems within networks.
These services are commonly used as legitimate technical support software, and may be allowed by application control within a target environment.
Remote access tools like VNC, Ammyy, and Teamviewer are used frequently when compared with other legitimate software commonly used by adversaries. (Citation: Symantec Living off the Land)
references:
- https://github.com/redcanaryco/atomic-red-team/blob/f339e7da7d05f6057fdfcdd3742bfcf365fee2a9/atomics/T1219/T1219.md#atomic-test-4---gotoassist-files-detected-test-on-windows
- https://github.com/redcanaryco/atomic-red-team/blob/f339e7da7d05f6057fdfcdd3742bfcf365fee2a9/atomics/T1219/T1219.md#atomic-test-3---logmein-files-detected-test-on-windows
- https://github.com/redcanaryco/atomic-red-team/blob/f339e7da7d05f6057fdfcdd3742bfcf365fee2a9/atomics/T1219/T1219.md#atomic-test-6---ammyy-admin-software-execution
- https://redcanary.com/blog/misbehaving-rats/
author: frack113, Connor Martin
date: 2022/07/11
modified: 2023/04/18
tags:
- attack.command_and_control
- attack.t1219
logsource:
product: windows
category: dns_query
detection:
selection:
QueryName|endswith:
- '.getgo.com'
- '.logmein.com'
- '.ammyy.com'
- '.netsupportsoftware.com' # For NetSupport Manager RAT
- 'remoteutilities.com' # Usage of Remote Utilities RAT
- '.net.anydesk.com'
- 'api.playanext.com'
- '.relay.splashtop.com'
- '.api.splashtop.com'
- 'app.atera.com'
- '.agentreporting.atera.com'
- '.pubsub.atera.com'
- 'logmeincdn.http.internapcdn.net'
- 'logmein-gateway.com'
- 'client.teamviewer.com'
  

The above sigma signature can be expressed as a Yara rule:

import "vt"
rule dns_remote_access
{
meta:
sigma_src = "https://github.com/SigmaHQ/sigma/blob/c05f864047ffbe793299499c79ec52920062159f/rules/windows/dns_query/dns_query_win_remote_access_software_domains.yml#L4"
condition:
  for any lookup in vt.behaviour.dns_lookups : (
    for any host in (".getgo.com",".logmein.com",".ammyy.com",".netsupportsoftware.com","remoteutilities.com","net.anydesk.com","relay.splashtop.com","api.splashtop.com","app.atea.com","agentreporting.atera.com","pubsub.atera.com","http.internapcdn.ne","logmein-gateway.com","client.teamviewer.com") : (
     lookup.hostname contains host
     ))
}

Example 3: Matching registry keys set

In this example we will search registry keys set. Using VT Intelligence you can search for strings within registry keys or values with a query like: behaviour_registry:SystemRestore\DisableConfig"
Consider the sigma rule:

title: Registry Disable System Restore
id: 5de03871-5d46-4539-a82d-3aa992a69a83
status: experimental
description: Detects the modification of the registry to disable a system restore on the computer
references:
    - https://github.com/redcanaryco/atomic-red-team/blob/f339e7da7d05f6057fdfcdd3742bfcf365fee2a9/atomics/T1490/T1490.md#atomic-test-9---disable-system-restore-through-registry
author: frack113
date: 2022/04/04
modified: 2022/09/09
tags:
    - attack.impact
    - attack.t1490
logsource:
    category: registry_set
    product: windows
detection:
    selection:
        EventType: Setvalue
        TargetObject|contains:
            - '\Policies\Microsoft\Windows NT\SystemRestore'
            - '\Microsoft\Windows NT\CurrentVersion\SystemRestore'
        TargetObject|endswith:
            - DisableConfig
            - DisableSR
        Details: 'DWORD (0x00000001)'
    condition: selection
falsepositives:
    - Unknown
level: high
  

The sigma rule as yara:

import "vt"

rule disable_restore
{
  meta:
    sigma_source = "https://github.com/SigmaHQ/sigma/blob/62d4fd26b05f4d81973e7c8e80d7c1a0c6a29d0e/rules/windows/registry/registry_set/registry_set_disable_system_restore.yml#L2"
    example1 = "08c2d3fec8cd9fcced634df7ad0f3db164ffe0cbfc263e2d8be026afca05bfcb"
  condition:
    for any reg in vt.behaviour.registry_keys_set : (
        ( reg.key contains "\\Policies\\Microsoft\\Windows NT\\SystemRestore" 
         or reg.key contains "\\Microsoft\\Windows NT\\CurrentVersion\\SystemRestore" )
        and 
          (reg.key contains "DisableSR"  or reg.key contains "DisableConfig")
        and (reg.value contains "1")
    )
}

Test your rule to ensure it matches desired samples:

Example 4: Matching files on disk

A sima rule from SigmaHQ to detect linux samples modifying /etc/profile.d

title: Potentially Suspicious Shell Script Creation in Profile Folder
id: 13f08f54-e705-4498-91fd-cce9d9cee9f1
status: experimental
description: Detects the creation of shell scripts under the "profile.d" path.
references:
    - https://blogs.jpcert.or.jp/en/2023/05/gobrat.html
    - https://jstnk9.github.io/jstnk9/research/GobRAT-Malware/
    - https://www.virustotal.com/gui/file/60bcd645450e4c846238cf0e7226dc40c84c96eba99f6b2cffcd0ab4a391c8b3/detection
    - https://www.virustotal.com/gui/file/3e44c807a25a56f4068b5b8186eee5002eed6f26d665a8b791c472ad154585d1/detection
author: Joseliyo Sanchez, @Joseliyo_Jstnk
date: 2023/06/02
tags:
    - attack.persistence
logsource:
    product: linux
    category: file_event
detection:
    selection:
        TargetFilename|contains: '/etc/profile.d/'
        TargetFilename|endswith:
            - '.csh'
            - '.sh'
    condition: selection
falsepositives:
    - Legitimate shell scripts in the "profile.d" directory could be common in your environment. Apply additional filter accordingly via "image", by adding specific filenames you "trust" or by correlating it with other events.
    - Regular file creation during system update or software installation by the package manager
level: low # Can be increased to a higher level after some tuning
  

This could be searched with a VT intelligence query like: behaviour_files:"/etc/profile.d/" and (behaviour_files:".sh" or behaviour_files:*.csh) and (tag:elf or tag:shell)

import "vt"

rule suspicious_profile_folder
{
  meta:
    sigma_source = "https://github.com/SigmaHQ/sigma/blob/c04bef2fbbe8beff6c7620d5d7ea6872dbe7acba/rules/linux/file_event/file_event_lnx_susp_shell_script_under_profile_directory.yml"
    example_hash1 = "e15e93db3ce3a8a22adb4b18e0e37b93f39c495e4a97008f9b1a9a42e1fac2b0"
    example_hash2 = "447431333b2c2a72ac213a9fa2da8c2b09383ae75c3b31a88acfa79b8d43b8d8"
   author = "author: Joseliyo Sanchez, @Joseliyo_Jstnk"
  condition:
    for any dropped in vt.behaviour.files_dropped : (
      dropped.path contains "/etc/profile.d/"
      and (dropped.path endswith ".sh" or dropped.path endswith ".csh")
    )
    or
    for any file_path in vt.behaviour.files_written : (
      file_path contains "/etc/profile.d/"
      and (file_path endswith ".sh" or file_path endswith ".csh")
    )
}
  

As yara:

import "vt"

rule suspicious_profile_folder
{
  meta:
    sigma_source = "https://github.com/SigmaHQ/sigma/blob/c04bef2fbbe8beff6c7620d5d7ea6872dbe7acba/rules/linux/file_event/file_event_lnx_susp_shell_script_under_profile_directory.yml"
    example_hash1 = "e15e93db3ce3a8a22adb4b18e0e37b93f39c495e4a97008f9b1a9a42e1fac2b0"
    example_hash2 = "447431333b2c2a72ac213a9fa2da8c2b09383ae75c3b31a88acfa79b8d43b8d8"
   author = "author: Joseliyo Sanchez, @Joseliyo_Jstnk"
  condition:
    for any dropped in vt.behaviour.files_dropped : (
      dropped.path contains "/etc/profile.d/"
      and (dropped.path endswith ".sh" or dropped.path endswith ".csh")
    )
    or
    for any file_path in vt.behaviour.files_written : (
      file_path contains "/etc/profile.d/"
      and (file_path endswith ".sh" or file_path endswith ".csh")
    )
}

Summary of translating sigma to yara:

You may wish to review the sigma specification and review the sigma rules detected on VirusTotal for examples.

Any data contained in the file behavior object can be matched on.

The table below may help in guiding you to the correct keywords to use.

Sigma Taxonomy	VirusTotal schema
file_access TargetFilename file_event TargetFilename sysmon EventID 11 (FileCreate)	vt.behaviour.files_written[] vt.behaviour.files_dropped[].path
registry_set	vt.behaviour.registry_keys_set[].key vt.behaviour.registry_keys_set[].value
registry_delete	vt.behaviour.registry_keys_deleted[]
process_creation ps_script file_event Image	vt.behaviour.command_executions[]
network_connection DestinationHostname dns_query QueryName dns query	vt.behaviour.dns_lookups[].hostname vt.behaviour.tls[].sni vt.behaviour.memory_pattern_urls[] vt.behaviour.memory_pattern_domains[]
file_delete	vt.behaviour.files_deleted[]

Malware threat hunting can be complex, by using Sigma rules and YARA rules, you can make the process more efficient and effective. Happy hunting.

Read More

Test your YARA rules against a collection of goodware before releasing them in production

The rising tide of malware threats has created an arms race in security tool accumulation, this has led to alarm fatigue in terms of noisy alerts and false positives. The last thing you need is more false alarms coming from buggy or suboptimal YARA rules, be it the ones you use in VT Hunting or the ones that you feed into your own security defenses.

As you may already know, VT Enterprise incorporates a component that allows you to match your own YARA rules against all newly uploaded files (Livehunt) as well as back in time against our historical malware collection (Retrohunt).

A common challenge for YARA users is that of potential false positives. False positives can have a negative effect on a users Livehunt feed by producing incorrect results. Similarly, a buggy rule can be a waste of your Retrohunt quota, and given that Retrohunt jobs are lengthy, it is also a waste of time. Since many security tools incorporate YARA these days, some users will be launching their rules against a fleet of machines that they manage, meaning that a buggy rule can be a big waste of resources.

In order to address this common pain point we are releasing a new Retrohunt feature: fast hunting over a goodware corpus. When you launch your Retrohunt jobs you can now select the corpus on which it should act:



The goodware corpus is a set of 1M files chosen from the NIST National Software Reference Library, accounting for 147GB. Jobs launched against this collection usually finish in under a minute. As such, we imagine that users may be modifying the way they use VT Hunting. Before writing a Livehunt YARA rule or launching a Retrohunt job, they probably will want to test it against this corpus and tweak the rule in order to prevent false positives and avoid unnecessary and lengthy Retrohunt iterations.



- Goodware Retrohunt jobs are correspondingly tagged -

In an effort to give back to the community behind VirusTotal and its premium services, we are making this feature entirely free. In other words, Retrohunt jobs against the goodware corpus do not consume Retrohunt quota.

This new feature builds upon some major improvements that have been recently released such as the new API endpoints to programmatically interact with VT Hunting. Stay tuned, soon we will be announcing far bigger enhancements to Retrohunt, you can take a sneak peek in our 2019 roadmap (Lightning-fast retrohunt).

Read More