Virtual Appliance Scan Caching

Scan Caching is used by the Virtual Appliance to maximize the efficiency of Anti-Malware and Integrity Monitoring Scans of virtual machines. Scan Caching improves the efficiency of scans by eliminating the unnecessary scanning of identical content across multiple VMs in large VMware deployments. A Scan Cache contains lists of files and other scan targets that have been scanned by a Deep Security protection module. If a scan target on a virtual machine is determined to be identical to a target that has already been scanned, the Virtual Appliance will not scan the target a second time. Attributes used to determine whether entities are identical are creation time, modification time, file size, and file name. In the case of Real-time Scan Caching, Deep Security will read partial content of files to determine if two files are identical. There is an option setting to use a file's Update Sequence Number (USN, Windows only) but its use should be limited to cloned virtual machines.

Scan Caching benefits Integrity Monitoring by sharing Integrity Monitoring scan results among cloned or similar virtual machines.

Scan Caching benefits Manual Malware Scans of cloned or similar virtual machines by increasing the speed up subsequent scans.

Scan Caching benefits Real-Time Malware Scanning by speeding up boot process scans and application access scans on cloned or similar virtual machines.

Scan Cache Configurations

A Scan Cache Configuration is a collection of settings that determines Expiry Time, the use of Update Sequence Numbers (USNs), files to exclude, and files to include.

Virtual machines that use the same Scan Cache Configuration also share the same Scan Cache.

You can see the list of existing Scan Cache Configurations by going Administration > System Settings > Advanced>Scan Cache Configurations and clicking View Scan Cache Configurations . Deep Security comes with several preconfigured default Scan Cache Configurations. These are implemented automatically by the Virtual Appliance depending the properties of the virtual machines being protected and the types of scan being performed.

Expiry Time determines the lifetime of individual entries in a Scan Cache. The default recommended settings are one day for Manual (on-demand) or Scheduled Malware Scans, 15 mins for Real-Time Malware Scans, and one day for Integrity Monitoring Scans.

Use USN (Windows only) specifies whether to make use of Windows NTFS Update Sequence Numbers, which is a 64-bit number used to record changes to an individual file. This option should only be set for cloned VMs.

Files Included and Files Excluded are regular expression patterns and lists of files to be included in or excluded from the Scan Cache. Files to be scanned are matched against the include list first.

Individual files and folders can be identified by name or you can use wildcards ("*" and "?") to refer to multiple files and locations with a single expression. (Use "*" to represent any zero or more characters, and use question mark "?" to represent any single character.)

The include and exclude lists only determine whether the scan of the file will take advantage of Scan Caching. The lists will not prevent a file from being scanned in the traditional way.

Malware Scan Cache Configuration

To select which Scan Cache Configuration is used by a virtual machine, open the Computer or Policy editorClosedYou can change these settings for a policy or for a specific computer. To change the settings for a policy, go to the Polices page and double-click the policy that you want to edit (or select the policy and click Details). To change the settings for a computer, go to the Computers page and double-click the computer that you want to edit (or select the computer and click Details). and go to Anti-Malware > Advanced > VM Scan Cache. You can select which Scan Cache Configuration is used for Real-Time Malware Scans and which Scan Cache Configuration is used for manual and scheduled scans.

Integrity Monitoring Scan Cache Configuration

To select which Scan Cache Configuration is used by a virtual machine, open the Computer or Policy editorClosedYou can change these settings for a policy or for a specific computer. To change the settings for a policy, go to the Polices page and double-click the policy that you want to edit (or select the policy and click Details). To change the settings for a computer, go to the Computers page and double-click the computer that you want to edit (or select the computer and click Details). and go to Integrity Monitoring > Advanced > VM Scan Cache.

Scan Cache Settings

Scan Cache Settings are not included in a Scan Cache Configuration because they determine how the Virtual Appliance manages Scan Caches rather than how Scan Caching is carried out. Scan Cache settings are controlled at the Policy level. You can find the Scan cache settings by opening a Policy editorClosedTo open the Policy editor, go to the Policies page and double-click the policy that you want to edit (or select the policy and click Details). and going to the Settings > General > Virtual Appliance Scans area.

Max Concurrent Scans determines the number of scans that the Virtual Appliance performs at the same time. The recommended number is five. If you increase this number beyond 10, scan performance may degrade. Scan requests are queued by the virtual appliance and carried out in the order in which they arrive. This setting applies to manual and scheduled scans.

Max On-Demand Malware Scan Cache Entries determines, for manual or scheduled malware scans, the maximum number of records that identify and describe a file or other type of scannable content to keep. One million entries use approximately 100 MB of memory.

Max Malware Real-Time Scan Cache Entries determines, for real-time malware scans, the maximum number of records that identify and describe a file or other type of scannable content to keep. One million entries use approximately 100MB of memory.

Max Integrity Monitoring Scan Cache Entries determines the maximum number of entities included in the baseline data for integrity monitoring. Two hundred thousand entities use approximately 100MB of memory.

When to change the default configuration

Scan caching is designed to avoid scanning identical files twice. Deep Security does not examine the entire contents of all files to determine if files are identical. Although when configured to do so, Deep Security can check the USN value of a file, and during Real-time Scans it will read partial content of files, it generally examines file attributes to determine if files are identical. It would be difficult but not impossible for some malware to make changes to a file and then restore those files attributes to what they were before the file was modified.

Deep Security limits this potential vulnerability by establishing short default cache expiry times. To strengthen the security you can use shorter expiry times on cache and you can use USN but doing so may reduce the performance benefit or require a larger cache setting. For the strongest security for VMs that you want to keep separate and never share scan results you can create dedicated policies for these VMs kind of like keeping them in separate zones. This might be appropriate if you have different departments or organizations sharing the same infrastructure. (In a multi-tenant Deep Security Manager, this is automatically enforced for each tenant.)

If you have a very large number of guest VMs per ESXi host (for example, a VDI environment), then you should monitor your disk I/O and CPU usage during scanning. If scanning takes too long, then you may need to increase the size of the cache or adjust the Scan Cache Settings until you get better performance. If you need to increase cache size, then you may need to adjust Deep Security Virtual Appliance system memory too.