Set up a multi-tenant environment

The multi-tenancy feature in Deep Security lets you create separate management environments within a single Deep Security Manager. It allows tenants to each have their own settings and policies and to monitor their own events. This can be useful if you want to create separate staging and production environments or if you need to create separate environments for different business units in your organization. You can also use multi-tenancy to provision Deep Security to customers in a service model.

Once you enable multi-tenancy, you (as the "primary tenant") retain all of the capabilities of a regular installation of Deep Security Manager. However, the tenants you subsequently create can have their access to Deep Security functionality restricted to varying degrees, based on how you configure the system for them.

In this topic:

• Multi-tenancy requirements
• Enable multi-tenancy
• Create a tenant
• Scalability guidelines
• Multi-tenancy tips
• Managing tenants
• Set up a multi-tenant environment
• Usage monitoring
• Configure database user accounts
• APIs
• Upgrade
• Supporting tenants
• Load balancers

Multi-tenancy requirements

You cannot set up multi-tenancy with:

• Any other license options for Deep Security from AWS Marketplace
• Deep Security Manager VM for Azure Marketplace
• Deep Security as a Service

You will need a separate activation code for multi-tenancy, and multi-tenancy has some extra database requirements in addition to the usual Deep Security Manager requirements. For details, see Prepare a database for Deep Security Manager on AWS and Configure database user accounts.

To maximize scalability, we recommend that you use a multi-node Deep Security Manager (see Run Deep Security Manager as multiple nodes). All manager nodes process GUI, heartbeat, or job requests for any tenant. For background processing, each tenant is assigned a manager node that takes care of job queuing, maintenance, and other background tasks. Tasks are rebalanced across remaining nodes when manager nodes are added or taken offline.

When you enable multi-tenancy, your current installation of Deep Security Manager becomes the primary tenant (t0) and has special privileges, including the ability to create tenants. When you create tenants, they will be restricted from using certain features and will not see the UI for those features in Deep Security Manager. For example, tenants cannot create other tenants. For details, see Set up a multi-tenant environment

Enable multi-tenancy

1. In the Deep Security Manager, go to Administration > System Settings > Advanced. In the Multi-Tenant Options area, click Enable Multi-Tenancy.
   
2. The Multi-Tenant Configuration wizard appears. Enter your multi-tenancy activation code and click Next.
3. Choose the license mode that you want to use:
   • Inherit Licensing from Primary Tenant: This option gives all tenants the same licenses that you (the primary tenant) have. This option is recommended if you are using multi-tenancy in a staging environment, or if you intend to set up tenancies for separate departments within your organization.
   • Per Tenant Licensing: This option is recommended if you are offering Deep Security as a service. With this configuration, you can use the Deep Security API to provide a license when you create a tenant, or the tenant can enter a license when they sign in to the Deep Security Manager for the first time.
4. Click Next. When the wizard closes, you’ll be able to see a new Administration > System Settings > Tenants page, where you can configure multi-tenancy options. For information about the options on that page, click Help in the upper-right corner of Deep Security Manager.

Create a tenant

Once multi-tenant mode is enabled, Tenants can be managed from the Tenants page that now appears in the Administration section.

For information about the database user account permissions that are required for adding tenants, see Configure database user accounts.

1. In the Deep Security Manager, go to Administration > Tenants and click New.
2. The New Tenant wizard appears. Enter a Tenant Account Name. The account name can be anything except "Primary", which is reserved for the primary tenant.
3. Enter an email address that will be used as a point of contact for the tenant.
4. Select the Locale. The Locale determines the language of the Deep Security Manager user interface for the tenant.
5. Select a Time Zone. All tenant-related events will be shown to the tenant users in the time zone that you specify here.
6. If your Deep Security installation is using more than one database, you will have the option to let Deep Security automatically select a database server on which to store the new tenant account ("Automatic -- No Preference") or you can specify a particular server.
   This option will not appear if you have only one database. Database servers that are no longer accepting new tenants will not appear in the list.
7. Enter a user name for the first user of the new tenant account.
8. Select one of the three password options:
   • No Email: The tenant’s first user's user name and password are defined here and no emails are sent.
   • Email Confirmation Link: You set the tenant’s first user's password. However, the account is not active until the user clicks a link in a confirmation email that will be sent.
   • Email Generated Password: This allows you to generate a tenant without specifying the password.

   All three options are available via the REST API. The confirmation option provides a suitable method for developing public registration. A CAPTCHA is recommended to ensure that the tenant creator is a human not an automated bot.

   The email confirmation ensures that the email provided belongs to the user before they can access the account.

9. Click Next to finish with the wizard and create the tenant.

Tenant creation can take up to four minutes due to the creation of the schema and the population of the initial data. This ensures each new tenant has the most up- to-date configuration and removes the burden of managing database templates, especially between multiple database servers.

Each tenant database has an overhead of around 100 MB of disk space (due to the initial rules, policies and events that populate the system).

Examples of messages sent to tenants

Email Confirmation Link: Account Confirmation Request

Welcome to Deep Security! To begin using your account, click the following confirmation URL. You can then access the console using your chosen password.
Account Name: AnyCo
User name: admin
Click the following URL to activate your account:
https://managerIP:portnumber/SignIn.screen?confirmation=1A16EC7A-D84F-D451-05F6-706095B6F646&tenantAccount=AnyCo&username=admin

Email Generated Password

First email : Account and Username Notification

Welcome to Deep Security! A new account has been created for you. Your password will be generated and provided in a separate email.

Account Name: AnyCo
Username: admin
You can access Deep Security using the following URL:
https://managerIP:portnumber/SignIn.screen? tenantAccount=AnyCo&username=admin

Second email: Password Notification

This is the automatically generated password for your Deep Security account. Your Account Name, Username, and a link to access Deep Security will follow in a separate email.

Password: z3IgRUQ0jaFi

Scalability guidelines

Deployments of 50-100 tenants or more should follow these guidelines to avoid scalability issues:

• Create a maximum of 2000 tenants for a set of Deep Security Manager nodes
• Create a maximum of 500 tenants on a single database server
• Use a separate database server for the primary tenant, with no other tenants
• Limit the number of agents per tenant to 3000
• Limit the number of total agents to 20000
• Use a maximum of 5 Deep Security Manager nodes
• Do not use any co-located relays

Multi-tenancy tips

Reconnaissance IP list

In a multi-tenant environment, tenants may need to add the Deep Security Manager IP address to the “Ignore Reconnaissance IP” list found in Policies > Common Objects > Lists > IP Lists. This is to avoid getting a "Reconnaissance Detected: Network or Port Scan” warning.

Use multiple database servers

Multi-tenancy relies on using multiple databases (if you are using Microsoft SQL) or multiple users (if you are using Oracle). To scale further, you can connect Deep Security Manager to multiple database servers and automatically distribute the new tenants across the available set of database servers. See Configure database user accounts.

Tenant pending deletion state

Tenants can be deleted but the process is not immediate. Deep Security ensures that all the tenant-related jobs are finished before the records are deleted. The longest job runs every week, so the tenant will be in the “pending deletion” state for approximately seven days before the database is removed.

Multi-tenant options under System Settings

Consider these options on the Administration > System Settings > Tenants page:

Allow Tenants to use the Relays in my "Default Relay Group" (for unassigned Relays): Gives tenants automatic access to relay-enabled agents set up in the primary tenant. This saves tenants the effort of setting up dedicated relay-enabled agents for security updates.

Allow Tenants to use the "Backup" Scheduled Task: In most cases, backups should be managed by the database administrator and this option should be left unchecked.

Allow Tenants to use the “Run Script” Scheduled task: Scripts present a potentially dangerous level of access to the system; however, the risk can be mitigated because scripts have to be installed on the Deep Security Manager using file-system access.

Managing tenants

The Tenants page (Administration > Tenants) displays the list of all tenants. A tenant can be in any of the following States:

• Created: Created, but activation email has not been sent to the tenant user.
• Confirmation Required: Created, but the activation link in the confirmation email sent to the tenant user has not been clicked. (You can manually override this state.)
• Active: Fully online and managed.
• Suspended: No longer accepting sign-ins.
• Pending Deletion: Tenants can be deleted, however the process is not immediate. The tenant will be in the pending deletion state for approximately seven days before the database is removed.
• Database Upgrade Failed: For tenants that failed the upgrade path. The Database Upgrade button can be used to resolve this situation.

Tenant Properties

Double-click on a tenant to view the tenant's Properties window.

General

The Locale, Time zone and State of the tenant can be altered. Be aware that changing the time zone and locale does not affect existing tenant users. It will only affect new Users in that Tenancy and Events and other parts of the UI that are not user-specific.

The Database Name indicates the name of the database used by this tenancy. The properties of the tenant database can be accessed via the hyperlink.

Modules

The Modules tab provides options for protection module visibility. The selected visibility can be used to tune which modules are visible for which tenants. By default all unlicensed modules are hidden. You can change this by deselecting Always Hide Unlicensed Modules. Alternatively, selected modules can be shown on a per-tenant basis.

If you select Inherit License from Primary Tenant, all features that you (the primary tenant) are licensed for will be visible to all tenants. This means that even if you deselect Always Hide Unlicensed Modules all unlicensed modules will be hidden if you select this inheritance option.

If you are using the "per tenant" licensing, only the licensed modules for each tenant will be visible by default.

If you are evaluating Deep Security in a test environment and want to see what a full multi-tenancy installation looks like, you can enable "Multi-Tenancy Demo Mode". When in Demo Mode, the manager populates its database with simulated tenants, computers, Events, Alerts, and other data. Initially, seven days worth of data is generated but new data is generated on an ongoing basis to keep the manager's Dashboard, Reports and Events pages populated with data.

Statistics

The Statistics tab shows information for the current tenant including database size, jobs processed, logins, security events and system events. The spark line show the last 24 hours at a glance.

Agent Activation

The Agent Activation tab displays a command that can be run from the agent install directory of this tenant's computers which will activate the agent on the computer so that the tenant can assign policies and perform other configuration procedures from the Deep Security Manager.

What does the tenant see?

When Multi-tenancy is enabled, the sign-in page has an additional Account Name text field.

Tenants are required to enter their account name in addition to their user name and password. The account name allows tenants to have overlapping user names. For example, if multiple tenants synchronize with the same Active Directory server.

Some features in the Deep Security Manager UI are not available to tenant users. The following areas are hidden for tenants:

• Manager Nodes Widget
• Multi-Tenant Widgets
• Administration > System Information
• Administration > Licenses (If Inherit option selected)
• Administration > Manager Nodes
• Administration > Tenants
• Administration > System Settings:
  • Tenant Tab
  • Security Tab > Sign In Message
  • Updates Tab > Setting for Allowing Tenants to use Relays from the Primary Tenant
  • Advanced Tab > Load Balancers
  • Advanced Tab > Pluggable Section
• Some of the help content not applicable to tenants
• Some reports not applicable to tenants
• Other features based on the Multi-Tenant Options (discussed later)
• Some Alert Types will also be hidden from tenants:
  • Heartbeat Server Failed
  • Low Disk Space
  • Manager Offline
  • Manager Time Out Of Sync
  • Newer Version of Deep Security Manager available
  • Number of Computers Exceeds Database Limit
  • And when inherited licensing is enabled any of the license-related alerts

It is also important to note that tenants cannot see any of the multi-tenant features of the primary tenant or any data from any other tenant. In addition, certain APIs are restricted since they are only usable with primary tenant rights (such as creating other tenants).

For more information on what is and is not available to tenant users, see Multi-tenant settings.

All tenants have the ability to use Role-Based Access Control with multiple user accounts to further sub-divide access. Additionally, they can use Active Directory integration for users to delegate the authentication to the domain. The Tenant Account Name is still required for any tenant authentications.

Agent-Initiated Activation

Agent-initiated activation is enabled by default for all tenants.

Tenants can see the arguments required for agent-initiated activation by clicking Administration > Updates > Software > Local, selecting the agent software and then clicking the Generate Deployment Scripts button. For example, the script for Agent-Initiated Activation on a Windows machine might look like this:

dsa_control -a dsm://<host or IP>:4120/ "tenantID:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" "token:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"

Tenant diagnostics

Tenants are not able to access manager diagnostic packages due to the sensitivity of the data contained within the packages. Tenants can still generate agent diagnostics by opening the Computer Editor and choosing Agent Diagnostics on the Actions tab of the Overview page.

Usage monitoring

Deep Security Manager records data about tenant usage. This information is displayed in the Tenant Protection Activity widget on the Dashboard, the Tenant Properties window's Statistics tab, and reports. This information can also be accessed through the Status Monitoring REST API, which can be enabled or disabled by going to Administration > System Settings > Advanced > Status Monitoring API.

Use the Status Monitoring REST API to customize the type of tenant information that you would like to see, depending on your environment. For enterprises, this can be useful to determine the usage by each business unit. You can also use the information to monitor the usage of the overall Deep Security system and look for indicators of abnormal activity. For example, if a single tenant experiences a spike in security event activity, it might be under attack.

Multi-tenant Dashboard

When multi-tenancy is enabled, primary tenant users have access to the following additional Dashboard widgets for monitoring tenant activity:

• Tenant Database Usage
• Tenant Job Activity
• Tenant Protection Activity
• Tenant Security Event Activity
• Tenant Sign-In Activity
• Tenant System Event Activity
• Tenants

The same information is available on the Administration > Tenants page (some in optional columns) and on the Statistics tab of a tenant's Properties window.

This information provides the ability to monitor the usage of the overall system and look for indicators of abnormal activity. For example, if a single tenant experiences a spike in Security Event Activity, they might be under attack.

Multi-tenant reports

To generate reports that contain the information you require, go to Event & Reports > Generate Reports and choose the report you'd like to generate from the drop-down menu. The following are reports for multi-tenant environments, and the information they include:

Security Module Usage Cumulative Report

• Tenant
• Hostname
• ID
• Anti-Malware hours
• Network hours
• System hours
• SAP hours
• Enterprise hours

Security Module Usage Report

• Tenant
• ID
• Hostname
• Display name
• Computer group
• Instance type
• Start date
• Start time
• Stop time
• Duration (seconds)
• Anti-malware
• Web Reputation
• Firewall
• Intrusion prevention
• Integrity monitoring
• Log Inspection
• Application Control
• SAP

Tenant Report

• Tenant name
• Database size
• Peak host count
• Protection hours
• Percentage of protected hours

Configure database user accounts

The majority of each tenant's data is stored in a separate database. This database can co-exist on the same database server as other tenants, or it can be isolated onto its own database server. In all cases, some data only exists in the primary database (the one installed with Deep Security Manager). When multiple database servers are available, tenants are created on the database with the least amount of load.

The segmentation of each tenant's data into a database provides additional benefits:

• Data destruction: Deleting a tenant removes all traces of that tenant's data (supported in the product).
• Backup: Each tenant's data can be subject to different backup policies. This can be useful for something like tenancy being used for staging and production where the staging environment requires less stringent backups (backups are the responsibility of the administrator setting up Deep Security Manager).
• Balancing: The potential for future re-balancing to maintain an even load on all database servers.

Configuring database user accounts

See also Configure database user accounts.

SQL Server

Since multi-tenancy requires the ability for the software to create databases, the dbcreator role is required on SQL Server:

For the user role of the primary tenant, it is important to assign DB owner to the main database:

If desired, you can further refine the rights to include only the ability to modify the schema and access the data.

With the dbcreator role, the databases created by the account will automatically be owned by the same user. For example, here are the properties for the user after the first tenant has been created:

To create the first account on a secondary database server, only the dbcreator server role is required. No user mapping has to be defined.

Oracle

Multi-tenancy in Oracle is similar to SQL Server but with a few important differences. Where SQL Server has a single user account per database server, Oracle uses one user account per tenant. The user that Deep Security was installed with maps to the primary tenant. That user can be granted permission to allocate additional users and tablespaces.

If multi-tenancy is enabled, the following Oracle permissions must be assigned:

Tenants are created as users with long random passwords and given the following rights:

For secondary Oracle servers, the first user account (a bootstrap user account) must be created. This user will have an essentially empty tablespace. The configuration is identical to the primary user account.

PostgreSQL

The user must have the right to create new databases and roles:

ALTER ROLE [username] CREATEDB CREATEROLE;

On a secondary database server, the hostname, username, and password are required. The username must have privileges to create additional users (roles) and databases.

Configuring multiple database servers

By default, all tenants are created on the same database server that Deep Security Manager was installed with. In order to provide additional scalability, Deep Security Manager supports adding additional database servers (sometimes referred to as a secondary database). When you add a tenant, you will have the option to let Deep Security automatically select a database server on which to store the new tenant account or you can specify a particular server.

To configure additional databases, go to Administration > System Settings > Tenants and then click View Database Servers in the Database Servers section. Click New to add a database server.

For SQL Server, the secondary database server requires a hostname, user name and password (named instance and domain). The user (the Deep Security Manager) must have the following permissions:

• Create databases
• Delete databases
• Define schema

This account is used not only to create the database but to authenticate to the databases that are created.

Oracle multi-tenant uses a different model. The new database definition defines a user that is bound to a tablespace. That user is used to "bootstrap" the creation of additional users on Oracle.

Removing or changing secondary databases

You can delete database servers (other than the primary database) if there are no tenants on the server.

If the hostname, user name, password or any details change for a secondary server, you can change these values in the Deep Security Manager console. To change values for the primary database, you must shut down all nodes of the Deep Security Manager and edit the dsm.properties file with the new details.

APIs

Deep Security Manager includes a number of REST APIs for:

1. Enabling Multi-Tenancy
2. Managing Tenants
3. Accessing Monitoring Data
4. Accessing Chargeback (Protection Activity) Data
5. Managing Secondary Database Servers

In addition, the legacy SOAP API includes a new authenticate method that accepts the Tenant Account Name as a third parameter.

For more information on the REST APIs please see How to use the Deep Security REST API.

Upgrade

Upgrade is unchanged from previous versions. The installer is executed and detects an existing installation. It will offer an upgrade option. If upgrade is selected, the installer first informs other nodes to shutdown and then begins the process of upgrading.

The primary tenant is upgraded first, followed by the tenants in parallel (five at a time). Once the installer finishes, the same installer package should be executed on the rest of the manager nodes.

In the event of a problem during the upgrade of a tenant, the tenant's State (on the Administration > Tenants page) will appear as Database Upgrade Failed (offline). The tenants interface can be used to force the upgrade process. If forcing the upgrade does not work, please contact support.

Supporting tenants

In certain cases, a primary tenant might require access to a tenant's user interface. The tenants list and tenant properties pages provide an option to "Authenticate As" a given tenant, granting them immediate read-only access.

Users are logged in as a special account on the tenant using the prefix "support_". For example, if primary tenant user jdoe logs on as a tenant, an account is created called "support_jdoe" with the "Full Access" role. The user is deleted when the support user times out or signs out of the account.

The tenant can see this user account created, sign in, sign out and deleted along with any other actions in the System events.

Users in the primary tenant also have additional diagnostic tools available to them:

1. The Administration > System Information page contains additional information about tenant memory usage and the state of threads. This can be used directly or helpful to Trend Micro support.
2. The server0.log on the disk of the manager nodes contains additional information on the name of the tenant (and the user if applicable) that caused the log. This can be helpful in determining the source of issues.

In some cases, tenants will require custom adjustments not available in the GUI. This usually comes at the request of Trend Micro support. The command line utility to alter these settings accepts the argument:

-Tenantname "account name"

to direct the setting change or other command line action at a specific tenant. If omitted, the action is on the primary tenant.

Load balancers

By default, multi-node manager provides the address of all manager nodes to all agents and virtual appliances. The agents and virtual appliances use the list of addresses to randomly select a node to contact and continue to try the rest of the list until no nodes can be reached (or are all busy). If it can't reach any nodes, it waits until the next heartbeat and tries again. This works very well in environments where the number of manager nodes is fixed and avoids having to configure a load balancer in front of the manager nodes for availability and scalability.

In multi-tenant environments, it may be desirable to add and remove manager nodes on demand (perhaps using auto-scaling features of cloud environments). In this case, adding and removing managers would cause an update of every agent and virtual appliance in the environment. To avoid this update, the load balancer setting can be used.

Load balancers can be configured to use different ports for the different types of traffic, or if the load balancer supports port re-direction it can be used to expose all of the required protocols over port 443 using three load balancers:

In all cases, the load balancer should be configured as TCP load balancer (not SSL Terminating). This ensures a given communication exchange will happen directly between the agent or virtual appliance and the manager from start to finish. The next connection may balance to a different node.