Troubleshooting: "Offline" agent
A computer's status changes to "Offline" or "Managed (Offline)" when it has exceeded your configured missed heartbeat threshold with the Manager. This can also appear in alerts and events.
Heartbeat connections can fail because a:
- Computer is powered off
- Computer has left the context of the private network
This can occur if you are using an on-premise installation of Deep Security Manager, and roaming endpoints (such as a laptop) cannot connect through the public Internet to the Manager on your private, internal network.
- DNS was down, or could not resolve the Manager's host name
- Firewall, IPS rule, or AWS security group is blocking the heartbeat port number
- Bi-directional communication is enabled, but only one direction is allowed or reliable
- Deep Security Manager, the agent, or both are under very high system resource load
- Deep Security Agent process (on Windows, the ds_agent service) might not be running
- Certificates for mutual authentication in the SSL or TLS connection have become invalid or revoked, or the system time is incorrect
- Deep Security rule update is not yet complete, temporarily interrupting connectivity
Check the service, routes, and port numbers to verify that the Deep Security Agent is running, and that it can communicate with Deep Security Manager.
On the computer with Deep Security Agent, verify that the Trend Micro Deep Security Agent service is running. Method varies by operating system.
- On Windows, open the Microsoft Windows Services Console (services.msc) or Task Manager.
On Linux, open a terminal and enter the command for a process listing, such as:
sudo ps -aux | grep ds_agent
sudo service ds_agent status
If agents connect to the Deep Security Manager via its domain name or hostname, not its IP address, test the DNS resolution:
nslookup [manager domain name]
If the test fails, verify that the agent is using the correct DNS proxy or server (internal domain names can't be resolved by a public DNS server such as Google or your ISP). If a name such as dsm.example.com cannot be resolved into its IP address, communication will fail, even though correct routes and firewall policies exist for the IP address.
- Ensure the "Force Allow DHCP DNS" and "Force Allow ICMP type3 code4" properties are enabled in the Advanced Network Engine settings for the computer or policy.
Telnet to required port numbers on Deep Security Manager to verify that a route exists, and the port is open:
telnet [manager IP]:[heartbeat port]
Ping is disabled on computers that use the default security policy for Deep Security Manager. Networks sometimes block ICMP ping and traceroute to block attackers' reconnaissance scans. So usually, you can't ping the Manager to test. But if the telnet succeeds, it proves most of the same things: that a route and correct firewall policy exist, and that Ethernet frame sizes are correct.)
If the telnet fails, however, there are several possible causes. To find the cause more quickly, during troubleshooting, you might want to temporarily enable ping and ICMP traffic.
If telnet fails, trace the route to discover which point on the network is interrupting connectivity. Methods vary by operating system.
On the Deep Security Manager, ping the Deep Security Agent and telnet to the heartbeat port number to verify that heartbeat and configuration traffic can reach the agent:
ping [agent IP]
telnet [agent IP]:[heartbeat port]
If the ping and telnet fail, use:
traceroute [agent IP]
to discover which point on the network is interrupting connectivity. Adjust firewall policies, routes, NAT port forwarding, or all three to correct the problem.
If IPS or firewall rules are blocking the connection between the Deep Security Agent and the Deep Security Manager, then on the Deep Security Agent, enter the command:
to reset policies on the Deep Security Agent. (In this situation, because the security on the agent is blocking communication with Deep Security Manager, the manager cannot connect in order to unassign the policy that is causing the problem.)You must re-activate the agent after running this command.