Detecting Linux Rootkit Intrusion Threats
Overview
Rootkit is a type of covert malware designed to conceal itself and other malicious processes, files, and network connections, while providing persistent backdoor access for attackers.
"Root" represents the highest privilege in the Linux system, and "Kit" represents a toolkit. A rootkit is essentially a hidden toolkit used after obtaining the highest system privilege.
Rootkit Risks
Rootkits have evolved from their early text-based mode, which was relatively easy to detect, to highly customized and modularized forms, often incorporating advanced persistent threat (APT) attacks.
Rootkits also pose technical challenges such as covert design, kernel-level evasion, polymorphism, and blocking of detection resources, making detection extremely difficult.
The specific threats are as follows:
- Permission maintenance and persistence
- Implanting hidden accounts
- Modifying startup scripts and scheduled tasks
- Hooking the system authentication module to bypass login authentication
- Data theft
- Stealing passwords by recording keystrokes
- Intercepting sensitive data transmitted over the network
- Reading sensitive information from memory
- Lateral movement
- Attacking other systems on the intranet as a jump server
- Stealing intranet credentials for credential reuse attacks
- Resource abuse
- Using server resources for cryptocurrency mining
- Building botnets to participate in DDoS attacks
- Trace clearing
- Deleting logs
- Hiding attack traces
Detection Principles
HSS uses a four-layer detection system to identify rootkits from different dimensions.
Layer 1: File detection
Rootkits hide malicious files, which cannot be detected by common ls commands. HSS does not rely on ls, readdir, or other similar interfaces that could be exploited. Instead, it directly reads the file system's underlying data structures.
- HSS traverses the directory inode nodes, reads the raw data of directory entries, and compares the raw file list with the file list obtained through system interfaces. If a file exists in the raw data but is not returned by the system interface, the file is flagged as hidden.
- HSS performs integrity checks on key binary files in the system. It calculates the hash values of key system commands (such as ps, ls, and netstat) and compares them with the pre-stored hash values in the database. If the hash values do not match, the files have been tampered with or replaced.
Layer 2: Kernel detection
System calls are the only way for user programs to access the kernel. Rootkits can modify the system call table to intercept specific operations. However, each kernel version has a fixed kernel symbol address. HSS pre-stores the correct addresses of key system call functions, checks in real time whether the function pointers in sys_call_table have been tampered with, and compares the current address against the expected address.
- Linux rootkits often exist as loadable kernel modules. HSS obtains the list of all currently loaded kernel modules, compares the module hash values against the whitelist database, and checks for unauthorized hidden modules. It also monitors the dynamic loading behavior of new modules.
- Rootkits often hook symbol resolution functions such as kallsyms_lookup_name to conceal their presence and to modify the symbol table exported by the kernel. HSS can monitor changes in key kernel function addresses and detect abnormal offsets of symbol addresses.
- In some advanced detection scenarios, HSS checks the integrity of kernel memory areas by detecting whether malicious code or abnormal kernel memory pages have been injected into kernel code segments and by performing pattern matching against known rootkit memory signatures.
Layer 3: Behavior detection
Even if a rootkit successfully conceals itself, its behavior still leaves traces. HSS detects anomalies in process behavior, network behavior, file operations, and permission operations.
Layer 4: Intelligence detection
HSS maintains an up-to-date rootkit signature database based on known rootkit file hashes, code signatures, typical behavior patterns, and network communication characteristics. When a match is detected, an alarm is triggered. HSS can also correlate detection results with threat intelligence data.
Handling Rootkit Alarms
The suspicious kernel module alarm lists the names of detected kernel modules (commonly known as rootkits) and the system functions hooked by each module. These system functions are hooked to hide malware from security software. For example, in the following figure, the kernel module hooks the filldir function, which is likely to be used for hiding malware.

Kernel hooking is a technique that intercepts and modifies function calls or behaviors within the operating system kernel.
Scenarios where kernel module names can be identified
- Log in to the server and run the following command to check whether there is any .ko file:
find / -name {Kernel module name}.ko
- If the kernel module exists in the command output, it is a kernel module required by the customer's services. (Malicious kernel modules will hide themselves so that they cannot be found by the find command.) In this case, confirm with the customer and handle it as a false positive.
- If the kernel module is not displayed in the command output, go to the next step.
- Run the following command to find the file path of the kernel module: This command can be used to find the dependency information of the kernel module, including the full path of the .ko file.
modprobe -D {Kernel module name}

- If the kernel module is hidden, the .ko file cannot be found by running ls -l. In this case, you need to know the full path of the .ko file.
Run the following commands to rename the .ko file of the kernel module and move it to a location where it cannot be loaded. Then, restart the server to prevent the rootkit from being loaded.
mv /lib/modules/3.10.0-862.14.1.5.h408.eulerosv2r7.x86_64/kernel/drivers/md/dm-mod.ko /root/dm-mod.ko.bak
- If the hidden .ko file path cannot be found using the preceding command, go to the /etc/modprobe.d/ directory and create a blacklist.conf file.
- Run the following command to add content to the file and restart the server to prevent the kernel module with the specified name from being loaded:
install {Kernel module name} /bin/true


As shown in the figure above, take the open-source Reptile rootkit as an example. After adding its two ko kernel module names to the configuration, the hiding operation will fail because kernel module loading has been blocked.
- If the kernel module fails to load, the hiding function is disabled. Run the following command to detect the previously hidden malicious files:
find / -name *{Kernel module name}*
- After the rootkit is unhidden, perform full-disk virus scanning and killing on the target server again.
Scenarios where kernel module names cannot be identified
Some rootkits are highly sophisticated. Once loaded, their names cannot be detected using current technical methods.
The current solution is to clone the disk of the compromised server and mount it to another server. This bypasses the rootkit's concealment. The files on the mounted disk can be checked and scanned for viruses.
What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot