Introduction
The proc filesystem `procfs` is a special filesystem in Unix-like operating systems that presents information about processes and other system details in a hierarchical file-like structure. This offers a more convenient and standardized method for dynamically accessing process data held in the kernel compared to traditional tracing methods or direct access to kernel memory. Typically, it is mounted at `/proc` during boot time. The proc filesystem serves as an interface to internal data structures about running processes in the kernel. In Linux, it can also be used to obtain information about the kernel and to modify certain kernel parameters at runtime via `sysctl`.
Process information in procfs
The directory `/proc/PID` contains various files that provide information about a specific process identified by PID (Process Identifier). The content of these files is generated on the fly by the kernel when a process reads them. Linux first introduced the `/proc` filesystem in version 0.97.3 in September 1992, and expanded it to include non-process-related data in version 0.98.6 in December 1992.
As of 2020, the Linux implementation of `procfs` includes a directory for each running process, including kernel processes, named `/proc/PID`, where PID is the process number. Each directory contains the following information:
- /proc/PID/cmdline: The command that originally started the process.
- /proc/PID/cwd: A symlink to the current working directory of the process.
- /proc/PID/environ: Contains the names and values of the environment variables affecting the process.
- /proc/PID/exe: A symlink to the original executable file, if it still exists (a process may continue running after its original executable has been deleted or replaced).
- /proc/PID/fd: A directory containing symbolic links for each open file descriptor.
- /proc/PID/fdinfo: A directory containing entries that describe the position and flags for each open file descriptor.
- /proc/PID/maps: A text file containing information about mapped files and memory blocks (like heap and stack).
- /proc/PID/mem: A binary image representing the process’s virtual memory, accessible only by a ptracing process.
- /proc/PID/root: A symlink to the root path as seen by the process. For most processes, this will be a link to /, unless the process is running in a chroot jail.
- /proc/PID/status: Contains basic information about the process, including its run state and memory usage.
- /proc/PID/task: A directory containing hard links to any tasks started by the process (i.e., the parent process).
Non-process-related system information
The `/proc` filesystem also includes non-process-related system information, including:
- /proc/cmdline: Provides the boot options passed to the kernel.
- /proc/loadavg: Contains statistics about the system’s load average over the last minutes.
- /proc/meminfo: Summarizes how the kernel is managing its memory.
- /proc/modules: Contains a list of currently loaded kernel modules and some indication of their dependencies.
- /proc/mounts: A symlink to self/mounts, listing currently mounted devices, their mount points, and filesystem types.
- /proc/net/: A directory containing useful information about the network stack.
Process monitoring with `ps and strace`
When working with Linux systems, understanding the processes running on your system is crucial. The ps command is a powerful tool for this purpose, offering various options for viewing and filtering process information. On Linux, the `ps` (Process Status) command retrieves data by reading files in the `proc` filesystem.
To see how ps operates in a Linux environment, you can use `strace` (System call Monitoring). `strace` is a diagnostic, debugging, and instructional userspace utility for Linux that monitors interactions between processes and the Linux kernel, including system calls, signal deliveries, and changes in process state. This functionality is made possible by the kernel feature known as `ptrace`.
To examine the data accessed by the `ps` command, you can use `strace -e openat ps`.
Figure 1. An example of the data accessed by the `ps` command using the `strace` command.
The following is a list of files accessed via `openat` by the `ps` command, showing that it collects data from `/proc/[PID]/*`. You can perform the same check `ss -tnup`.
Figure 2. A list of files accessed via openat by the ps command.
The `ss` command also collects data from `/proc/[PID]/*`.
Can processes be hidden by manipulating the `proc` filesystem?
So what happens when an attempt is made to remove a process ID from `/proc/`? As illustrated below, the kernel prevents users, including root, from removing process IDs from `/proc/`.
Figure 3. Attempts to remove a process ID from `/proc/` will result in “Operation not permitted”.
But what happens when we mount an empty folder with the same directory of the `/proc/`?
Since Linux 2.4.0, it has been possible to remount a part of the file hierarchy to a different location using the following command:
mount –bind olddir newdir
Or the shorter option:
/olddir /newdir none bind
In the /etc/fstab file, this can be configured as:
/olddir /newdir none bind
After this command, the same contents will be accessible from both locations. It is also possible to remount a single file. This operation affects only part of the filesystem and does not include submounts.
Figure 4. A screenshot of a process successfully hidden from the administrator.
And there you have it. We managed to hide a process from the administrator. This tactic is also employed by an unclassified threat actor group. Traditional incident response (IR) triage typically relies on script lists, but this technique can be used for security evasion.
Recommendations
Group-IB recommends that administrators should:
- Enable`SElinux/AppArmor`.
- Always check the mounting table to ensure that there is no folder mounted to `/proc/`, as shown below:
