Linux Commands and Operations
This section contains a collection of useful Linux commands, operations, and examples for system administration and daily tasks.
Sudo management
Add user to sudo group
sudo usermod -aG sudo <username>
Allowing Users to Run Sudo Without a Password
Another useful feature of the sudo command is the ability to allow users to run it without being prompted for a password. This can be done by adding the following line to your sudo configuration file
sudo visudo
Add the following line to the file
<username> ALL=(ALL) NOPASSWD: ALL
Adding Extra Useful Configurations for Sudo
You can also add extra configurations to your sudo configuration file. For example, you might want to allow a user to run only one or two commands with sudo without being prompted for a password. To do this, you would add the following lines to your sudo configuration file
sudo visudo
Add the following lines to the file
<username> ALL=(ALL) NOPASSWD: /bin/ls
<username> ALL=(ALL) NOPASSWD: /usr/bin/whoami
This allows the user to run the ls and whoami commands without being prompted for a password.
ACL (Access Control Lists)
ACLs provide more fine-grained control over file and directory permissions than standard Linux permissions. They allow you to grant specific permissions to specific users or groups that are not the owner or owning group of a resource.
Check permission
getfacl <file_name>
Example output:
# file: myfile
# owner: user1
# group: group1
user::rw-
user:user2:r--
group::r--
mask::r--
other::r--
Set permission to a file for a specific user
sudo setfacl -m u:<username>:<permissions> <file_name>
Where <permissions>
can be any combination of:
r
- readw
- writex
- execute
Example:
sudo setfacl -m u:john:rwx myfile.txt
TIP
You can check if a file has ACLs applied by looking for a '+' at the end of the permission string in ls -l
output.
Recursively apply additional permission on folder
sudo setfacl -Rm u:<username>:rwx /path/to/data
WARNING
The -R
option applies the ACL to the directory and all its contents recursively.
htpasswd
The htpasswd
utility is used to create and update user authentication files for HTTP basic authentication.
Create htpasswd file
htpasswd -c /etc/nginx/.htpasswd <username>
WARNING
The -c
flag creates a new file and will overwrite an existing file!
Add user to htpasswd file
htpasswd /etc/nginx/.htpasswd <username>
Generate password with bcrypt
htpasswd -bnBC 10 "" <password> | tr -d ':\n'
Where:
-b
specifies the password is provided on the command line-n
displays the results to stdout-B
uses bcrypt encryption-C 10
sets the bcrypt cost parameter
Add user with bcrypt password
htpasswd -b -B /etc/nginx/.htpasswd <username> <password>
TIP
Bcrypt is more secure than the default encryption method. Use it whenever possible.
File Management
Create directories by file creation date and move it there
#!/usr/bin/env bash
for file in *.JPG
do
creationDate=$(stat --printf="%y" $file | cut -d ' ' -f 1)
# echo $creationDate
mkdir -p $creationDate
mv $file $creationDate
done
TIP
This script takes each JPG file, extracts its creation date, creates a directory with that date, and moves the file into it.
Modify creation date from filename
#!/usr/bin/env bash
for file in *.jpg
do
creationDate=$(stat --printf="%n" $file | sed -E 's|([a-zA-Z]+\_)(.{8})\_(.{4})(.{2}).+|\2\3.\4|')
touch -a -m -t $creationDate $file
# echo $creationDate
done
INFO
This script extracts a date pattern from filenames and sets each file's timestamp accordingly.
Check disk space
df -h
Example output:
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 30G 15G 14G 52% /
/dev/sdb1 950G 300G 601G 34% /data
File Transfer and Synchronization
Copy files locally using rsync
rsync -avzh /source/directory /destination/directory
Where:
-a
- archive mode (preserves permissions, ownership, timestamps, etc.)-v
- verbose output-z
- compress data during transfer-h
- human-readable output format
Effect of Trailing Slash in rsync
Command | Result |
---|---|
rsync -avzh source/ target/ | Copies contents of source/ into target/ (without creating an extra source/ directory inside target/ ) |
rsync -avzh source target/ | Copies the entire source directory into target/ , resulting in target/source/ containing everything from source/ |
Examples:
Copying only contents of source/
rsync -avzh source/ target/
target/
will contain files inside source/
, but not source/
itself.
Copying the whole directory source
into target/
:
rsync -avzh source target/
target/source/
will contain all the contents of the original source/
directory
Copy over Network
Copy from a remote server using rsync info
WARNING
rsync must be installed on both systems
rsync -avzh user@remote_server:/path/to/remote/directory /path/to/local/directory
TIP
To copy from local to remote, swap the source and destination:
rsync -avzh /path/to/local/directory user@remote_server:/path/to/remote/directory
Using scp (Secure Copy)
scp -r user@remote_server:/path/to/remote/directory /path/to/local/directory
Where:
-r
- recursively copy entire directories
Archive Operations
Archive Creation
Archive a single file
tar -cvf archive_name.tar file_name
Where:
-c
- create a new archive-v
- verbose mode (shows files being processed)-f
- specify archive file name
Archive a folder
tar -cvf archive_name.tar directory_name/
Create a compressed archive (gzip)
tar -czvf archive_name.tar.gz directory_name/
Create a compressed archive (bzip2)
tar -cjvf archive_name.tar.bz2 directory_name/
Large Archive Handling
Archive and Split
tar -cvf - file_name | split -b 1000m - archive_name.tar
This creates multiple 1GB files named archive_name.tar.aa, archive_name.tar.ab, etc.
Split an existing archive
split -b 1000m archive_name.tar archive_name.tar.
Archive Extraction
Unpack a tar archive
tar -xvf archive_name.tar
Where:
-x
- extract files from an archive-v
- verbose mode-f
- specify archive file name
Unpack a compressed tar archive (auto-detect compression)
tar -xvf archive_name.tar.gz
Unpack a split tar archive
cat file.tar.* | tar xvf -
TIP
To view the contents of an archive without extracting it, use:
tar -tvf archive_name.tar
File Listings
Sort files by date
List files with newest first
ls -lat
Where:
-l
- long listing format (shows permissions, size, date, etc.)-a
- show all files, including hidden ones (those starting with a dot)-t
- sort by modification time, newest first
List files with newest last
ls -lat | tac
INFO
The tac
command reverses the order of lines, making the newest files appear at the end instead of the beginning.
Symbolic Links
Symbolic links (also called symlinks or soft links) are special files that refer to other files or directories. They provide a way to create shortcuts or references to files located elsewhere in the filesystem.
Creating Symbolic Links
To create a symbolic link in Linux, use the ln
command with the -s
option:
ln -s <source_file_or_directory> <symbolic_link_name>
Where:
ln
: The command used to create links-s
: The option to create a symbolic link (without this, a hard link would be created)<source_file_or_directory>
: The target file or directory you want to link to<symbolic_link_name>
: The name and location of the symbolic link you want to create
Examples
Creating a symbolic link to a file
ln -s /path/to/source/file.txt /path/to/symlink.txt
This creates a symbolic link named symlink.txt
that points to the file file.txt
.
Creating a symbolic link to a directory
ln -s /path/to/source/directory /path/to/symlink_directory
This creates a symbolic link named symlink_directory
that points to the directory /path/to/source/directory
.
Creating a symbolic link in the current directory
ln -s /path/to/source/file.txt symlink.txt
This creates a symbolic link named symlink.txt
in the current directory that points to the file /path/to/source/file.txt
.
Managing Symbolic Links
Checking symbolic links
ls -la
In the output, symbolic links are indicated with an l
at the beginning of the permissions string and show the target path with a ->
indicator.
Removing symbolic links
rm symlink_name
or
unlink symlink_name
WARNING
Do not add a trailing slash when removing a symbolic link to a directory, as it might delete the contents of the target directory instead.
Types of Symbolic Links
Absolute vs. Relative Symbolic Links
Absolute links use the full path to the target file/directory:
bashln -s /full/path/to/target /path/to/link
Relative links use a path relative to the link's location:
bashln -s ../relative/path/to/target link_name
TIP
Relative links are more portable when moving directory structures, but will break if the relationship between the link and target changes.
Creating Certificate Authority and Domain Certificates with OpenSSL
This guide walks you through the process of creating a Certificate Authority (CA) and a wildcard domain certificate using OpenSSL. It provides scripts to generate the necessary entities.
Initial Setup
We will be working with OpenSSL, a robust, full-featured, and flexible toolkit that implements the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) protocols.
In order to get started, you must have OpenSSL installed on your machine. It's typically pre-installed on many Unix/Linux based distributions. For Windows, you can download it here.
Generating a CA Key Pair and Self-Signed Certificate
Use the following bash script to generate a CA key pair and a self-signed certificate:
# Prepare Config File
cat > openssl-ca.cnf <<EOF
[ req ]
default_bits = 2048
default_md = sha256
prompt = no
distinguished_name = req_distinguished_name
x509_extensions = x509_ext
[ req_distinguished_name ]
C = US
ST = CA
L = Palo Alto
O = Your Organization
OU = Your Department
CN = Your Certificate Common Name
[ x509_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1 = *.yourdomain.com
EOF
# Create CA key pair
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out ca.key
# Create self signed certificate
openssl req -x509 -new -nodes -key ca.key -days 1024 -out ca.pem -config openssl-ca.cnf
After executing this script, ca.key and ca.pem files are created. ca.key is your private key, and ca.pem is your self-signed certificate.
Generating a Wildcard Domain Certificate signed by your CA
Here is the script to create a wildcard domain certificate:
# Prepare Config File
cat > openssl-domain.cnf <<EOF
[ req ]
default_bits = 2048
default_md = sha256
prompt = no
distinguished_name = req_distinguished_name
req_extensions = req_ext
[ req_distinguished_name ]
C = US
ST = CA
L = Palo Alto
O = Your Organization
OU = Your Department
CN = *.ds.local
[ req_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1 = *.localhost
DNS.2 = localhost
EOF
# Generate Domain Private Key
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out domain.key
# Generate CSR for Domain
openssl req -new -key domain.key -out domain.csr -config openssl-domain.cnf
# Generate Domain Certificate signed by CA
openssl x509 -req -in domain.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out domain.crt -days 365 -extensions req_ext -extfile openssl-domain.cnf
After running this script, domain.key will be the private key for the wildcard domain and domain.crt will be the wildcard domain's certificate signed by your self-signed CA.
Additional OpenSSL Commands
Below are several useful OpenSSL commands that can be used to validate a password, extract a certificate, or extract a key from a .p12 file. Validate an Export Password (.pem) To validate a password of a pem file you can use the following command:
openssl rsa -in private.pem -check
This command will prompt for the password. If the password is correct, it will output: RSA key OK. This means your private key's password is valid.
Extract Certificate from a .p12 File
You can extract a certificate from a .p12 file using the following command:
openssl pkcs12 -in file.p12 -out certificate.crt -nokeys
This command extracts the certificate from the file.p12 and outputs it to certificate.crt. The -nokeys option is used to exclude keys.
Extract Key from a .p12 File
To extract the key from a .p12 file, you can use the following command:
openssl pkcs12 -in file.p12 -out private.key -nocerts -nodes
This command extracts the key from the file.p12 and outputs it to private.key. The -nocerts option is used to exclude certificates. The -nodes option indicates that the private key should not be encrypted.
These commands are part of ensuring your SSL certificates are stored and used correctly, facilitating secure communications in your applications. Ensure to replace the placeholders (file.p12, private.pem, certificate.crt, and private.key) with your actual file names.
MTLS configuration
To generate mTLS (Mutual TLS) certificates, you can use OpenSSL, a widely used software library for applications that secure communications over computer networks. Here's a step-by-step guide:
Generate the Root Certificate Authority (CA) The first step is to create your own Certificate Authority (CA). The CA is a trusted entity that signs the certificates.
bashopenssl genrsa -out ca.key 2048 openssl req -x509 -new -nodes -key ca.key -sha256 -days 1024 -out ca.crt
Generate the Server Certificate Next, create the server certificate and key. These files will be used by the server during the SSL/TLS handshake.
bashopenssl genrsa -out server.key 2048
Create a certificate signing request (CSR) for the server. The CSR is a block of encoded text that is given to a CA when applying for an SSL/TLS certificate.
bashopenssl req -new -key server.key -out server.csr
Now, use the CA to create a certificate for the server.
bashopenssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 500 -sha256
Generate the Client Certificate Similar to the server, create the client certificate and key.
bashopenssl genrsa -out client.key 2048
Create a CSR for the client.
bashopenssl req -new -key client.key -out client.csr
Use the CA to create a certificate for the client.
bashopenssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days 500 -sha256
Now, you have a set of certificates and keys for the server (server.crt
, server.key
) and the client (client.crt
, client.key
), as well as the CA root certificate (ca.crt
). These can be used to establish an mTLS connection.
Please replace the certificate validity (-days
) as per your requirements. Also, when running the openssl req
command, it will prompt for information that will be incorporated into your certificate request. You can fill it as per your requirements.
Pack into PKCS#12 (P12) file
To pack a client certificate, key, and CA into a PKCS#12 (P12) file, you can use the openssl
command-line tool. Here's how you can do it:
openssl pkcs12 -export -out certificate.p12 -inkey client.key -in client.crt -certfile ca.crt
In this command:
certificate.p12
is the output P12 file.client.key
is the client's private key.client.crt
is the client's certificate.ca.crt
is the certificate of the Certificate Authority (CA).
You will be prompted to enter an export password. This password will be required to import the P12 file.
Please replace client.key
, client.crt
, ca.crt
, and certificate.p12
with your actual file paths.
Process Management
Monitoring Processes
View all running processes
ps aux
Where:
a
: show processes for all usersu
: display the process's user/ownerx
: also show processes not attached to a terminal
Interactive process viewer
top
TIP
Press q
to quit, h
for help, k
to kill a process
Enhanced interactive process viewer
htop
INFO
You may need to install htop: sudo apt install htop
or sudo yum install htop
Process Control
Starting a new process in the background
To start a new process in the background, append an &
at the end of the command:
your_command &
This will start your_command
in the background and return you to the command prompt.
Moving existing processes to the background
To move an existing process to the background:
- Press
Ctrl + Z
to pause the process - Run the
bg
command to resume it in the background
bg [job_id]
Example:
bg 1
List all background jobs
jobs
Bring a background process to the foreground
fg [job_id]
Example:
fg 1
Process Persistence
Starting a process that survives terminal closure
To start a new process that will continue running even after you log out, use the nohup
command:
nohup your_command &
WARNING
Output will be directed to nohup.out
in the current directory unless redirected
Process Priority Management
Start a process with modified priority
nice -n [priority] command
Where [priority] ranges from -20 (highest) to 19 (lowest), with 0 being the default.
Example:
nice -n 10 find / -name "*.log"
Change priority of a running process
renice [priority] -p [pid]
Example:
renice 10 -p 1234
Killing Processes
Kill a process by PID
kill [pid]
Force kill a process
kill -9 [pid]
WARNING
The -9
signal (SIGKILL) forces termination and doesn't allow the process to clean up. Use as a last resort.
Kill a process by name
pkill [process_name]
Kill all processes for a specific user
pkill -u [username]
SSH Keys
SSH Keys
SSH keys provide a secure way to authenticate with remote systems without using passwords. They consist of a public key that is placed on the remote system and a private key that remains on your local machine.
Generating SSH Keys
- Open a terminal and run the following command:
ssh-keygen -t ed25519 -C "your_email@example.com"
-t ed25519
: Specifies the type of key to create. Ed25519 is recommended as it provides better security with faster performance-C "your_email@example.com"
: Adds a comment to help identify the key
When prompted, specify where to save the key (default:
~/.ssh/id_ed25519
) or press Enter to accept the default locationEnter a secure passphrase when prompted (recommended) or press Enter twice for no passphrase
Alternative: RSA Keys
If you need to use RSA keys (for older systems that don't support Ed25519):
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
-t rsa
: Specifies RSA type-b 4096
: Specifies the key length in bits
Key Locations
- Private key:
~/.ssh/id_ed25519
(orid_rsa
for RSA) - Public key:
~/.ssh/id_ed25519.pub
(orid_rsa.pub
for RSA)
Viewing Your Public Key
To display your public key:
cat ~/.ssh/id_ed25519.pub
Adding Your Key to a Remote Server
- Using ssh-copy-id (easiest method):
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@remote-host
- Manual method:
# Copy your public key
cat ~/.ssh/id_ed25519.pub | ssh user@remote-host "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"
Key Permissions
Proper permissions are crucial for SSH to work:
# Set correct permissions for SSH directory
chmod 700 ~/.ssh
# Set correct permissions for private key
chmod 600 ~/.ssh/id_ed25519
# Set correct permissions for public key and authorized_keys
chmod 644 ~/.ssh/id_ed25519.pub
chmod 644 ~/.ssh/authorized_keys
Testing the Connection
After setting up your SSH key, test the connection:
ssh user@remote-host
Managing Multiple Keys
If you need different keys for different services (e.g., GitHub, GitLab, work server), you can:
- Generate keys with different names:
ssh-keygen -t ed25519 -C "work@example.com" -f ~/.ssh/id_ed25519_work
ssh-keygen -t ed25519 -C "github@example.com" -f ~/.ssh/id_ed25519_github
- Create or edit
~/.ssh/config
:
# Work server
Host work
HostName work.example.com
User username
IdentityFile ~/.ssh/id_ed25519_work
# GitHub
Host github.com
IdentityFile ~/.ssh/id_ed25519_github
Security Best Practices
- Always use a strong passphrase for your SSH keys
- Keep your private key secure and never share it
- Use different keys for different services
- Regularly audit and rotate your SSH keys
- Back up your SSH keys securely
- Use ssh-agent to avoid typing your passphrase repeatedly:
# Start ssh-agent
eval $(ssh-agent)
# Add your key
ssh-add ~/.ssh/id_ed25519