Wide range of Linux commands with their usage and descriptions

Here’s a more extensive HTML table of common Linux commands with their usage and descriptions:

Command Usage Description
ls ls [options] [directory] Lists the contents of a directory.
cd cd [directory] Changes the current directory to the specified directory.
mkdir mkdir [directory] Creates a new directory.
rm rm [options] [file] Deletes files or directories.
rmdir rmdir [directory] Removes an empty directory.
cp cp [options] source destination Copies files or directories.
mv mv [options] source destination Moves or renames files or directories.
touch touch [file] Creates an empty file or updates the timestamp of an existing file.
chmod chmod [options] mode file Changes file permissions.
chown chown [options] user[:group] file Changes file owner and group.
ps ps [options] Displays information about active processes.
kill kill [options] pid Terminates a process by its ID.
df df [options] Shows disk space usage.
du du [options] [directory] Shows disk usage of files and directories.
tar tar [options] file Archives files.
gzip gzip [file] Compresses files.
gunzip gunzip [file] Decompresses gzip files.
zip zip [options] zipfile files Creates a compressed archive.
unzip unzip [file.zip] Extracts files from a zip archive.
wget wget [options] url Downloads files from the internet.
curl curl [options] url Transfers data from or to a server.
ping ping [options] hostname Checks the network connection to a server.
top top Displays real-time system processes and resource usage.
nano nano [file] Opens a file in the Nano text editor.
vim vim [file] Opens a file in the Vim text editor.
echo echo [text] Displays a line of text or variable value.
cat cat [file] Displays the content of a file.
less less [file] Views the content of a file page by page.
head head [options] [file] Displays the first lines of a file.
tail tail [options] [file] Displays the last lines of a file.
find find [path] [options] Searches for files in a directory hierarchy.
locate locate [file] Finds the location of a file.
man man [command] Displays the manual page for a command.
ssh ssh [options] user@hostname Connects to a remote server via SSH.
scp scp [options] source destination Copies files between hosts over SSH.
sudo sudo [command] Executes a command as another user, typically root.
apt-get apt-get [options] command Manages packages on Debian-based systems.
yum yum [options] command Manages packages on Red Hat-based systems.
systemctl systemctl [command] Controls the systemd system and service manager.
service service [service] [command] Manages system services.
passwd passwd [options] [user] Changes a user’s password.
adduser adduser [options] user Adds a new user to the system.
deluser deluser [options] user Removes a user from the system.
crontab crontab [options] [file] Schedules regular tasks with cron.
alias alias name=’command’ Creates a shortcut for a command.
unalias unalias name Removes a command alias.
uname uname [options] Prints system information, such as the kernel name, version, and more.
hostname hostname Displays or sets the system’s hostname.
free free [options] Shows memory and swap usage.
df df [options] Shows disk space usage.
du du [options] [directory] Shows disk usage of files and directories.
history history Displays the list of previously executed commands.
uname uname -a Displays system information including the kernel version.
uptime uptime Shows how long the system has been running, including load averages.
who who Displays information about users currently logged into the system.
useradd useradd [options] username Adds a new user account.
usermod usermod [options] username Modifies a user account.
userdel userdel [options] username Deletes a user account.
groupadd groupadd [options] groupname Creates a new group.
groupdel groupdel groupname Deletes a group.
passwd passwd [options] [username] Changes a user’s password.
top top Displays real-time information about system processes and resource usage.
killall killall [process name] Terminates all processes with the specified name.
reboot reboot Reboots the system.
shutdown shutdown [options] [time] Shuts down the system.
mount mount [options] device directory Mounts a filesystem.
umount umount [options] directory Unmounts a filesystem.
fdisk fdisk [options] device Partition a hard disk.
mkfs mkfs [options] device Formats a filesystem on a device.
fsck fsck [options] [filesystem] Checks and repairs a filesystem.
df df [options] Reports the amount of disk space used and available on filesystems.
du du [options] [directory] Estimates file space usage.
ln ln [options] source [target] Creates hard and symbolic links between files.
tee tee [options] [file] Reads from standard input and writes to standard output and files.
chmod chmod [options] mode file Changes file modes or Access Control Lists (ACLs).
chown chown [options] user[:group] file Changes file owner and group.
chgrp chgrp [options] group file Changes the group ownership of a file.
dd dd [options] [if=file] [of=file] Converts and copies a file.
grep grep [options] pattern [file] Searches for patterns in files.
sed sed [options] script [file] Stream editor for filtering and transforming text.
awk awk [options] ‘program’ [file] A programming language that can be used for text processing and data extraction.
cut cut [options] file Removes sections from each line of files.
sort sort [options] file Sorts lines of text files.
uniq uniq [options] file Removes duplicate lines from a file.
diff diff [options] file1 file2 Compares files line by line.
comm comm [options] file1 file2 Compares two sorted files line by line.
head head [options] [file] Outputs the first part of files.
tail tail [options] [file] Outputs the last part of files.
paste paste [options] file1 file2 Merges lines of files side by side.
tr tr [options] set1 set2 Translates or deletes characters.
echo echo [options] string Displays a line of text.
expr expr expression Evaluates expressions.
bc bc An arbitrary precision calculator language.
xargs xargs [options] [command] Builds and executes command lines from standard input.
printf printf [format] [arguments] Formats and prints data.
date date [options] [+format] Displays or sets the system date and time.
cal cal [options] [month] [year] Displays a calendar.
uptime uptime Shows how long the system has been running.
whoami whoami Prints the current username.
env env [options] Displays or modifies the environment.
alias alias name=’command’ Creates an alias for a command.
unalias unalias name Removes an alias.
export export [variable] Sets an environment variable.
unexport unexport [variable] Unsets an environment variable.
seq seq [options] first last Prints a sequence of numbers.
basename basename [path] [suffix] Strips directory and suffix from filenames.
dirname dirname [path] Strips the last component from a file path.
bc bc [options] Interactive algebraic language calculator.
yes yes [string] Outputs a string repeatedly until killed.
nohup nohup command [arguments] Runs a command immune to hangups.
sleep sleep [number][suffix] Delays for a specified amount of time.
bc bc Basic calculator with support for floating-point arithmetic.
clear clear Clears the terminal screen.
jobs jobs Lists the active jobs.
fg fg [job] Brings a job to the foreground.
bg bg [job] Resumes a suspended job in the background.
disown disown [job] Removes jobs from the job table.
type type [command] Describes how a command would be interpreted.
read read [options] variable Reads a line of input into a variable.
trap trap [command] [signal] Catches and handles signals.
wait wait [pid] Waits for processes to complete.
kill kill [signal] pid Sends a signal to a process.
pkill pkill [options] pattern Stops processes by name.
killall killall [options] name Kills processes by name.
bg bg [job] Resumes a job in the background.
fg fg [job] Brings a job to the foreground.
jobs jobs Lists background jobs.
alias alias name=’command’ Creates a shortcut for a command.
unalias unalias name Removes an alias.
echo echo [string] Displays a line of text.
cal cal [options] [month] [year] Displays a calendar.
yes yes [string] Outputs a string until killed.
bc bc [options] Interactive calculator.
time time [command] Measures the duration of a command’s execution.
uptime uptime Shows how long the system has been running.
who who Displays who is logged in.
w w Shows who is logged in and what they are doing.
users users Shows the currently logged-in users.
last last [options] [username] Shows the last logins of users.
lastb lastb [options] Shows the last bad login attempts.
uname uname [options] Displays system information.
df df [options] Shows disk space usage.
du du [options] [directory] Shows disk usage.
lsof lsof [options] Lists open files and the processes using them.
strace strace [options] command Traces system calls and signals in a process.
nc (netcat) nc [options] host port Performs network operations, such as port scanning, or file transfer.
tcpdump tcpdump [options] Captures and analyzes network packets.
rsync rsync [options] source destination Synchronizes files and directories between two locations.
screen screen [options] Manages multiple shell sessions within one terminal.
tmux tmux [options] Terminal multiplexer to manage multiple terminal sessions.
htop htop Interactive process viewer (an enhanced version of top).
vmstat vmstat [options] Reports virtual memory statistics.
iotop iotop [options] Monitors disk I/O usage by processes.
iftop iftop [options] Displays bandwidth usage on an interface.
arp arp [options] Displays or modifies the ARP (Address Resolution Protocol) cache.
arping arping [options] host Sends ARP requests to hosts to discover MAC addresses.
ip ip [options] object command Utility for managing network devices, addresses, routing, and more.
ipcs ipcs [options] Displays information about IPC (Inter-Process Communication) facilities.
ipcrm ipcrm [options] Removes IPC facilities, such as message queues, semaphores, and shared memory.
ss ss [options] Displays socket statistics.
traceroute traceroute [options] host Displays the route packets take to reach a network host.
tracepath tracepath [options] host Similar to traceroute but does not require superuser privileges.
nmap nmap [options] [target] Scans networks and hosts for open ports and services.
dig dig [options] [domain] Queries DNS servers for information about a domain.
host host [options] [domain] Performs DNS lookups.
ethtool ethtool [options] interface Displays or changes Ethernet device settings.
iptables iptables [options] Configures the Linux kernel’s network packet filtering rules.
ufw ufw [options] Uncomplicated Firewall, front-end for iptables to manage firewall rules.
fail2ban fail2ban-client [options] Monitors log files and bans IPs that show malicious signs.
logrotate logrotate [options] Automatically rotates, compresses, and removes old log files.
journalctl journalctl [options] Views logs collected by systemd’s journal service.
rsyslog rsyslogd [options] Rocket-fast system for log processing.
lsblk lsblk [options] Lists information about block devices.
blkid blkid [options] [device] Prints block device attributes like UUID, filesystem type, etc.
hdparm hdparm [options] device Gets or sets SATA/IDE device parameters.
lsusb lsusb [options] Lists USB devices connected to the system.
lspci lspci [options] Lists all PCI devices.
modprobe modprobe [options] module Adds or removes a module from the Linux kernel.
lsmod lsmod Displays loaded kernel modules.
insmod insmod module Inserts a module into the Linux kernel.
rmmod rmmod module Removes a module from the Linux kernel.
depmod depmod [options] Generates modules.dep and map files for kernel modules.
mkinitcpio mkinitcpio [options] Creates an initial ramdisk environment.
mkinitrd mkinitrd [options] Creates an initial ramdisk for preloading modules during boot.
dracut dracut [options] Another tool to generate initramfs images.
setfacl setfacl [options] [file] Sets file access control lists (ACLs).
getfacl getfacl [options] [file] Gets file access control lists (ACLs).
chattr chattr [options] [files] Changes file attributes on a Linux filesystem.
lsattr lsattr [options] [files] Lists file attributes on a Linux filesystem.
fuser fuser [options] [files] Identifies processes using files or sockets.
inotifywait inotifywait [options] [file] Waits for changes to files using inotify.
inotifywatch inotifywatch [options] [file] Watches changes to files using inotify.
ionice ionice [options] [command] Sets or gets the I/O scheduling class and priority of a program.
renice renice [options] [priority] [PID] Alters the priority of running processes.
lscpu lscpu Displays information about the CPU architecture.
lsns lsns [options] Lists information about Linux namespaces.
unshare unshare [options] [command] Runs a program with some namespaces unshared from the parent.
chroot chroot [new root] [command] Changes the root directory for a command or interactive shell.
pivot_root pivot_root [new root] [put old] Changes the current root filesystem.
dd dd if=[source] of=[destination] [options] Converts and copies files.
shred shred [options] [file] Overwrites a file to hide its contents, making recovery difficult.
stat stat [options] [file] Displays detailed information about a file or file system.
namei namei [options] pathname Follows a pathname until a terminal point is reached, showing the directory entries found along the way.
findmnt findmnt [options] Finds a filesystem, listing mount points.
hdparm hdparm [options] [device] Gets or sets SATA/IDE device parameters.
dmidecode dmidecode [options] Fetches and decodes the DMI table (hardware information).
lsscsi lsscsi [options] Lists information about SCSI devices.
lsblk lsblk [options] Lists information about block devices.
wget wget [options] [url] Retrieves files from the web using HTTP, HTTPS, and FTP.
curl curl [options] [url] Transfers data from or to a server using various protocols.
aria2c aria2c [options] [url] Downloads files from the web with multi-threading and parallelization.
jq jq [options] [file] Processes JSON data, like sed for JSON.
envsubst envsubst [options] [file] Substitutes environment variables in shell format strings.
rev rev [file] Reverses lines of a file or input.
column column [options] [file] Formats text into columns.
hexdump hexdump [options] [file] Displays file contents in hexadecimal, decimal, octal, or ASCII.
xdelta3 xdelta3 [options] Computes differences between files, commonly used for patching.
asciinema asciinema [command] Records and shares terminal sessions, with playback in the browser.
zless zless [file] Views compressed text files with paging (like less).
xxd xxd [options] [file] Creates a hex dump or does the reverse.
units units [options] Converts between different units of measurement.
cal cal [options] Displays a simple calendar.
factor factor [number] Prints the prime factors of a number.
seq seq [options] [first] [increment] [last] Generates a sequence of numbers.
bc bc [options] Arbitrary precision calculator language.
dc dc [options] Reverse-polish notation calculator.

 

MySQL vs. PostgreSQL: A Comprehensive Comparison for Full Applications

Choosing the right database management system (DBMS) is crucial for the success of any application. MySQL and PostgreSQL are two of the most popular open-source relational databases, each with its own strengths and weaknesses. This article will delve into their key features, advantages, disadvantages, and use cases to help you make an informed decision.

Understanding MySQL and PostgreSQL

MySQL is a widely used, high-performance relational database management system known for its speed and simplicity. It’s often the default choice for web applications due to its ease of use and scalability.

PostgreSQL, on the other hand, is a powerful, object-relational database system that emphasizes data integrity, reliability, and advanced features. It’s suitable for complex applications requiring robust data handling and analysis.

Key Features Comparison

Feature MySQL PostgreSQL
ACID Compliance Yes Yes
Data Types Basic Richer, including arrays, JSON, and more
Indexing Supports various indexing types Supports advanced indexing, including GIN and BRIN
Transactions Supports transactions Strong support for transactions and isolation levels
Foreign Key Constraints Supports Supports
Triggers Supports Supports
Stored Procedures Supports Supports
Full Text Search Basic Advanced, with support for ranking and relevancy
Replication Supports Supports, with advanced features like streaming replication
High Availability Supports clustering Supports clustering and advanced replication features

Advantages and Disadvantages

MySQL

Advantages:

  • High performance for read-heavy workloads
  • Easy to use and administer
  • Large community and extensive support
  • Widely adopted in the industry
  • Cost-effective

Disadvantages:

  • Limited data types and features compared to PostgreSQL
  • Potentially weaker data integrity for complex applications
  • Scalability challenges for extremely large datasets

PostgreSQL

Advantages:

  • Advanced features like JSON, arrays, and full-text search
  • Strong data integrity and consistency
  • Scalability and performance improvements in recent versions
  • Active community and growing ecosystem
  • Suitable for complex applications

Disadvantages:

  • Can be more complex to set up and administer
  • Performance might be slightly lower for simple workloads compared to MySQL
  • Smaller market share than MySQL

Use Cases

MySQL is ideal for:

  • Web applications with high read traffic
  • Content management systems (CMS)
  • Online stores
  • Applications with simple data structures

PostgreSQL is well-suited for:

  • Complex web applications with heavy write loads
  • Geospatial applications
  • Data warehousing and analytics
  • Enterprise applications requiring advanced features
  • Applications with high data integrity requirements

When to Choose Which Database

Ultimately, the best database for your application depends on specific requirements:

  • Prioritize performance and simplicity: MySQL is a good choice.
  • Need advanced features, data integrity, and scalability: PostgreSQL is a strong contender.
  • Balancing performance and features: Consider both options and benchmark them with your specific workload.

It’s essential to evaluate your application’s needs carefully and consider factors such as data volume, complexity, scalability, and performance expectations. In some cases, using both databases in a hybrid architecture might be beneficial.

By understanding the strengths and weaknesses of MySQL and PostgreSQL, you can make an informed decision that will positively impact your application’s performance and reliability.

Simple Tips to Improve your Website Security

Understand the simple rule, Any loophole in your system is an open invitation for hackers to attack your website at there will and fantasty.

So the Thmb rule is simple, Keep all your doors secured and patch and monitor loopholes. Here we offer your few simple advices that could help you secure your website effectively and safe from attacks.

1. Keep All your Password Secured, The so called hackers employ scripts that brute force attack password using the possible permutation and combination. So Most Important enforce strong password Policy, use Larger than 10 charectores, Alpha numeric, with capital letters and special charectors combination.

Second aspect is never store your password in any FREE Public email accounts mailboxes or storage area, always adopt an alternate way to keep your password secure.

2. All Software NOT Upto Date are Open Invitation to hackers, Start from OS of both the Client and Server Operating system is uptodate, well patched and hardended to thwart any maleware or bit attacks. Than look for your FTP Clients, We strongly recomend avoid using pirated copies of any software including the FTP clients, in several cases it was found that the FTP client itself sending FTP credentials to hackers. We recomned use SFTP instead of FTP service and clients. Thirdly ensure your Front end and backend langaues like PHP, ASP, JSP, Perl, python, PostGREESQL and MySQL are latest one. and you have properly configured the front end and backend languages not to leake your memories due to poor safe gaurd and restrictions. Forthly ensure any CMS like WordPress or ecommerce application like magento etc are latest one. avoid plugins from not known sources, as this are also found to be backholes for hackers to enter your server.

3. Use Secured hosting only, use HTTPS protocol that provide security over the internet and ensures users are communicating with server in secured manner and while data being transfered between client and server its not being compromised. Install a secured SSL Certificate on your website.

4. Scan your website and webs server for any vulnerabilties, As several times you need to do external scan to check any vulnerabilties. and by known so you can patch and harden the vulnerabitityy.

Few of the Tools you can try :

https://securityheaders.com/
https://pentest-tools.com/website-vulnerability-scanning/web-server-scanner
https://www.qualys.com/forms/freescan/
https://sitecheck.sucuri.net/
https://www.ssllabs.com/ssltest/

Understanding SPF, DKIM, and DMARC: A Simple Guide

Email security is a key part of internet communication. But what are SPF, DKIM, and DMARC, and how do they work? This guide will explain it all in simple terms to make these concepts clearer.

Table of Contents

  1. What Is This Guide For and Why Bother?
  2. Why Choose This Guide?
  3. What This Guide Is Not For
  4. SPF, DKIM, and DMARC: Simplified
  5. Real World Examples of where SPF, DKIM, and DMARC are used
  6. Now I Know These Things, What’s Next?
  7. Checking Your SPF, DKIM, DMARC Status
  8. FAQ’s with SPF, DKIM and DMARC
  9. Wrapping Up
  10. Contributing
  11. Sharing is Caring
  12. Contact
  13. References

What Is This Guide For and Why Bother?

If you are involved in developing, supporting, or maintaining an application that sends emails, this guide is a must read. This guide is your key to peace of mind, knowing that your emails will reach your customers as intended and your domain is shielded from abuse from cybercriminals and spammers. It’s about ensuring they reach the intended destination – the recipient’s inbox, not the spam or junk folder. For instance, You’ve built an e-commerce application or a SaaS platform that sends transactional emails like order confirmations or password resets or important customer notification emails. These emails are crucial touchpoints for your customers. But what if they never see them? What if these important communications end up in spam or junk? While email is one of the most common communication channels, it’s also a favorite target for cybercriminals and a playground for spammers. Here are some real-world examples of how they can abuse email systems:

  • Phishing Attacks: A cybercriminal wants to steal sensitive information from the customers of a well-known bank. The criminal could spin up or use a compromised server to send emails that appear to come from the bank’s domain, asking customers to update their account information. If the bank hasn’t implemented SPF, the email could pass the receiving server’s checks and land in the customer’s inbox. The customer, thinking the email is from their bank, clicks the link and enters their login details on a fake website controlled by the criminal. The criminal can now access the customer’s bank account.

  • Brand Impersonation: A cybercriminal could impersonate a popular e-commerce platform and send emails to users asking them to confirm their purchase of an expensive item. The email could contain a link to a fake customer support page where the user is asked to enter their login details to cancel the purchase. If the e-commerce platform hasn’t implemented DKIM, the email could pass the receiving server’s checks and land in the user’s inbox. The user, thinking the email is from the e-commerce platform, enters their login details on the fake page, giving the criminal access to their account.

  • Business Email Compromise (BEC): A cybercriminal could impersonate a company’s CEO or another high-ranking official and send an email to the finance department, asking them to make a payment to a new vendor. If the company hasn’t implemented DMARC, the email could pass the receiving server’s checks and land in the finance department’s inbox. The finance department, thinking the email is from the CEO, could make the payment to the criminal’s bank account.

By understanding and implementing SPF, DKIM, and DMARC, you can protect your domain from being used in these types of attacks, safeguard your customers and employees, and maintain your reputation. So, why bother? Because your emails matter, your customers matter, and your reputation matters.

Why Choose This Guide?

With so many articles out in the internet why should I choose this guide? This guide stands out for its simplicity, clarity, and convenience. It demystifies SPF, DKIM, and DMARC with clear explanations and examples avoiding the technical jargon as much as possible. Hosted on GitHub, it integrates seamlessly with your development environment, providing quick access to information right from your IDE (visual studio code ,etc.) or command line. Plus, it’s a document that will stay in Github and guaranteed that won’t go anywhere that can be edited by you or anyone or the community to ensure it stays updated and relevant.

What This Guide Is Not For

While this guide aims to simplify SPF, DKIM, and DMARC, it’s not intended to be a comprehensive guide about these topics. It’s not a guide for setting up an email server, nor does it cover advanced topics like encryption or secure email gateways.

SPF, DKIM, and DMARC: Simplified

SPF (Sender Policy Framework)

SPF: It’s like a list of friends who can send emails for you. The SPF Record is this list. If an email says it’s from you but it’s not sent by a friend on your list, it’s probably not really from you. As the owner of a domain, you can use SPF to create a list of ’email friends’ – these are the mail servers that are allowed to send emails on your behalf. This helps stop people who aren’t your ’email friends’ from pretending to be you. The SPF Record, a DNS TXT record, is where you keep this list of ’email friends’. The DNS TXT record for an SPF your ’email friends’ typically looks like this:

v=spf1 ip4:123.123.123.123 ~all

Here’s the command I usually run to fetch that:

dig TXT example.com

DKIM (DomainKeys Identified Mail)

DKIM: It’s like a secret note inside your emails. When you send an email, you put a secret note inside. This note is made using a special secret code only you know. When your email arrives, the receiver checks the secret note using a public code that everyone knows. This public code is stored in a place called the DKIM Record. If the secret note matches the public code, the email is really from you and hasn’t been changed. This helps stop bad people from pretending to send emails from you or changing your emails. This public code, also known as a public key, is stored in a DNS TXT record known as the DKIM Record, which is accessible to everyone. It’s like the decoder for your secret code. The DNS TXT record, where the public code (or public key) for DKIM is stored, typically looks like this:

v=DKIM1; k=rsa; p=NICfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDBolTXCqbxwoRBffyg2efs+Dtlc+CjxKz9grZGBaISRvN7EOZNoGDTyjbDIG8CnEK479niIL4rPAVriT54MhUZfC5UU4OFXTvOW8FWzk6++a0JzYu+FAwYnOQE9R8npKNOl2iDK/kheneVcD4IKCK7IhuWf8w4lnR6QEW3hpTsawIDAQ0B"

Here’s the command I usually run to fetch that:

dig TXT selector1._domainkey.example.com

Note: Replace selector1 with your actual selector, and example.com with your actual domain. This command will fetch the DNS TXT record where your public code is stored.

DMARC (Domain-based Message Authentication, Reporting & Conformance)

DMARC: It’s like the boss of SPF and DKIM. It takes the rules from SPF and DKIM and makes a big rule book. This rule book tells everyone what to do if an email from your domain doesn’t follow the rules. For example, one rule could be to send a report if an email doesn’t pass the checks. The DMARC Record, a place everyone can see, holds this rule book. If an email passes the SPF and DKIM checks, the receiver then looks at the DMARC rule book to decide what to do with the email. They might follow the rule to send a report, or they might follow another rule depending on what your rule book says. DMARC allows domain owners to declare their rules in the rule book. This rule book, stored in the DMARC Record, a DNS TXT record, specifies your DMARC policies and how receivers should handle mail that violates these rules. If both SPF and DKIM checks pass, the receiver then checks the DMARC rule book to decide what to do with the email. The DNS TXT record for DMARC ‘rule book’ typically looks like this:

v=DMARC1; p=none; rua=mailto:postmaster@example.com

Here’s the command I usually run to fetch that:

dig _dmarc.example.com TXT

Real World Examples of SPF, DKIM, and DMARC Are Used

Let’s look at how SPF, DKIM, and DMARC work in real-world example scenarios:

  • Mobile Apps: Mobile apps that send emails, such as a fitness app sending workout summaries or a banking app sending transaction alerts, also use SPF, DKIM, and DMARC. When the app sends an email, the receiving server checks if the sending server’s IP is in the SPF record of the sender’s domain. It then uses the DKIM record to verify the email’s DKIM signature. If both checks pass, the server applies the DMARC policy to decide what to do with the email. This ensures that the emails reach the user’s inbox and not the spam folder, and protects the app’s reputation by preventing email spoofing.

  • Email Service Providers: Providers like Gmail, Yahoo, and Outlook use SPF, DKIM, and DMARC to authenticate incoming emails. For instance, when an email arrives, Gmail checks if the sending server’s IP is in the SPF record of the sender’s domain. It then uses the DKIM record to verify the email’s DKIM signature. If both checks pass, Gmail applies the DMARC policy to decide what to do with the email.

  • Social Media Platforms: Social media platforms like LinkedIn, Facebook, or Twitter that send notification emails also use SPF, DKIM, and DMARC. When a user receives a notification email, their email provider checks if the sending server’s IP is in the SPF record of the social media platform’s domain. It then uses the DKIM record to verify the email’s DKIM signature. If both checks pass, the provider applies the DMARC policy to decide what to do with the email. This ensures that the emails reach the user’s inbox and not the spam folder, and protects the social media platform’s reputation by preventing email spoofing.

  • Businesses: Businesses use SPF, DKIM, and DMARC to protect their email communication and brand reputation. For example, a business might send promotional emails to its customers. By implementing SPF, DKIM, and DMARC, the business ensures that its emails are not marked as spam and that its domain is not used for email spoofing.

  • Government Agencies: Government agencies use SPF, DKIM, and DMARC to secure their email communication and prevent phishing attacks. For instance, a government agency might send notifications to citizens. By using SPF, DKIM, and DMARC, the agency ensures that its emails reach the citizens’ inbox and that cybercriminals cannot send phishing emails that appear to come from the agency.

Now I Know These Things, What’s Next?

Now that you’ve learned the basics of SPF, DKIM, and DMARC, you might be thinking about using these tools to make your emails more secure. Here’s a simple guide to help you get started:

  1. Identify the Email Address and Domain: First, you need to know the email address and domain your app uses. You’ll need to add SPF, DKIM, and DMARC records to this domain. A simple way to find this out is by sending an email from your app to yourself. For example, you could sign up for an account on your site and click on ‘forgot password’ to receive an email.

  2. Current Status: Next, check if you already have SPF, DKIM, and DMARC records. If you do, make sure they’re set up correctly. You can learn how to do this in the next section, ‘Checking Your SPF, DKIM, DMARC Status’.

  3. Domain Access: Make sure you have the rights to change the DNS records of your domain. You’ll need this to add SPF, DKIM, and DMARC records. If you don’t have access, you’ll need to request the person who does to add these records for you.

  4. DMARC Monitoring: Once you’ve set up DMARC, you’ll need to keep an eye on DMARC reports to make sure everything’s working as it should and fix any problems. Decide who will do this and which email address will receive the DMARC reports.

The usual order is to set up the SPF record first, then DKIM, and finally DMARC.

Checking Your SPF, DKIM, DMARC Status

This is a straightforward with tools like MXToolbox and DMARCTester. Here’s how you can use these tools:

  1. MXToolbox:
    • Visit https://mxtoolbox.com/
    • Use the ‘SPF Record Lookup’, ‘DKIM Record Lookup’, and ‘DMARC Record Lookup’ tools to check the respective records for your domain.
  2. DMARCTester:
    • Visit https://www.dmarctester.com/
    • This site offers two ways to check your email security:
      • Send an Email: The site generates a unique email address for you. You can then send an email from your application or mail server to this address.

      • Paste Email Headers: Alternatively, you can send an email from your application to your own email address, then copy the email headers and paste them into the tool.

Remember, these checks help you understand what’s missing or needs improvement to enhance your email security and reputation, make sure you take note of that and take action. Note: When using online tools, only share what’s needed. Always check the site’s privacy rules to keep your info safe. I’m sharing these tools because they’re helpful, not because I am affiliated with them.

FAQ’s with SPF, DKIM and DMARC

  1. What email address should I use for DMARC reporting? It’s a good idea to use an email address that multiple people can check. This is often a shared mailbox. Ideally, this email address should be from the same domain that you’re setting up DMARC for. If you decide to use an email address from a different domain, you’ll need to add an extra step: You’ll have to add a special record (called a DNS TXT record) to authorize the other domain to receive DMARC reports.

  2. What’s the difference between ~all, -all, ?all, and +all in an SPF record? These are used to tell receiving servers what to do if an email comes from a server that isn’t listed in your SPF record.

    • ~all (SoftFail): This means “It’s okay if the server isn’t on my list, but be aware that it might not be legit.” The email will still be accepted, but it might be marked as suspicious. This is often used when you’re still testing your SPF record or making changes to it.

    • -all (Fail): This means “Only accept emails from servers on my list. Reject everything else.” This is used when you’re sure of all the servers that should be sending emails for your domain.

    • ?all (Neutral): This means “I’m not saying whether servers should be on my list or not. Treat the email as you normally would.” This doesn’t really give any instructions about how to handle the email, so it’s not used very often.

    • +all (Pass): This means “Accept emails from all servers, even if they’re not on my list.” This isn’t recommended because it could allow spammers to send emails that look like they’re from your domain.

    The choice between these depends on how strictly you want to enforce SPF rules for your domain. It’s generally recommended to use ~all while testing or setting up your SPF record, and switch to -all once you are confident that your SPF record is correct.

  3. Can I set up DMARC without SPF? Technically, you can, but it’s not a good idea. DMARC is like a security guard for your emails. It uses two tools, SPF and DKIM, to check if an email is really from you. If an email fails both the SPF and DKIM checks, it also fails the DMARC check. If you set up DMARC without SPF, it’s like the security guard is missing one of its tools. It can still use DKIM to check emails, but it won’t be as effective. SPF isn’t perfect and can’t stop all fake emails on its own. That’s why it’s best to use it together with DKIM and DMARC. This gives you a more complete email security system.

  4. I’ve looked at an email header and I see multiple SPF fails and some SPF passes. Which one should I believe? Think of an email header like a story. The most recent events are at the top, and the oldest events are at the bottom. So, the original sender’s information is usually towards the bottom of the header. If you see multiple SPF fails and a couple of SPF passes, it might feel like the story is getting confusing. But don’t worry! You should trust the SPF check that’s related to your domain or the domains that you trust (like your ‘friends list’). The other SPF checks are for other domains that were part of the email’s journey, and their pass or fail status doesn’t affect your domain’s SPF status.

Wrapping Up

Just like a secret handshake, SPF, DKIM, and DMARC are the hidden heroes of email security. They’re the reason your email recipients can trust messages from your domain. So, the next time you hit ‘send’, remember that these three musketeers are working tirelessly behind the scenes to keep your email safe.

Contributing

Spotted a mistake or missing info in this guide? Don’t be shy! Raise an issue or better yet, fork this repo and raise a PR. Your contributions help make this guide better for everyone.

Sharing is Caring

You’re welcome to share, clone, fork, or bookmark this content. All we ask is that you give credit where it’s due 🙂 “Understanding SPF, DKIM, and DMARC: A Simple Guide” by Nicanor II Flavier, used under CC BY 4.0. To view the original material, visit https://github.com/nicanorflavier/spf-dkim-dmarc-simplified

Contact

If you have any questions or suggestions, feel free to reach out. You can find my contact details on my GitHub profile https://github.com/nicanorflavier

References

Here are some useful resources if you want to learn more about SPF, DKIM, and DMARC:

Courtesy: https://github.com/nicanorflavier/spf-dkim-dmarc-simplified.git