How to Troubleshoot Failed Systemd Services on Linux

Systemd is the service manager used by many modern Linux distributions, including Ubuntu, Debian, AlmaLinux, Rocky Linux, and other Linux-based operating systems. It manages system services, starts processes during boot, and helps administrators monitor whether applications are running correctly.

Sometimes, a Linux service may fail to start, stop unexpectedly, or enter a failed state after a system update or configuration change. This can affect websites, databases, web servers, and other applications running on a VPS.

In this guide, you will learn how to troubleshoot failed systemd services on Linux, inspect service logs, identify common errors, and restore services safely.

What Is a Failed Systemd Service?

A systemd service is a background process managed by the systemd service manager. Examples include Nginx, Apache, MySQL, MariaDB, and SSH.

When a service cannot start or exits unexpectedly, systemd may mark it as failed.

Common reasons include:

• Incorrect configuration files.
• Missing dependencies or required files.
• Permission or ownership problems.
• Port conflicts with another application.
• Insufficient disk space or system resources.
• Incorrect service definitions.
• Application crashes or failed software updates.

Identifying the exact error message is the first step toward resolving the problem.

1. Check the Status of the Failed Service

Start by checking the status of the service you want to troubleshoot.

For example, to inspect Nginx:

sudo systemctl status nginx

For Apache on Ubuntu or Debian:

sudo systemctl status apache2

For Apache on AlmaLinux, Rocky Linux, or similar distributions:

sudo systemctl status httpd

The output typically shows whether the service is active, inactive, or failed. It may also include the process exit code and recent log messages.

Example output:

● nginx.service - A high performance web server
     Loaded: loaded
     Active: failed (Result: exit-code)

This indicates that the service failed, but it does not necessarily explain the underlying cause. Continue by checking the service logs.

2. View Systemd Logs for the Failed Service

Systemd records service events and errors in the system journal. You can use journalctl to inspect those messages.

To view recent logs for Nginx:

sudo journalctl -u nginx -n 100 --no-pager

Replace nginx with the name of the service you are troubleshooting.

To view logs from the current boot:

sudo journalctl -u nginx -b --no-pager

To display only higher-priority messages:

sudo journalctl -u nginx -p warning --no-pager

Look for messages indicating:

• Configuration syntax errors.
• Missing files or directories.
• Permission denied errors.
• Address or port conflicts.
• Failed dependencies.
• Repeated process crashes.

For example, an error indicating that a configuration file cannot be parsed suggests that you should validate the application’s configuration before attempting another restart.

3. Verify the Service Name

Sometimes, troubleshooting fails simply because the wrong service name was used.

List services that are currently failed:

systemctl --failed

To list active services:

systemctl list-units --type=service --state=active

To find installed service unit files, including services that are not currently running:

systemctl list-unit-files --type=service

If you are unsure about a service’s exact name, filter the output. For example:

systemctl list-unit-files --type=service | grep -i mysql

Remember that the service name can vary by distribution and software package. Apache, for example, commonly uses apache2 on Ubuntu and Debian, but httpd on many RHEL-based systems.

4. Check the Service Configuration

A configuration error is one of the most common reasons an application service refuses to start.

Before restarting a failed service, validate its configuration using the appropriate application-specific command.

For Nginx:

sudo nginx -t

For Apache on Ubuntu or Debian:

sudo apache2ctl configtest

For Apache on AlmaLinux or Rocky Linux:

sudo httpd -t

If the configuration test reports an error, correct the relevant file before restarting the service.

For example, an invalid directive, missing configuration file, or malformed virtual host definition may prevent a web server from starting.

Important: Not every systemd service has a dedicated configuration validation command. Consult the relevant application’s documentation when troubleshooting services other than Nginx or Apache.

5. Check for Port Conflicts

A service may fail to start if another process is already listening on the required port.

For example, web servers commonly use TCP ports 80 and 443, while SSH typically uses port 22.

To inspect listening ports:

sudo ss -lntp

To check which process is using port 80:

sudo ss -lntp 'sport = :80'

If another process is using the port required by your service, identify that process before taking action.

Do not terminate processes blindly. The existing listener may belong to another legitimate application or web server.

If the port conflict is caused by an incorrectly configured service, correct its configuration or adjust the intended port assignment before trying again.

6. Check File Permissions and Required Files

A service can fail when it cannot read its configuration, access a required directory, or execute its program.

Check the permissions of relevant files and directories:

ls -l /path/to/file

Replace /path/to/file with the actual path reported in the logs.

For example, a service may be unable to read a configuration file because its ownership or permissions are incorrect.

Before changing permissions, confirm which user runs the service and what access it requires. Avoid applying broad permissions such as chmod -R 777, which can create serious security problems.

Also check whether required files exist:

sudo test -f /path/to/file && echo "File exists"

If the logs identify a missing file, determine whether it was deleted, moved, or referenced incorrectly in the configuration.

7. Check Dependencies and Related Services

Some services require other services or system components to be available before they can start.

For example, an application may depend on a database, a mounted filesystem, or a network service.

Inspect the unit’s dependency information:

systemctl list-dependencies nginx.service

Replace nginx.service with the appropriate unit name.

Check the status of a suspected dependency:

sudo systemctl status mysql

The dependency’s name may differ depending on the distribution and installed database package.

If the required service has failed, investigate and fix that service first. Restarting the original application repeatedly will not solve a problem caused by an unavailable dependency.

8. Restart the Service After Fixing the Problem

Once you have identified and corrected the underlying issue, restart the service.

For Nginx:

sudo systemctl restart nginx

For Apache on Ubuntu or Debian:

sudo systemctl restart apache2

For Apache on AlmaLinux or Rocky Linux:

sudo systemctl restart httpd

Then check the status again:

sudo systemctl status nginx

Confirm that the service is active and review the logs for any new errors.

If the service is still failing, return to the logs and investigate the new error message rather than repeating the same restart command.

9. Reset the Failed State When Necessary

Systemd may continue to show a service as failed after the underlying problem has been fixed.

You can clear the recorded failed state with:

sudo systemctl reset-failed nginx

Replace nginx with the appropriate service name.

This command clears systemd’s recorded failed state and relevant start-rate-limit counters. It does not repair the application, correct a configuration error, or start the service automatically.

After resetting the state, start the service if necessary:

sudo systemctl start nginx

If the service fails again, inspect its status and logs to determine why.

10. Check Whether the Service Starts Automatically

If a service works when started manually but does not start after a reboot, check whether it is enabled.

Run:

systemctl is-enabled nginx

If the service should start automatically at boot, enable it:

sudo systemctl enable nginx

To enable and start the service immediately:

sudo systemctl enable --now nginx

Only enable services that are intended to run on your server. Some services are deliberately configured to start on demand or only under particular conditions.

11. Troubleshoot a Custom Systemd Service

If you run a custom application or script, its systemd unit file may contain an incorrect executable path, working directory, user, or environment configuration.

Show the service definition:

systemctl cat myapp.service

Replace myapp.service with your actual service name.

Pay particular attention to directives such as:

• ExecStart: the command systemd runs.
• WorkingDirectory: the directory in which the application runs.
• User and Group: the account used to run the process.
• EnvironmentFile: an optional file containing environment variables.
• Restart: the configured restart behavior.

After editing a unit file, reload systemd’s configuration:

sudo systemctl daemon-reload

Then restart the service:

sudo systemctl restart myapp.service

If you changed only an application configuration file and not the unit file, daemon-reload is usually unnecessary. Follow the application’s own reload or restart procedure.

12. Investigate Resource or Operating System Problems

Sometimes, the service configuration is correct, but the service cannot start because of a wider system problem.

Check available disk space:

df -h

Check memory usage:

free -h

Review recent kernel messages:

sudo journalctl -k -b -p warning --no-pager

Look for evidence of filesystem errors, out-of-memory events, or other system-level problems.

If the VPS is out of disk space, investigate which files are consuming storage before deleting anything. If memory pressure is severe, identify the processes responsible rather than terminating critical services without understanding their purpose.

For a VPS that is unresponsive, repeatedly rebooting may not address the underlying cause. Use the available console or recovery tools to investigate the operating system when normal access is unavailable.

Common Systemd Troubleshooting Commands

Command	Purpose
systemctl status SERVICE	Check service status
systemctl --failed	List failed units
journalctl -u SERVICE -n 100	View recent service logs
systemctl restart SERVICE	Restart a service
systemctl reset-failed SERVICE	Clear recorded failed state
systemctl is-enabled SERVICE	Check boot-time enablement
systemctl cat SERVICE	Inspect the unit definition
systemctl daemon-reload	Reload systemd unit definitions
df -h	Check filesystem space
free -h	Check memory usage

Replace SERVICE with the relevant service name. Most administrative troubleshooting commands require sudo or root privileges.

Manage Your Applications on a Reliable VPS

Reliable server infrastructure is important when running websites, databases, APIs, and background applications. Understanding how to troubleshoot systemd services helps you diagnose problems and restore services without unnecessary restarts or risky system changes.

If you are looking for a VPS for your projects, explore the available options from ServerNext:

• USA VPS Server
• BDIX VPS Server
• Singapore VPS Server
• Germany NVMe VPS Server
• Offshore VPS Server

Visit ServerNext to explore the available VPS solutions for your requirements.