Site Down Alert Documentation

⭐ Why this plugin is unlike anything else available

Proactive site-down notification — not reactive

▸ Other monitoring tools tell you after the site is down. Site Down Alert proactively sends a configured maintenance notification to selected user groups before a planned outage, at the time window you specify, via email, Moodle message, or both.

▸ Alert scheduling is inside Moodle admin — no third-party monitoring service, uptime tool, or external email platform required.

▸ Customisable per-event: message text, affected groups, notification timing (e.g., 24 hours before and 1 hour before), and expected restoration time are all configurable per outage event.

Download Site Down Alert

Early Warning

The Moodle cron task runs every 5 minutes checking for signs of impending failure — high DB latency, memory pressure, disk near-full, cron stalled — and emails you before a crash occurs.

Deep Diagnostics

The standalone server monitor script checks the full stack — HTTP/HTTPS, SSL, DNS, disk, memory, CPU/load, sockets, web server, PHP-FPM, database, Redis, Moodle filesystem, cron/tasks, backups, security, and system health — and is intended to run every 3 minutes via crontab. (The in-Moodle scheduled task runs every 5 minutes.)

Root Cause Alerts

When a site goes down, the alert email names the exact root cause (e.g. PHP-FPM pool exhausted, disk full, SSL expired) with the precise SSH commands to fix it immediately.

How It Works — Two-Part Architecture

Part 1 — Moodle Plugin

Runs inside your Moodle installation as a scheduled task (every 5 minutes by default). It has direct access to Moodle's internals and can detect warning signs that external monitors cannot see:

Database round-trip latency monitoring
PHP memory usage vs ini limits
Moodle dataroot, temp, and sessions writability
Moodle cron health and last-run timestamp
Disk usage on the Moodle data partition
Maintenance mode detection
Lightweight /local/downalert/ping.php health endpoint for UptimeRobot or your own external monitor
Admin health dashboard at /local/downalert/report.php

Part 2 — Server Monitor Script

A standalone PHP script (server-monitor/monitor.php) that runs on your server via crontab — independently of Moodle. Because it runs even when Moodle itself is down, it can diagnose the root cause of a full outage:

Checks every site in its config list every 3 minutes
Runs server-level checks once per cycle and shares results across all sites (no duplicate work)
Performs 3,000+ lines of diagnostic checks across 13 server layers
32 named root-cause conditions with specific diagnosis
Emails you the exact SSH commands to fix the problem
Repeat alert suppression — only emails once until the issue is resolved
Recovery notification when the site comes back up

Everything the Server Monitor Checks

The server monitor script performs checks across every layer of the stack. Server-level checks (disk, CPU, memory, web server, PHP-FPM, MySQL, Redis) run once per cycle regardless of how many sites are monitored — results are shared across all site checks to avoid redundant work.

HTTP / HTTPS Response

Full request timing: DNS lookup, TCP connect, SSL handshake, Time-to-First-Byte (TTFB), total download — every millisecond recorded
Slow TTFB warning (configurable threshold, default 3000 ms)
Redirect loop detection — flags CURLE_TOO_MANY_REDIRECTS
HTTP → HTTPS redirect verification — confirms the plain HTTP URL issues a 301/302/307/308 to the HTTPS version
Response body scanning for Moodle maintenance mode text, PHP fatal errors, parse errors, uncaught exceptions, database connection errors, Moodle error boxes
Blank-page detection — 200 OK with body < 200 bytes means PHP crashed after headers were sent
Security headers audit: HSTS, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Content-Security-Policy
Compression check — confirms gzip or brotli is enabled on the server

SSL Certificate

Subject Alternative Name (SAN) verification — confirms the cert actually covers the monitored hostname, including wildcard matching
Self-signed detection — flags when the certificate issuer equals the subject (not trusted by browsers)
Certificate chain length check
Certificate issuer displayed in diagnostics
Expiry countdown — graded severity: < 21 days = warning, < 7 days = critical, < 0 days = site reported as DOWN

DNS

Forward resolution verification for the monitored domain
PTR reverse-DNS check — confirms the IP resolves back to a known hostname
www subdomain resolution check
DNS response time measurement — warns if DNS is slow (> 1000 ms), which causes slow page loads for all visitors
Record count shown in diagnostics

Disk — All Mounts

Reads /proc/mounts and checks every locally-mounted filesystem — not just the system root or Moodle data directory
Disk usage % per mount with configurable warn/critical thresholds
Inode exhaustion check separately per mount via df -i — a server with plenty of disk space but zero free inodes behaves identically to a full disk
Disk hardware errors scanned from dmesg — I/O errors, SCSI errors, bad sector events all trigger an alert

CPU & Memory

CPU steal % — two /proc/stat reads 500 ms apart for accuracy. High steal means the hypervisor is throttling your VM; a perfectly healthy server will still feel sluggish and time-out
CPU I/O wait % — high iowait means the disk subsystem is the bottleneck
Swap usage % — warns at 50%, critical at 80%. Swap usage on a production server is an early sign of memory pressure
OOM killer events — scanned from dmesg and journalctl -k, shows which processes were killed and how many times since boot
Zombie process count — orphaned processes that cannot be reaped indicate a parent process bug

Network & Sockets

TIME_WAIT count via ss -s (falls back to /proc/net/tcp) — warns above 500. A TIME_WAIT storm means connections are being created faster than TCP's 4-minute timeout recycles them, risking conntrack table exhaustion and connection refused errors
CLOSE_WAIT count — warns above 100. CLOSE_WAIT sockets that never close indicate a PHP or nginx connection leak
ESTABLISHED and SYN_RECV socket counts displayed in diagnostics
Open file descriptor count vs kernel hard limit — ulimit exhaustion causes EMFILE errors and service crashes

Web Server (nginx / Apache)

nginx_status and Apache server-status parsed — active connections, reading/writing/waiting worker counts, requests per second
Configuration syntax test — nginx -t or apache2ctl configtest. A bad config file means the service cannot restart after the next scheduled reload, turning any future config change into a site outage
Worker saturation detection — warns when active workers approach MaxRequestWorkers

PHP-FPM

max_children reached count from the FPM status endpoint — this is the single most reliable indicator of FPM pool exhaustion history. Any non-zero value means the server has already been unable to accept PHP requests
Slow request count from FPM status
OPcache health via php -r: hit rate %, memory usage %, wasted %, JIT enabled status, cached script count
OPcache hit rate below 90% triggers a warning — indicates scripts are not being cached effectively

MySQL / MariaDB

Connection count vs max_connections % — warns at 80%, critical at 95%. Hitting the limit causes 'Too many connections' and Moodle goes down instantly
Threads_running — a sudden spike means queries are piling up, often due to a long-running query blocking others
InnoDB buffer pool hit rate — below 95% means queries are hitting disk instead of RAM, degrading performance significantly
Slow query count since last restart
Table lock wait count — high values indicate MyISAM tables or missing indexes
Aborted connections count — clients disconnecting unexpectedly
MySQL uptime — a recently restarted MySQL is a likely cause of site issues
Replication lag in seconds (IO thread and SQL thread status) for replicated setups

Redis

redis-cli ping — basic connectivity check
Memory used vs max configured vs % utilisation
Connected clients count
Evicted keys count — any evictions mean Redis is under memory pressure and discarding cached data, causing Moodle to fall back to DB for every cache miss
Keyspace hit rate — low hit rate means the cache is not effective

Moodle Filesystem & Config

config.php readable and exists
config.php world-readable security check — if config.php is o+r, any system user can read your database password
config.php size sanity check (must be > 200 bytes, an empty or corrupt config.php is detected)
Moodle dataroot directory: exists, readable, writable
Moodle temp directory: writable
Moodle cache directory: writable
Moodle sessions directory: writable
Moodle filedir: readable
maintenance.html presence confirms the site is intentionally in maintenance mode
.maintenance upgrade lock file detection — Moodle writes this during upgrades; if it persists it blocks all normal page requests

Moodle Cron & Scheduled Tasks

Stuck cron detection — parses ps aux TIME column and flags any cron.php process running longer than 30 minutes
Failed scheduled task count via the Moodle CLI (when available)
Backup recency check — recursively scans the Moodle backup directory for files > 10 KB and reports the newest file age. Warns if the latest backup is older than 7 days, critical if older than 30 days

Security & System Health

fail2ban process running — confirms brute-force protection is active
SSH authentication failure count today — from /var/log/auth.log or /var/log/secure. Sudden spikes indicate an active attack
UFW / iptables firewall active check
System uptime — recent reboots flag a likely cause for service interruptions
NTP synchronisation — tries timedatectl, ntpstat, and chronyc in order. Gets offset in ms. NTP drift causes SSL certificate errors, cron timing issues, and TOTP authentication failures

32 Named Root-Cause Diagnoses

When a site is down or degraded, the monitor identifies the primary root cause from the list below and includes it prominently in the alert email along with specific SSH commands to resolve it. You'll know the exact problem within 3 minutes of it occurring.

TIMEOUT

Site did not respond within the timeout window. Could be full server load, network issue, or service crash.

SSL_EXPIRED

SSL certificate has passed its expiry date. All browsers will block access.

SSL_EXPIRING_SOON

Certificate expires within 7 days. Auto-renewal may have failed.

SSL_HOSTNAME_MISMATCH

Certificate SAN does not cover the monitored domain. Common after moving to a new certificate or domain.

DNS_FAILURE

The domain does not resolve. Likely a DNS configuration error or expired domain.

DNS_SLOW

DNS response time exceeded 1000 ms. Visible to all visitors as slow page loads.

CONNECTION_REFUSED

TCP connection rejected. nginx/Apache is not running or is bound to a different port.

HTTP_500

Server returned a 500 Internal Server Error. Usually a PHP fatal error or misconfiguration.

HTTP_503

Service unavailable. PHP-FPM, the upstream application server, or the database is down.

PHP_FATAL_IN_BODY

HTTP 200 was returned but the body contains a PHP fatal/parse error. Headers were sent before the crash.

MOODLE_MAINTENANCE

Moodle maintenance mode is active. Either intentional or a stuck upgrade.

MOODLE_UPGRADE_LOCK

.maintenance lock file present. An upgrade or migration is in progress or got stuck.

DB_CONNECTIONS_FULL

MySQL connections at or near max_connections. Moodle will get 'Too many connections' on next request.

INNODB_BUFFER_DEGRADED

InnoDB buffer pool hit rate below 95%. Moodle queries are hitting disk on most requests.

MYSQL_REPLICATION_LAG

Replication lag detected on a replica server. Reads may return stale data.

REDIS_DOWN

Redis did not respond to ping. Moodle sessions and caching fall back to the database.

REDIS_MEMORY_PRESSURE

Redis is evicting keys. The cache is full and discarding data — cache miss rate will spike.

PHP_FPM_POOL_EXHAUSTED

PHP-FPM max_children was reached. Moodle was unable to serve requests; visitors received 502/504 errors.

OPCACHE_DEGRADED

OPcache hit rate below 90%. PHP scripts are being recompiled on every request.

DISK_FULL

A disk mount is critically low on space. Moodle cannot write session data, logs, or uploaded files.

INODE_EXHAUSTED

A filesystem has run out of inodes. The disk may have free space but no new files can be created.

DISK_HARDWARE_ERROR

I/O errors or bad sectors detected in dmesg. Risk of data loss — check the disk immediately.

SWAP_CRITICAL

Swap usage above 80%. System is under severe memory pressure and will eventually crash or kill processes.

OOM_KILL

The Linux OOM killer has terminated one or more processes. Check which process was killed.

CPU_STEAL_HIGH

High CPU steal from the hypervisor. Your VM is being throttled by the host — contact your cloud provider.

CPU_IOWAIT_HIGH

High I/O wait. Disk is the bottleneck — consider upgrading to SSD or reducing disk I/O.

TIME_WAIT_STORM

Excessive TIME_WAIT sockets. Connection creation rate exceeds TCP's recycle speed.

CLOSE_WAIT_LEAK

Many CLOSE_WAIT sockets. Likely a PHP or nginx connection leak — consider restarting the service.

MOODLE_CRON_STUCK

A Moodle cron.php process has been running for over 30 minutes. May be blocking other tasks.

MOODLE_TASKS_FAILED

Moodle scheduled tasks have recorded failures. Check Site admin → Server → Scheduled tasks.

NTP_NOT_SYNCED

System clock is not synchronised. Can cause SSL errors, TOTP failures, and cron timing issues.

BACKUP_STALE

No recent Moodle backup found. Backup job may have failed silently — review scheduled tasks.

Installation

Step 1

Install the Moodle Plugin

1.Download the plugin ZIP from the button above.
2.In your Moodle site: Site administration → Plugins → Install plugins.
3.Upload the ZIP file, click Install plugin from the ZIP file, then follow the on-screen upgrade steps.
4.After installation completes, Moodle will redirect you to the plugin settings. Configure your alert email address before leaving.

Step 2

Configure the Moodle Plugin Settings

Navigate to: Site administration → Plugins → Local plugins → Site Down Alert

Settings Reference

Alert email address

alertemail

The email address that receives all warning and down alerts. Can be a distribution list. Required — no alerts will be sent without this.

Enable monitoring

enabled

Master switch. When disabled, the cron task runs but does not send any emails.

Check interval

checkinterval

How often the Moodle cron task runs checks, in minutes. Default: 5 minutes. Lower values give faster early warnings but add more cron load.

Database latency threshold (ms)

dblatency

Milliseconds above which a DB round-trip is flagged as high latency. Default: 500 ms. If your DB is slower than this, a warning email is sent before performance degrades further.

Disk usage warning threshold (%)

diskwarn

Disk usage percentage that triggers a warning email. Default: 80%.

Disk usage critical threshold (%)

diskcritical

Disk usage percentage that triggers a critical email. Default: 90%. At this level Moodle may start failing file uploads and session writes.

Memory usage threshold (%)

memorywarn

PHP memory usage (as % of memory_limit) that triggers a warning. Default: 80%.

Cron stale threshold (minutes)

cronstale

How long Moodle's cron can go without running before a warning is sent. Default: 15 minutes. Moodle's cron must run at least every few minutes for scheduled tasks to execute on time.

From email address

fromemail

The sender address on alert emails. Defaults to the Moodle no-reply address. Setting this to a monitored mailbox makes replies easy.

Step 3

Set Up the Server Monitor Script

The server monitor script is included in the plugin ZIP at server-monitor/monitor.php. It runs independently of Moodle via a server crontab. This is what detects full outages and provides root-cause diagnosis.

1. Copy the script to your server

# Extract and place the monitor script

sudo mkdir -p /opt/site-monitor

sudo cp monitor.php /opt/site-monitor/monitor.php

sudo chmod 750 /opt/site-monitor/monitor.php

2. Create the configuration file

Create /opt/site-monitor/config.php with your site list and SMTP settings:

<?php

// Sites to monitor

$config['sites'] = [

'https://yourmoodle.edu.au',

'https://learning-backup.yourrto.com.au',

];

// Alert email (where to send alerts)

$config['alert_email'] = 'admin@yourrto.com.au';

// SMTP settings for sending alerts

$config['smtp_host'] = 'smtp.yourdomain.com';

$config['smtp_port'] = 587;

$config['smtp_user'] = 'alerts@yourdomain.com';

$config['smtp_pass'] = 'your-smtp-password';

$config['smtp_from'] = 'alerts@yourdomain.com';

$config['smtp_from_name']= 'Site Monitor';

// Optional: MySQL credentials for DB checks

// Leave blank to use socket auth (if running as mysql user)

$config['mysql_host'] = 'localhost';

$config['mysql_user'] = 'monitor_user';

$config['mysql_pass'] = 'monitor_password';

// Timeouts

$config['http_timeout'] = 30; // seconds

$config['ttfb_warn_ms'] = 3000; // warn if TTFB > 3s

// Disk thresholds (%)

$config['disk_warn'] = 80;

$config['disk_critical'] = 90;

// Swap thresholds (%)

$config['swap_warn'] = 50;

$config['swap_critical'] = 80;

// State directory (for repeat-alert suppression)

$config['state_dir'] = '/opt/site-monitor/state';

3. Create the MySQL monitoring user (optional but recommended)

If you want the script to check MySQL connection counts, InnoDB buffer pool health, and replication lag, create a read-only monitoring user:

-- Run in MySQL as root

CREATE USER 'monitor_user'@'localhost' IDENTIFIED BY 'monitor_password';

GRANT PROCESS, REPLICATION CLIENT ON *.* TO 'monitor_user'@'localhost';

FLUSH PRIVILEGES;

4. Add to crontab

Add the monitor to your server's crontab to run every 3 minutes. Run as a user with access to ss, dmesg, and other system tools — ideally a dedicated monitor user with the necessary sudo rules, or root.

# Open crontab as root (or your monitor user)

sudo crontab -e

# Add this line:

*/3 * * * * /usr/bin/php /opt/site-monitor/monitor.php >> /var/log/site-monitor.log 2>&1

Important: The script adds ~0.5 seconds to its runtime due to the double /proc/stat read used for accurate CPU steal measurement. This is expected and intentional.

5. Create the state directory

The script uses a state directory to track which sites are currently down, so it doesn't send repeat alerts every 3 minutes for an ongoing outage. It also sends a recovery notification when a site comes back up.

sudo mkdir -p /opt/site-monitor/state

sudo chmod 700 /opt/site-monitor/state

6. Test manually before activating the cron

# Run once manually to see output and verify SMTP works

sudo php /opt/site-monitor/monitor.php

UptimeRobot / External Monitor Integration

The Moodle plugin exposes a lightweight health endpoint at /local/downalert/ping.php. This endpoint requires no authentication and returns a JSON status report with HTTP 200 when the site is healthy, or HTTP 503 when it is degraded. You can point UptimeRobot, Freshping, Better Uptime, or any HTTP monitor at this URL.

// Healthy response (HTTP 200)

{

"status": "ok",

"checks": {

"database": {"ok": true, "latency_ms": 12},

"disk": {"ok": true, "usage_pct": 64},

"memory": {"ok": true, "usage_pct": 71},

"cron": {"ok": true, "last_run_minutes_ago": 3},

"dataroot": {"ok": true, "writable": true},

"maintenance": {"ok": true, "active": false}

}

When any check fails, the endpoint returns HTTP 503 with a "status": "degraded" or "status": "critical" field, which will trigger your UptimeRobot alert.

Recommended UptimeRobot setup: Set the monitor type to "Keyword" and check that the response body contains "status":"ok". This catches cases where the server returns HTTP 200 but the body indicates a degraded state.

Accessing Settings and the Health Report in Moodle

Plugin Settings

To access the plugin settings in Moodle:

Site administration
→ Plugins
→ Local plugins
→ Site Down Alert

Or navigate directly to:

/admin/settings.php?section=local_downalert

Here you set your alert email, enable/disable monitoring, and configure all thresholds.

Health Report Dashboard

The health report shows the current status of all monitored checks in real time. Access it at:

/local/downalert/report.php

Or from within Moodle:

Site administration
→ Reports
→ Site Down Alert

The report shows a colour-coded table of every check (OK / Warning / Critical) with the current measured value and the threshold that was applied.

Alert Email Format

Alert emails are structured for immediate action. They include:

Down Alert Email Contains:

Site URL that is down
Time detected and duration down
Primary root-cause code (e.g. PHP_FPM_POOL_EXHAUSTED)
Plain-English explanation of the root cause
Exact SSH commands to investigate and fix the specific issue
Summary of all server health checks that passed and failed
HTTP timing breakdown (DNS, connect, handshake, TTFB)

Warning & Recovery Emails:

Warning emails are sent when a site is up but checks are in warning state (e.g. disk at 83%, TTFB slow, swap high)
Recovery emails are sent when a previously-down site responds successfully again
Repeat suppression — once a down alert is sent, no further down alerts are sent for that site until it recovers, preventing inbox flooding during extended outages
Subject lines clearly labelled: [DOWN], [WARNING], [RECOVERED]

Requirements

Moodle Plugin

Supported Moodle and PHP versions — see the archive-bound Requirements for this release
Moodle cron running (required for scheduled tasks)
No per-use AI credits required; access is acquired through the LMS Labs credit balance
No external API dependencies

Server Monitor Script

PHP CLI version supported by this release (see archive-bound Requirements)
PHP cURL extension (for HTTP checks)
Linux server (uses /proc/stat, /proc/mounts, ss, dmesg)
SMTP credentials for sending alert emails
MySQL monitoring user (optional, for DB checks)
Write access to state directory for alert suppression

Frequently Asked Questions

Do I need both parts? Can I use just the Moodle plugin?

Yes — each part is independent. The Moodle plugin alone gives you early warning and the ping.php health endpoint for UptimeRobot. The server monitor script adds deep diagnostic capability and root-cause analysis for full outages. Most sites benefit from both: the plugin catches problems early (before the site goes down), the script diagnoses them accurately when it does go down.

What happens if the server crashes completely? Can the monitor still run?

The server monitor script runs via crontab on the server. If the server itself goes down (kernel panic, power failure, OOM kill of the cron daemon), the script cannot run. For true external monitoring, use UptimeRobot or a similar service pointed at the ping.php endpoint. The server monitor script excels at diagnosing problems where the server is up but Moodle is down — which is the vast majority of outages.

How does repeat-alert suppression work?

When a site goes down, the monitor writes a state file to the configured state directory. On subsequent runs, if the site is still down and a state file exists, no further email is sent. When the site recovers, the state file is removed and a recovery email is sent. This means you get exactly one down alert and one recovery alert per outage, regardless of duration.

Will the MySQL checks work without credentials?

If you are running the monitor as the system mysql user or as root with socket authentication enabled, the script will attempt a socket connection without credentials. For most setups, creating a dedicated read-only monitoring user with PROCESS and REPLICATION CLIENT grants is the recommended approach.

The CPU steal check takes longer than other checks — why?

CPU steal percentage is calculated by reading /proc/stat twice with a 500 ms pause between reads. This pause is required to get an accurate measurement of the steal over a real time window rather than a cumulative average since boot. The total overhead is approximately 0.5 seconds per monitor run.

Can I monitor multiple sites on different servers?

The server monitor script monitors multiple sites listed in its config but runs server-level checks on the local machine only. To monitor sites on different servers, deploy a copy of the script on each server. The HTTP checks (which run for each site URL) can check any reachable URL from the script's server.

How do I configure the Moodle cron task frequency?

After installation, go to Site administration → Server → Scheduled tasks. Find 'Site Down Alert monitoring check' in the list and click the edit icon to change its schedule. The default is every 5 minutes (*/5 * * * *). You can make it more frequent for critical sites, but note that running it every minute on a large Moodle site will add more load.

Does the plugin require AI Grader Central Config (local_aiconfig)?

No. Site Down Alert is fully standalone and requires no AI credits, no API key, and no Central Config plugin. It works entirely with your Moodle database and PHP environment.