NIST-800.53-VM/README.md

# NIST 800-53 Hardened Ubuntu 24.04 - Proxmox VM Template

A hardened Ubuntu 24.04 LTS VM template for Proxmox VE, aligned with NIST 800-53 security controls and validated against CIS Ubuntu 24.04 benchmarks via Wazuh SCA.

The template is built through three scripts run in sequence on a fresh Ubuntu 24.04 Server (minimal) installation. Once complete, the VM is converted to a Proxmox template for rapid, secure clone deployment.

---

## Table of Contents

- [Architecture Overview](#architecture-overview)
- [Prerequisites](#prerequisites)
- [VM Creation in Proxmox](#vm-creation-in-proxmox)
- [SSH Key Setup (Windows)](#ssh-key-setup-windows)
- [Script 1 - partition-layout.sh](#script-1--partition-layoutsh)
- [Script 2 - nist-800-53-harden-v2.sh](#script-2--nist-800-53-harden-v2sh)
- [Script 3 - remediate.sh](#script-3--remediatesh)
- [Deployment Workflow](#deployment-workflow)
- [Post-Template: Cloning](#post-template-cloning)
- [Firewall Management](#firewall-management)
- [NIST 800-53 Control Mapping](#nist-800-53-control-mapping)
- [Wazuh SCA Check Coverage](#wazuh-sca-check-coverage)
- [Known Exceptions](#known-exceptions)
- [Maintenance](#maintenance)
- [Troubleshooting](#troubleshooting)

---

## Architecture Overview

```
┌──────────────────────────────────────────────────────────┐
│                    Proxmox VM                            │
├──────────────┬───────────────────────────────────────────┤
│  Disk 1      │  Disk 2 (LVM - vg_nist)                   │
│  /dev/sda    │  /dev/sdb                                 │
│  64 GB       │  50 GB                                    │
│              │                                           │
│  sda1: EFI   │  lv_var ............ /var        (10 GB)  │
│  sda2: /boot │  lv_var_log ........ /var/log     (8 GB)  │
│  sda3: LVM   │  lv_var_log_audit .. /var/log/audit (4 GB)│
│    └─ /      │  lv_home ........... /home        (8 GB)  │
│              │  lv_tmp ............ /tmp         (2 GB)  │
│              │  lv_var_tmp ........ /var/tmp     (2 GB)  │
│              │  [~16 GB free for future growth]          │
├──────────────┴───────────────────────────────────────────┤
│  Cloud-Init Drive (auto-injected ISO on /dev/sr0)        │
└──────────────────────────────────────────────────────────┘
```

Each mount point carries restrictive options:

| Mount Point       | Options                          |
|-------------------|----------------------------------|
| `/home`           | `nosuid,nodev`                   |
| `/tmp`            | `nosuid,nodev,noexec`            |
| `/var`            | `nosuid,nodev`                   |
| `/var/tmp`        | `nosuid,nodev,noexec`            |
| `/var/log`        | `nosuid,nodev,noexec`            |
| `/var/log/audit`  | `nosuid,nodev,noexec`            |
| `/dev/shm`        | `nosuid,nodev,noexec`            |

---

## Prerequisites

- Proxmox VE 8.x or later
- Ubuntu 24.04 LTS Server ISO (minimal install)
- An SSH key pair on your admin workstation (see [SSH Key Setup](#ssh-key-setup-windows))
- Wazuh agent (optional, for compliance scanning - the scripts pre-configure outbound firewall rules for Wazuh on ports 1514/1515)

---

## VM Creation in Proxmox

Create the VM with these settings before installing Ubuntu:

**System tab:**

- Machine: `q35`
- BIOS: `OVMF (UEFI)` with EFI Disk, pre-enroll keys checked (Secure Boot)
- TPM: `v2.0`
- SCSI Controller: `VirtIO SCSI single`
- Qemu Agent: checked

**Disks:**

- Disk 1 (OS): SCSI, 20–64 GB, Discard checked, IO Thread checked
- Disk 2 (LVM): Added after OS install - SCSI, 50 GB, Discard checked, IO Thread checked

**CPU:** Type `host` (exposes AES-NI), 2+ cores

**Memory:** 2048 MB minimum, Ballooning checked

**Network:** VirtIO (paravirtualized), Firewall checked

**After OS installation, add:**

- The second 50 GB disk (Hardware → Add → Hard Disk)
- A Cloud-Init drive (Hardware → Add → CloudInit Drive)

**Cloud-Init tab:**

- User: your admin username (e.g., `chris`)
- SSH public key: paste your `id_ed25519.pub`
- IP Config: DHCP or static
- DNS: your internal servers

---

## SSH Key Setup (Windows)

Generate an Ed25519 key pair from PowerShell on your admin workstation:

```powershell
ssh-keygen -t ed25519 -C "admin@proxmox"
```

Accept the default path (`C:\Users\YourName\.ssh\id_ed25519`) and set a strong passphrase.

Copy the public key:

```powershell
cat ~/.ssh/id_ed25519.pub
```

This one public key is used for all VMs cloned from the template. Paste it into both the Cloud-Init tab in Proxmox and (during initial setup) the template VM's `~/.ssh/authorized_keys`:

```bash
mkdir -p ~/.ssh && chmod 700 ~/.ssh
echo "ssh-ed25519 AAAA...your-key..." >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
```

**Optional - enable the SSH agent** so you don't retype the passphrase every session:

```powershell
# One-time (run as Admin)
Get-Service ssh-agent | Set-Service -StartupType Automatic -PassThru | Start-Service

# Then from regular PowerShell
ssh-add ~/.ssh/id_ed25519
```

---

## Script 1 - partition-layout.sh

**Purpose:** Migrates a default single-partition Ubuntu install into a NIST/CIS-compliant LVM layout with six separate mount points on the second disk.

**When to run:** After Ubuntu install, after attaching the second disk, before the hardening script.

```bash
chmod +x partition-layout.sh
sudo ./partition-layout.sh /dev/sdb
```

**What it does:**

1. Installs LVM2 if missing
2. Creates a GPT partition and LVM volume group (`vg_nist`) on the second disk
3. Creates six logical volumes with ext4 filesystems
4. Migrates existing data from `/var`, `/home`, `/tmp`, etc. using rsync
5. Writes fstab entries with NIST-compliant mount options
6. Mounts everything and sets correct permissions (sticky bits, directory modes)
7. Leaves ~16 GB free in the volume group for future growth

**Post-run:** Reboot and verify all six volumes mount automatically:

```bash
sudo reboot
# After reboot:
df -h | grep vg_nist
findmnt -t ext4 | grep vg_nist
```

**Growing a volume later:**

```bash
sudo lvextend -L +5G /dev/vg_nist/lv_var_log
sudo resize2fs /dev/vg_nist/lv_var_log
```

**Known issue:** The script may fail to mount `/var/log/audit` because the mount point doesn't exist on the freshly mounted `/var/log` volume. If this happens:

```bash
sudo mkdir -p /var/log/audit
sudo mount /var/log/audit
sudo chmod 700 /var/log/audit
```

---

## Script 2 - nist-800-53-harden-v2.sh

**Purpose:** Comprehensive OS hardening covering NIST 800-53 controls and CIS Ubuntu 24.04 benchmarks. Uses iptables (not UFW) for Docker compatibility and clean compliance scans.

**When to run:** After partition-layout.sh and a successful reboot confirming all mounts.

```bash
# Optional: set admin username (defaults to chris)
export ADMIN_USER="chris"

chmod +x nist-800-53-harden-v2.sh
sudo ./nist-800-53-harden-v2.sh
```

**What it does (by NIST control family):**

**Access Control (AC):**

- Locks root account, removes unnecessary users
- Configures pam_faillock (5 attempts / 15-min lockout)
- Sets login warning banners (`/etc/issue`, `/etc/issue.net`, `/etc/motd`)
- 15-minute idle session timeout (TMOUT with readonly guard)
- SSH: key-only auth, AllowUsers, PermitRootLogin no, DisableForwarding

**Audit and Accountability (AU):**

- Comprehensive auditd rules covering CIS checks 35731–35748 (sudoers, user emulation, time changes, network changes, file access, DAC modifications, mounts, deletions, MAC policy, chcon/setfacl/chacl/usermod, kernel modules)
- Audit log retention: `keep_logs`, space warnings, admin halt on full
- Chrony NTP (systemd-timesyncd disabled and masked)
- Journald: persistent, compressed, rotated, size-limited
- Rsyslog: file creation mode 0640, remote forwarding placeholder

**Identification and Authentication (IA):**

- Password quality: minlen=15, minclass=4, maxsequence=3, maxrepeat=3
- Password aging: 60-day max, 1-day min, 14-day warning, 30-day inactive lock
- PAM: faillock, pwhistory (remember=24, enforce_for_root), nullok removed
- opasswd file created with strict permissions

**System and Communications Protection (SC):**

- iptables firewall with default deny on all chains
- ICMP type-filtered with rate-limited ping (1/sec, burst 4)
- SSH rate-limited (4 new/min per source IP)
- Outbound allowed: DNS (53), HTTP (80), HTTPS (443), NTP (123), Wazuh (1514/1515)
- DOCKER-USER chain pre-created for Docker compatibility
- Rules persisted via iptables-persistent
- SSH cryptography: Ed25519/RSA-SHA2 host keys, Curve25519/AES-GCM ciphers, SHA2-ETM MACs
- /dev/shm hardened with noexec,nosuid,nodev

**System and Information Integrity (SI):**

- Kernel hardening: ASLR, ptrace restriction, BPF hardening, core dumps disabled, dmesg restricted
- AIDE file integrity monitoring with audit tool coverage
- Unattended security upgrades enabled
- AppArmor enforced and added to GRUB command line
- `audit=1 audit_backlog_limit=8192` added to GRUB

**Configuration Management (CM):**

- Unnecessary services disabled and masked
- Filesystem modules blacklisted (afs, ceph, cifs, exfat, fat, fscache, fuse, gfs2, nfs, nfsd, smbfs, cramfs, squashfs, etc.)
- Network protocol modules blacklisted (dccp, sctp, rds, tipc)
- USB storage disabled
- Telnet, FTP, rsync, nftables, UFW, apport, snapd removed
- File permissions hardened, cron/at restricted, su restricted to sudo group
- Sudo I/O logging enabled

**Template Preparation:**

- QEMU guest agent enabled
- Cloud-init configured for Proxmox (NoCloud/ConfigDrive)
- Machine-id truncated (regenerated on clone)
- SSH host keys removed (regenerated on first boot via firstboot-harden.service)
- initramfs rebuilt, apt cache cleaned, logs truncated

**Post-run:** AIDE database initialization takes 10–15 minutes. After the script completes, reboot:

```bash
sudo reboot
```

---

## Script 3 - remediate.sh

**Purpose:** Post-scan remediation script that fixes remaining Wazuh SCA failures after the initial hardening. Addresses specific check IDs with targeted fixes.

**When to run:** After the hardening script and a Wazuh SCA scan reveals remaining failures. Can also be run standalone on an already-hardened system.

> **Note:** This script was written during the initial hardening iteration and uses UFW for firewall rules (sections 35619–35639). If you ran `nist-800-53-harden-v2.sh` (which uses iptables and removes UFW), the UFW sections will be skipped harmlessly since UFW won't be installed. The iptables rules from the v2 hardening script satisfy the same controls.

```bash
chmod +x remediate.sh
sudo ./remediate.sh
```

**What it fixes (by Wazuh check ID):**

| Check ID    | Description                                        |
|-------------|----------------------------------------------------|
| 35509       | Filesystem kernel modules blacklisted              |
| 35522       | `nodev` added to `/var` mount                      |
| 35537       | AppArmor enabled in GRUB command line              |
| 35545       | Apport disabled, masked, and purged                |
| 35573       | rsync removed                                      |
| 35585       | telnet removed                                     |
| 35587       | ftp removed                                        |
| 35589       | systemd-timesyncd masked (chrony is NTP source)    |
| 35600       | `/etc/cron.allow` created                          |
| 35601       | `/etc/at.allow` created                            |
| 35604–35607 | dccp, sctp, rds, tipc modules blacklisted          |
| 35619–35639 | Firewall fully configured (UFW in remediate.sh, iptables in v2) |
| 35641–35642 | SSH host key permissions set                       |
| 35643–35661 | Complete SSH hardening rewrite                     |
| 35664       | Sudo logfile configured                            |
| 35668       | `su` restricted to sudo group via pam_wheel        |
| 35672–35690 | Full PAM stack rewrite (common-auth, common-account, common-password, common-session) |
| 35681/35683 | Password quality: minclass, maxsequence added      |
| 35694–35698 | Password aging and inactive lock on all users      |
| 35703       | Root umask set in `.bashrc` and `.bash_profile`    |
| 35708       | Journald rotation configured                       |
| 35719       | Rsyslog file creation mode 0640                    |
| 35722       | Log file permissions tightened                     |
| 35725–35726 | `audit=1 audit_backlog_limit=8192` in GRUB         |
| 35728–35730 | auditd log retention: keep_logs, space alerts      |
| 35731–35748 | Complete CIS-compliant audit ruleset               |
| 35752       | Audit config file permissions (640)                |
| 35755       | Audit tool permissions                             |
| 35760       | AIDE integrity rules for audit tools               |
| 35770       | opasswd/opasswd.old permissions                    |

---

## Deployment Workflow

The full sequence from start to template:

```
1.  Create VM in Proxmox (settings above)
2.  Install Ubuntu 24.04 Server (minimal)
3.  Attach second 50 GB disk in Proxmox Hardware tab
4.  Add Cloud-Init drive in Proxmox Hardware tab
5.  Boot VM, log in via Proxmox console (noVNC)
6.  Add your SSH public key to ~/.ssh/authorized_keys
7.  Copy scripts to the VM
8.  Run:  sudo ./partition-layout.sh /dev/sdb
9.  Reboot, verify mounts with df -h
10. Run:  sudo ./nist-800-53-harden-v2.sh
11. Reboot, verify SSH access with key auth
12. (Optional) Install Wazuh agent, run SCA scan
13. (Optional) Run:  sudo ./remediate.sh
14. Reboot, re-scan, review results
15. Set GRUB password (manual - see below)
16. Remove Ubuntu install ISO from CD drive
17. Shut down VM
18. Right-click → Convert to Template (or: qm template <VMID>)
```

### Setting the GRUB Password (Manual Step)

This is the one control that cannot be automated because it requires interactive hash generation:

```bash
# Generate the hash
sudo grub-mkpasswd-pbkdf2
# Enter and confirm your password, copy the output hash

# Create the GRUB config
sudo tee /etc/grub.d/40_custom > /dev/null <<EOF
#!/bin/sh
exec tail -n +3 \$0
set superusers="grubadmin"
password_pbkdf2 grubadmin <PASTE_YOUR_HASH_HERE>
EOF

sudo chmod 755 /etc/grub.d/40_custom
sudo update-grub
```

---

## Post-Template: Cloning

To deploy a new VM from the template:

1. Right-click the template → **Clone** → **Full Clone**
2. Set VM ID and name
3. Go to the clone's **Cloud-Init tab** and configure:
   - User (or keep the default)
   - SSH public key (or keep the template's key)
   - IP configuration (static or DHCP)
   - DNS servers
4. Start the VM

On first boot, the `firstboot-harden.service` automatically:

- Regenerates SSH host keys
- Sets correct host key permissions
- Rebuilds the AIDE integrity database
- Creates `/var/lib/firstboot-done` to prevent re-running

The Cloud-Init subsystem handles hostname, user creation, SSH key injection, and network configuration.

---

## Firewall Management

The v2 hardening script uses iptables with persistent rules. Common management tasks:

**View current rules:**

```bash
sudo iptables -L -v -n --line-numbers
sudo ip6tables -L -v -n --line-numbers
```

**Add a new inbound service (e.g., HTTPS):**

```bash
# Insert before the LOG rule (check line numbers first)
sudo iptables -I INPUT <line> -p tcp --dport 443 -m conntrack --ctstate NEW -j ACCEPT
sudo iptables-save | sudo tee /etc/iptables/rules.v4
```

**Add a new outbound service:**

```bash
sudo iptables -I OUTPUT <line> -p tcp --dport 8080 -m conntrack --ctstate NEW -j ACCEPT
sudo iptables-save | sudo tee /etc/iptables/rules.v4
```

**Docker compatibility:** The script pre-creates a `DOCKER-USER` chain. When Docker is installed, it inserts its own rules but respects the `DOCKER-USER` chain for custom filtering. To restrict Docker container traffic:

```bash
sudo iptables -I DOCKER-USER -s 10.0.0.0/8 -j DROP
sudo iptables-save | sudo tee /etc/iptables/rules.v4
```

**Persistent rules** are stored in `/etc/iptables/rules.v4` and `/etc/iptables/rules.v6`, loaded on boot by `netfilter-persistent`.

---

## NIST 800-53 Control Mapping

| Control    | Title                                  | Implementation                                             |
|------------|----------------------------------------|------------------------------------------------------------|
| AC-2       | Account Management                     | Root locked, unnecessary users removed, umask 027          |
| AC-3       | Access Enforcement                     | AppArmor, su restricted to sudo group                      |
| AC-7       | Unsuccessful Logon Attempts            | pam_faillock: 5 attempts, 15-min lockout                   |
| AC-8       | System Use Notification                | /etc/issue, /etc/issue.net, /etc/motd banners              |
| AC-11      | Session Lock                           | TMOUT=900 (15-min idle timeout)                            |
| AC-17      | Remote Access                          | SSH: key-only, AllowUsers, modern crypto                   |
| AU-2/3/12  | Audit Events, Content, Generation      | Comprehensive auditd rules (CIS 4.1.3.1–18)                |
| AU-4       | Audit Log Storage                      | keep_logs, space alerts, admin halt                        |
| AU-8       | Time Stamps                            | Chrony NTP, systemd-timesyncd disabled                     |
| CM-2       | Baseline Configuration                 | Full system update before hardening                        |
| CM-6       | Configuration Settings                 | File permissions, sudo logging, cron/at restrictions       |
| CM-7       | Least Functionality                    | Services disabled, modules blacklisted, packages purged    |
| IA-5       | Authenticator Management               | Password quality/aging, PAM stack, opasswd                 |
| MP-2       | Media Access                           | Mount options (noexec, nosuid, nodev)                      |
| MP-7       | Media Use                              | USB storage module disabled                                |
| SC-5       | Denial-of-Service Protection           | SYN cookies, iptables rate limiting                        |
| SC-7       | Boundary Protection                    | iptables default-deny, connection tracking                 |
| SC-8/13    | Transmission Confidentiality/Crypto    | SSH: Curve25519, AES-GCM, SHA2-ETM                         |
| SC-28      | Protection of Information at Rest      | Filesystem mount hardening                                 |
| SI-2       | Flaw Remediation                       | Unattended security upgrades                               |
| SI-7       | Software Integrity                     | AIDE file integrity monitoring                             |
| SI-16      | Memory Protection                      | ASLR, ptrace, BPF, core dumps disabled                     |

---

## Wazuh SCA Check Coverage

Total checks addressed across all three scripts: **70+**

Checks are tracked by their Wazuh SCA IDs (35xxx series) corresponding to the CIS Ubuntu Linux 24.04 LTS Benchmark. See the summary tables at the end of each script for a complete list of check IDs mapped to their fixes.

---

## Known Exceptions

These checks will continue to fail or require environment-specific action. Document them as compensating controls for auditors.

| Check ID | Description                        | Reason / Compensating Control                                                  |
|----------|------------------------------------|--------------------------------------------------------------------------------|
| 35509    | `fat` module not disabled          | Required for UEFI boot partition (`/boot/efi`). Built into kernel on 24.04 - blacklisting the loadable module is safe but the check may still flag it. |
| 35540    | GRUB bootloader password           | Requires interactive hash generation. Set manually (see Deployment Workflow).  |
| 35589    | systemd-timesyncd not enabled      | Chrony is the NTP source (AU-8). timesyncd is intentionally disabled. Both satisfy the control; the scanner may not detect chrony. |
| 35710    | systemd-journal-upload auth        | Only required if using journal-upload to a remote journal gateway. Most setups use Wazuh or rsyslog instead. |
| 35720    | rsyslog remote logging             | Environment-specific. Placeholder config at `/etc/rsyslog.d/99-remote.conf`. Wazuh agent serves as a compensating control for centralized log collection. |

---

## Maintenance

**Quarterly review checklist:**

- Run Wazuh SCA scan and review any new failures
- Review `/var/log/sudo.log` and audit logs
- Verify AIDE integrity: `sudo aide --check`
- Check that unattended-upgrades is running: `sudo systemctl status unattended-upgrades`
- Review and rotate known-exception documentation
- Verify NTP sync: `chronyc tracking`
- Check LVM free space: `sudo vgdisplay vg_nist | grep Free`

**Updating the template:**

1. Clone the template to a temporary VM
2. Boot, apply updates (`sudo apt update && sudo apt upgrade`)
3. Re-run the hardening script (it's idempotent)
4. Rebuild AIDE database: `sudo aideinit -y -f`
5. Clean up and shut down
6. Convert to new template, retire the old one

---

## Troubleshooting

**Can't SSH in after hardening:**

The hardening script sets `PasswordAuthentication no` and `AllowUsers chris` (or your configured admin user). Verify your SSH key is in `~/.ssh/authorized_keys` and your username matches `AllowUsers`. Use the Proxmox noVNC console as a fallback - it's a virtual console, not SSH.

**TMOUT readonly variable error on login:**

The v2 script includes a guard for this. If you see the error on an older install, replace the timeout script:

```bash
sudo tee /etc/profile.d/nist-timeout.sh > /dev/null <<'EOF'
if [ -z "${TMOUT:-}" ]; then
    readonly TMOUT=900
    export TMOUT
fi
EOF
```

**AIDE takes a long time:**

Initial database creation scans all of `/boot`, `/bin`, `/sbin`, `/lib`, `/usr`, and `/etc`. This takes 10–20 minutes on a VM. Subsequent checks are faster. Verify it's running with `top -c | grep aide`.

**Docker bypasses firewall rules:**

The v2 script pre-creates the `DOCKER-USER` chain. Docker respects rules inserted there. To block specific traffic to containers, add rules to `DOCKER-USER`, not `INPUT`.

**iptables rules lost after reboot:**

Verify `netfilter-persistent` is enabled: `sudo systemctl is-enabled netfilter-persistent`. Rules should be in `/etc/iptables/rules.v4` and `rules.v6`. To re-save current rules: `sudo iptables-save | sudo tee /etc/iptables/rules.v4`.

**Wazuh still flags firewall checks (35626–35639):**

If you ran `remediate.sh` (which uses UFW) and then `nist-800-53-harden-v2.sh` (which removes UFW), the iptables rules satisfy the same CIS controls. Some Wazuh SCA checks are written for a specific firewall tool. Add Wazuh rule-level exceptions for the ones that don't match your chosen tool (see the Wazuh exception guidance in the conversation that produced these scripts).

---

## File Reference

```
.
├── README.md                       # This file
├── partition-layout.sh             # Step 1: LVM partition layout
├── nist-800-53-harden-v2.sh        # Step 2: OS hardening (iptables, CIS/Wazuh aligned)
└── remediate.sh                    # Step 3: Post-scan targeted fixes
```

## License

These scripts are provided as-is for educational and operational use. Review and test thoroughly in a non-production environment before deployment. Compliance is a shared responsibility - these scripts address technical controls but do not replace organizational policies, procedures, or risk assessments.