2024 Server Migration

This page is to collect information about our current configuration and options for switching to a new server going forward, since CentOS 7 goes end-of-life at 2024-06-30. Please feel free to edit to add missing information or correct errors.

Server selection

Linode has been acquired by Akamai and raised their prices. Could potentially downgrade from the $12/mo (2GB RAM) plan to the $5 (1GB RAM "Nanode") plan depending on distro system requirements. Could also consider dropping paid backup service or using something different.

Other VPS providers can be considered, not sure if they are any cheaper/better.

Cloud Plans

Looking at 1GB and 2GB plans.

Vendor	Plan	$/Mo	Memory	CPU	Storage	Transfer
Linode	Nanode 1GB	$5	1GB	1	25GB	1TB
Linode	Linode 2GB	$12	2GB	1	50GB	2TB
DigitalOcean	1GiB	$6	1GiB	1	25GiB	1TB
DigitalOcean	2GiB	$12	2GiB	1	50GiB	2TB
Hostwinds		$4.99	1GB	1	30GB	1TB
Hostwinds		$9.90	2GB	1	50GB	2TB
Vultr		$5	1GB	1	25GB	1TB
Vultr		$10	2GB	1	55GB	2TB

All services seem to be similar, with price differences only by a dollar or two.

OS selection

CentOS Linux is dead. CentOS Stream is only viable as long as Red Hat/IBM has a business case for it. Can consider other options like pure Debian or Ubuntu LTS.

Service enumeration

Services that are currently running on the WPLUG Linode server. These should include things that are user-facing only, not infrastructure.

• E-mail (Postfix)
• Mailing lists (Mailman)
• Wiki (MediaWiki)
• Monkeybot IRC bot (infobot)
  • While one of the oldest WPLUG services, since freenode is essentially dead, there may not be good reason to keep it alive.
• Ticketing system (Trac)
  • This is virtually unused, likely not worth deploying.

Infrastructure software which supports the services above.

• Web server (Apache) - it would be possible to use Nginx instead, but I (Vance) am not familiar with setting it up
• PHP (Apache mod_php) for MediaWiki - could be replaced with PHP-FPM
• Let's Encrypt for TLS certificate (certbot)
• Python for Mailman, Let's Encrypt, and Trac
• Perl for Monkeybot
• MariaDB for MediaWiki - may be possible to use SQLite or PostgreSQL instead
• Greylisting daemon (Postgrey)
• Fail2ban

Support lifetime

• CentOS Stream 9 - 2027-05-31 (limited to RHEL Full Support lifetime)
• Rocky Linux 9 - 2032-05-31
• AlmaLinux 9 - 2032-05-31
• Debian 12 "bookworm" - 2028-06-10 or later
• Ubuntu 22.04 LTS - 2027-04

System requirements

	CPU	RAM	Disk
CentOS / Rocky / Alma 9	amd64, arm64, ppc64el	1.5 GB	10 GB
Debian 12	amd64, arm64, i386 (686 or higher), ppc64el, mipsel, mips64el	512 MB	4 GB
Ubuntu 22.04	1 GHz amd64, arm64, ppc64el	1 GB	2.5 GB

Currently we are using 22 GB of disk, as follows:

594M    /boot
35M     /etc
11G     /home    - monkeybot.log is 7.3GB
25M     /opt
2.2G    /root
4.0K    /srv
36K     /tmp
2.1G    /usr
6.3G    /var

Software availability

This table is to track, for the different distros under consideration, whether the software we need is available within its repositories. We want to minimize the number of applications which have to be maintained manually.

Key:

• B: in distro's base repository
• A: in an additional repository provided by the distro
• T: in a third-party repository
• ~: not available in any known repository
• ?: availability unknown

	CentOS / Rocky / Alma 9	Debian 12	Ubuntu 22.04
postfix	B 3.5.9	B 3.7.5	B 3.6.4
mailman	~	B 3.3.8	A (3) 20200530
postgrey	T 1.37EPEL	B 1.37	A 1.36
mediawiki	~	B 1.39	A 1.35
infobot	~	~	~
trac	~	A 1.6 (backports)	A 1.5.3
apache	B 2.4.53	B 2.4.57	B 2.4.52
nginx	B 1.20.1 / 1.22.1	B 1.22.1	B 1.18.0
php	B 8.0 / 8.1	B 8.2	B 8.1
php-fpm	B 8.0 / 8.1	B 8.2	A 8.1
python2	~	~	A 2.7.18
python3	B 3.9.16	B 3.11.2	B 3.10.4
perl5	B 5.32	B 5.36	B 5.34
mariadb	B 10.5	B 10.11	A 10.6
mysql	B 8.0	~	B 8.0
sqlite	B 3.34	B 3.40	B 3.37
fail2ban	T 1.0.2EPEL	B 1.0.2	A 0.11.2
certbot	T 2.6.0EPEL	B 2.1.0	A 1.21.0

Third-party repositories:

• EPEL - Extra Packages for Enterprise Linux

Nice-to-haves

• We have an archive of static web pages from the pre-2007 server "penguin" - it would be nice to make this history available somehow.

Migration Notes

New virtual server:

• Name: penguin
• OS: Debian 12
• Region: Atlanta
• Plan: Nanode 1GB
• IPv4: 139.144.29.132
• IPv6: 2600:3c02::f03c:94ff:fe10:c62f

New staging server:

• Name: nestling
• OS: Debian 11
• Region: Chicago
• Plan: Nanode 1GB Linode 2 GB due to high memory use of Mailman 3.
• IPv4: 172.234.26.233
• IPv6: 2600:3c06::f03c:95ff:fed6:b745

This server was selected to run this OS version and location because it was the only image and location that featured cloud-init which was necessary for automated builds. Once Debian 12 gets that feature, we'll migrate over.

Server Setup Steps

I'm setting up ansible to do the server build and configuration entirely. Nesting is the staging server. Here are the steps completed:

• Create or rebuild new linode, with cloud-init that adds admin users and ssh public keys.
• Set hostname, /etc/hosts, disable root ssh login
• Add board group and create board home directory
• apt update
• Install apache, mariadb, php
• Install and run certbot
• Install mediawiki and restore images and sql data
• Mail - mua, postfix, opendkim, SPF, DKIM, DMARC
• Backup mediawiki data and config for future rebuild and migration
• Restore mediawiki data and config
• Add ufw management and set up firewall
• fail2ban complete and added to penguin also
• mail aliases
• established staging.wplug.org subdomain
• mailman3 installation finally successful
• limit outbound mail in postfix so we don't spam people while testing
• ran out of memory on the 1GB plan so bumped it up to 2GB RAM
• added 512MB /swapfile
• mailman 2.1 to 3 migration

Currently working on:

• mailman3 - figuring out settings and backup/restore

Next steps:

• Reconfigure apache to have wiki as front page.
• Rename django admin page, set up a fail2ban for that.
• Figure out where hyperkitty mailman3 index files are being saved to. /opt/mailman/mm?