This article describes a redundant architecture for Apache web servers where database services are separated and not covered here. We focus on the file storage cluster and web server setup.
First, set up the data cluster. This example uses DragonFlyBSD for its HAMMER filesystem, which provides excellent NFS performance (NFSv3).
Install the machines that will act as NFS data servers in the usual way. Use two RAID arrays on each server: one RAID for the system (RAID1) and one for data (RAID5 or RAID10). During installation configure only the system RAID (e.g., da0).
Refer to an installation guide for DragonFlyBSD for details.
After installing DragonFlyBSD, create a Hammer filesystem on the data RAID and mount it:
newfs_hammer -L webdatas /dev/da1
mkdir /nfs
mount -t hammer /dev/da1 /nfs
Create PFS volumes (virtual volumes used for Hammer replication) and mount them. We need three PFS volumes to correspond to /usr/local, /usr/ports/distfiles and /var/db/pkg on the web servers. Run these commands on the NFS master:
mkdir /nfs/pfs/
mkdir /nfs/fsbd_fs/
hammer pfs-master /nfs/pfs/fbsd_fs
mount_null /nfs/pfs/fbsd_fs /nfs/fbsd_fs
mkdir /nfs/fbsd_fs
hammer pfs-master /nfs/pfs/fbsd_distfiles
mount_null /nfs/pfs/fbsd_distfiles /nfs/fbsd_distfiles
mkdir /nfs/fbsd_pkg
hammer pfs-master /nfs/pfs/fbsd_pkg
mount_null /nfs/pfs/fbsd_pkg /nfs/fbsd_pkg
Record the shared-uuid values shown during PFS creation; you will need them when configuring the slave.
On the slave, configure the PFS volumes using the shared-uuid values from the master:
mkdir /nfs/pfs/
mkdir /nfs/fsbd_fs/
hammer pfs-slave /nfs/pfs/fbsd_fs shared-uuid=xxx
mkdir /nfs/fbsd_fs
hammer pfs-slave /nfs/pfs/fbsd_distfiles shared-uuid=yyy
mkdir /nfs/fbsd_pkg
hammer pfs-slave /nfs/pfs/fbsd_pkg shared-uuid=zzz
Note: pfs-slave volumes are not mounted directly; replication is performed offline.
Add the following entries to /etc/fstab:
## Cluster
/dev/da1 /nfs hammer rw 0 0
/nfs/pfs/fbsd_fs /nfs/fbsd_fs null rw 0 0
/nfs/pfs/fbsd_distfiles /nfs/fbsd_distfiles null rw 0 0
/nfs/pfs/fbsd_pkg /nfs/fbsd_pkg null rw 0 0
Note: ensure all physical disks are assembled and available before attempting to mount the PFS volumes.
Configure the NFS server by editing /etc/exports. Replace the network address with your web servers’ network or specific hosts:
/nfs/fbsd_fs -maproot=root -network=192.168.1.0 -mask 255.255.255.0
/nfs/fbsd_distfiles -maproot=root -network=192.168.1.0 -mask 255.255.255.0
/nfs/fbsd_pkg -maproot=root -network=192.168.1.0 -mask 255.255.255.0
Enable the services in /etc/rc.conf:
nfs_server_enable="YES"
mountd_enable="YES"
mountd_flags="-r"
rpcbind_enable="YES"
nfsd_flags="-n 16"
Start the services:
/etc/rc.d/rpcbind start
/etc/rc.d/mountd start
/etc/rc.d/nfsd start
You can replicate using a pull or push model. This example uses a push model (master pushes to slave):
hammer mirror-stream /nfs/pfs/fbsd_fs root@192.168.1.1:/nfs/pfs/fbsd_fs &
hammer mirror-stream /nfs/pfs/fbsd_distfiles root@192.168.1.1:/nfs/pfs/fbsd_distfiles &
hammer mirror-stream /nfs/pfs/fbsd_pkg root@192.168.1.1:/nfs/pfs/fbsd_pkg &
You may add these commands to /etc/rc.local to run them at boot.
Your NFS server is now ready.
Refer to an installation guide for FreeBSD. After installation, extract the ports tree:
portsnap fetch extract
Enable NFS client services in /etc/rc.conf:
nfs_client_enable="YES"
rpcbind_enable="YES"
Start the services:
service nfsclient start
service rpcbind start
On each web server mount the NFS volumes:
mount_nfs -o tcp,nolockd,rw 192.168.1.1:/nfs/fbsd_web /usr/local/
mount_nfs -o tcp,nolockd,rw 192.168.1.1:/nfs/fbsd_pkg /var/db/pkg/
mount_nfs -o tcp,nolockd,rw 192.168.1.1:/nfs/fbsd_distfiles /usr/ports/distfiles/
Make the mounts persistent by adding them to /etc/fstab:
## NFS mount
10.117.100.65:/nfs/fbsd_web /usr/local nfs rw,tcp,nolockd 0 0
10.117.100.65:/nfs/fbsd_distfiles /usr/ports/distfiles nfs rw,tcp,nolockd 0 0
10.117.100.65:/nfs/fbsd_pkg /var/db/pkg nfs rw,tcp,nolockd 0 0
Using TCP keeps a persistent connection to the NFS server and generally performs better than UDP for this workload. We disable lockd here because the exported data is static; if you host a database you must enable lockd and statd.
With the web servers prepared, install the required ports such as Apache and PHP. Example commands:
cd /usr/ports/www/apache22
make install clean
cd /usr/ports/lang/php5
make install clean
cd /usr/ports/www/wordpress
make install clean
Keep default options unless you have specific needs. Building ports may take a long time on a single-core CPU.
When a port is built and installed on one server, the files should appear on the second web server via the shared NFS storage.
Enable Apache at boot:
apache22_enable="YES"
On reboot you may find Apache does not start automatically because the NFS mounts are not yet available (the binaries are not in PATH). Work around this by starting Apache from /etc/rc.local:
service apache22 start