m0n0wall Developers' Handbook

Manuel Kasper

Edited by

Chris Buechler

Additional Contributors listed in the m0n0wall Handbook.

m0n0wall Version 1.2, October 2005

All rights reserved.

Redistribution and use in any form, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions must retain the above copyright notice, this list of conditions and the following disclaimer.

  • Neither the name of the m0n0wall Documentation Project nor the names of its contributors may be used to endorse or promote products derived from this documentation without specific prior written permission.

THIS DOCUMENTATION IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS DOCUMENTATION OR THE ASSOCIATED SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

September 2005

Abstract

A guide for m0n0wall developers.


Table of Contents

1. The complete guide to building a m0n0wall image from scratch
1.1. Installing FreeBSD
1.2. Using cvsup to get the latest FreeBSD patches
1.3. make world
1.4. Building the m0n0wall root file system
1.5. Applying m0n0wall patches for FreeBSD
1.6. Building the kernel
1.7. Building the software packages
1.8. Installing supplementary tools
1.9. Building the boot loader
1.10. Adding the libraries
1.11. Adding the PHP-based configuration scripts
1.12. Adding the webGUI
1.13. Creating mfsroot
1.14. Putting it all together: creating the image
1.15. Creating a bootable CD-ROM version
2. m0n0wall Hacker's Guide
2.1. Setting Up a Host Development Environment
2.2. PXE Booting
2.3. Setting up m0n0wall on the host
2.4. NFS
2.5. Unpacking/Editing/Packing the Source
2.6. Writing a CF Image
2.7. Submitting Changes
3. Other Documentation
4. Development Frequently Asked Questions
4.1. I don't have a FreeBSD box to develop new features for m0n0wall, is there an easier way?
4.2. What patches have been applied to the FreeBSD source that is used in m0n0wall, and are they available?
4.3. Where can I find the m0n0wall source code?
4.4. Is there any information available on the unsupported extensions support?

Chapter 1. The complete guide to building a m0n0wall image from scratch

This guide explains, in detail, all the steps that are required in order to build a complete m0n0wall image from scratch. The primary intention is not that people will use the guide to actually build their own images from scratch, as that is rarely necessary, but to document and preserve all the details so that people who would like to make changes to an existing image can see how m0n0wall and all of its parts are built.

This guide assumes that FreeBSD 4.11-RELEASE is used. Note that there are other ways of doing this, like keeping a separate copy of the FreeBSD source tree and installing world into a subdirectory to leave the build system itself untouched. However, it is assumed that a separate, dedicated (possibly virtual) machine is going to be used for this task anyway, which is also recommended since many steps need to be done as root (among other things to get the file ownership and permissions right without too much effort).

Note that this is not "m0n0wall image building for dummies", and readers are expected to know what they're doing. As such, not all necessary commands and full file paths are listed (the obvious ones have been left out), so don't expect it to work if you just run all the commands listed in this guide.

Finally, many of these tasks can and should be automated (especially the image building part). How this is done is left to the reader - there are no official m0n0wall build scripts.

1.1. Installing FreeBSD

Install FreeBSD 4.11-RELEASE as usual, but use one file system only (i.e. don't create a separate partition for /usr)! See the section about make world for an explanation why this is necessary. Make sure that you choose the "Developer" distribution set. Installing the ports collection is also a good idea.

1.2. Using cvsup to get the latest FreeBSD patches

m0n0wall is always based on FreeBSD -RELEASE versions, never -STABLE (since what exactly -STABLE is depends very much on the exact point in time that the sources were checked out). However, in order to get the latest security patches, the release maintenance branches are used. For FreeBSD 4.11-RELEASE, this is RELENG_4_11. You'll have to use cvsup to get the latest patches.

1.2.1. Installing cvsup

The easiest way to do this is to use the package:

pkg_add -r cvsup-without-gui
rehash

1.2.2. Configuring the supfile

  • copy the example supfile

    cp /usr/share/examples/cvsup/stable-supfile /etc/supfile
  • modify /etc/supfile

    • choose a nearby cvsup host (where it says "CHANGE_THIS")

    • set tag=RELENG_4_11 (instead of RELENG_4)

    • add the line

      ports-all tag=.

      to the end of the file to update the ports collection too

1.2.3. Running cvsup

cvsup -g -L 2 /etc/supfile

Note

This can take a while.

1.3. make world

Now that the latest FreeBSD patches have been installed, it's time to recompile the entire operating system. Note that this would be necessary even if there weren't any patches: by default, FreeBSD has all the essential system binaries (mostly those in /bin and /sbin) linked statically so that the system can boot even if /usr (which holds important libraries like libc) is not available (which is often the case when it's on a separate file system). This takes up huge amounts of space though, so we obviously want all the binaries to be linked dynamically for our m0n0wall image.

1.3.1. Editing /etc/make.conf

Create /etc/make.conf and put the following lines in it:

NOSHARED=no
IPFW2=TRUE

The last line ensures that IPFW2 is built (instead of the older IPFW1 version, which is the default) when we compile the kernel/modules/ipfw executable later.

1.3.2. making world

Execute the following commands to rebuild and install the entire operating system:

cd /usr/src
make -j4 buildworld

Note

This will take a while.

shutdown now
cd /usr/src
make installworld
make buildkernel
make installkernel
reboot

1.4. Building the m0n0wall root file system

The following steps assume that your m0n0wall root file system will be built in $MWROOT, so create a directory to hold the root file system and set MWROOT to point to it.

1.4.1. Creating the root directory structure

cd $MWROOT
mkdir bin cf conf.default dev etc ftmp mnt modules proc root sbin tmp usr var
mkdir etc/inc
ln -s /cf/conf conf
cd usr
mkdir bin lib libexec local sbin share
cd local
mkdir bin captiveportal lib sbin www
ln -s /var/run/htpasswd www/.htpasswd
  • /cf is where the Compact Flash card (or the hard disk, or the floppy disk) is mounted later on

  • /conf.default contains the factory default config.xml

  • /conf (or /cf/conf effectively due to the symlink) contains the current config.xml

  • /ftmp is used to mount a temporary memory file system during a firmware upgrade through the webGUI

  • /etc/inc contains PHP include files that are used by both the boot scripts and the webGUI

  • /usr/local/www contains the webGUI pages

  • /usr/local/captiveportal is the web root directory for the HTTP server process to which HTTP connections are redirected if the captive portal is on and the user hasn't authenticated yet

  • the symbolic link for .htpasswd is required as the boot scripts will write the webGUI user/password to /var/run/htpasswd

1.4.2. Copying required binaries

To facilitate this, a list of binaries that need to be copied is provided along with a simple perl script to copy them over (taken from the miniBSD tutorial). Download the script from http://m0n0.ch/wall/downloads/freebsd-4.11/mkmini.pl and the list of binaries from http://m0n0.ch/wall/downloads/freebsd-4.11/m0n0wall.files.

perl mkmini.pl m0n0wall.files / $MWROOT

1.4.3. Preparing /etc

Most of the base contents of /etc aren't very interesting, so they're available as a tarball to save the time it would take to put them together from FreeBSD sources. Download from http://m0n0.ch/wall/downloads/freebsd-4.11/etc.tgz and extract as follows:

tar -xzf etc.tgz -C $MWROOT

1.4.3.1. Adding /etc/version

/etc/version holds the version number of the image. Create with e.g.

echo "1.21" > $MWROOT/etc/version

1.4.3.2. Adding /etc/version.buildtime

/etc/version.buildtime is displayed in the webGUI. Create as follows:

date > $MWROOT/etc/version.buildtime

1.4.3.3. Adding /etc/platform

/etc/platform identifies the platform for which the image is built and is very important, as the PHP scripts use it to determine e.g. how the configuration is to be stored or which options should be available in the webGUI. Possible values are:

  • net45xx

  • net48xx

  • wrap

  • generic-pc

  • generic-pc-cdrom

echo "net45xx" > $MWROOT/etc/platform

1.4.4. Preparing /dev

FreeBSD 4.x doesn't have devfs yet, so we need to add a few essential device files. You could copy the contents of /dev on your FreeBSD box or use MAKEDEV; however, that would result in many device files that aren't necessary. A tarball of all the devices that are typically needed in m0n0wall is provided at http://m0n0.ch/wall/downloads/freebsd-4.11/dev.tgz. Extract as follows:

tar -xzf dev.tgz -C $MWROOT

1.4.5. Adding the default config.xml

Download the latest default config.xml from http://m0n0.ch/wall/downloads/config.xml and put it in $MWROOT/conf.default/config.xml.

1.4.6. Adding zoneinfo.tgz

FreeBSD keeps time zone information files in /usr/share/zoneinfo. In order to save space, m0n0wall uses a stripped down version of the contents of that directory and keeps them in a tarball. Download zoneinfo.tgz from http://m0n0.ch/wall/downloads/freebsd-4.11/zoneinfo.tgz and place it in $MWROOT/usr/share/zoneinfo.tgz.

1.5. Applying m0n0wall patches for FreeBSD

In order to build a m0n0wall image where all the features work as in the official releases, various patches to the kernel and userland have to be applied. Download these patches from http://m0n0.ch/wall/downloads/freebsd-4.11/patches.tgz and extract them somewhere (the following commands assume that your patches are in $MWPATCHDIR).

1.5.1. Userland patches

1.5.1.1. ipfilter: ipf.c

This needs to be patched so it doesn't exit when it encounters an error while adding a rule. The most common example where this is not good at all is when you have a duplicate rule.

cd /usr/src
patch < $MWPATCHDIR/user/ipf.c.patch
cd sbin/ipf
make
install -s /usr/obj/usr/src/sbin/ipf/ipf $MWROOT/sbin

1.5.1.2. syslogd circular logging support

Since m0n0wall keeps everything in a memory file system for reliability, disk space is scarce. On the other hand, even the biggest disk can eventually become full with logs, so it's better to solve this problem in another way. m0n0wall uses circular logging, which means that the oldest log entries are overwritten by new ones when there's not enough space. The syslogd that comes with FreeBSD needs to be patched with clog (http://software.wwwi.com/syslogd/, modified to work with 4.11's syslogd) to support this kind of logging.

cd /usr/src
patch < $MWPATCHDIR/user/syslogd.c.patch
cd usr.sbin
tar xfvz $MWPATCHDIR/user/clog-1.0.1.tar.gz
cd syslogd
make
install -s /usr/obj/usr/src/usr.sbin/syslogd/syslogd $MWROOT/usr/sbin
cd ../clog
make obj && make
install -s /usr/obj/usr/src/usr.sbin/clog/clog $MWROOT/usr/sbin

1.5.1.3. dhclient-script

The /sbin/dhclient-script that comes with FreeBSD needs to be modified to write out DNS server information for the PHP scripts to use, and also not to invoke the hooks upon uninteresting events.

cd $MWROOT/sbin
patch < $MWPATCHDIR/user/dhclient-script.patch
rm dhclient-script.orig

1.5.2. Kernel patches

Numerous patches to the FreeBSD kernel need to be applied for m0n0wall to make everything work as desired; here's the list of changes (paths relative to /usr/src/sys):

  • conf/options

    add new kernel config options for ipfilter state table size and to enable the forced MSS clamping feature (see below)

  • contrib/ipfilter/netinet/fil.c, contrib/ipfilter/netinet/ip_compat.h, contrib/ipfilter/netinet/ip_fil.h

    moving includes around and change some #ifs to make it compile properly

  • contrib/ipfilter/netinet/ip_nat.c, contrib/ipfilter/netinet/ip_nat.h, contrib/ipfilter/netinet/mlfk_ipl.c

    checksum calculation fixes by Fred Wright and forced MSS clamping feature patch by Manuel Kasper

  • contrib/ipfilter/netinet/ip_state.c

    window scaling-related fixes by Fred Wright

  • i386/isa/clock.c

    write back weekday properly to CMOS to avoid date/time reset on Soekris machines

  • kern/subr_diskslice.c

    comment out a warning about raw partition size != slice size since we don't care and everybody uses different CF cards or hard drives

  • net/if_ethersubr.c

    comment out a warning about dropping multicast packets when bridging

  • netgraph/ng_ppp.c

    protocol-field compression fix by Fred Wright

  • netinet/ip_input.c

    prevent packets from being passed through ipfilter twice when they come out of a dummynet pipe

  • netinet/ip_output.c

    reverse ipfw/ipfilter processing order to be symmetric with the input side

  • netipsec/key.c, netipsec/key_var.h

    new SA preferral patch by Fred Wright

  • pci/if_sis.c, pci/if_sisreg.h

    improve handling of stopped sis devices where IRQ sharing is present (such as on the net4801); improved DP83815 short cable bug fixes by Fred Wright

  • i386/i386/identcpu.c

    identify NS Geode CPUs properly (net4801, WRAP)

  • i386/i386/vm_machdep.c

    add reset code for NS Geode (for keyboard controller-less WRAP)

  • dev/ata/ata-pci.c

    add kernel option to disable probing of ATA slave devices (causes ~30 second hang on boot with WRAP)

  • dev/ata/ata-disk.c

    patch ATA standby support to allow changes at runtime

  • modules/ipfw/Makefile

    build with IPFIREWALL_DEFAULT_TO_ACCEPT

  • dev/wi/if_wi.c

    add DELAY(1) in wi_cmd as a workaround for timeouts with some PRISM-based cards

  • pci/if_xl.c, pci/if_xlreg.h

    add definitions for 3C920B-EMB-WNM

Apply the patches as follows:

cd /usr/src
patch -p0 < $MWPATCHDIR/kernel/kernel-411.patch

1.6. Building the kernel

1.6.1. Compiling and compressing the kernel

Download the kernel configuration file for the platform that you want to build for from http://m0n0.ch/wall/downloads/freebsd-4.11 and copy it to /sys/i386/conf, then execute the following commands to compile the kernel and modules:

cd /sys/i386/conf
config M0N0WALL_[PLATFORM]
cd /sys/compile/M0N0WALL_[PLATFORM]
make depend && make

Compress the kernel using gzip:

gzip -9 kernel

The kernel will be installed later, as it doesn't go directly into the root file system.

1.6.2. Installing the modules

The dummynet, if_tap, if_vlan and ipfw modules are needed for m0n0wall:

cd modules/usr/src/sys/modules
cp dummynet/dummynet.ko if_tap/if_tap.ko if_vlan/if_vlan.ko ipfw/ipfw.ko $MWROOT/modules

1.7. Building the software packages

This section deals with properly compiling all the third-party software packages that are used in some way in m0n0wall. Where useful, the FreeBSD ports system is used (especially if a package requires FreeBSD-relevant patches). For some packages, the standard "./configure && make && make install" procedure can be used; others need a few extra configure options to produce a small binary, while still others need patches to work properly on m0n0wall.

1.7.1. PHP

Install autoconf213 from the FreeBSD ports collection and create some links so that PHP's buildconf scripts find autoconf:

cd /usr/ports/devel/autoconf213
make install clean
ln -s /usr/local/bin/autoconf213 /usr/local/bin/autoconf
ln -s /usr/local/bin/autoheader213 /usr/local/bin/autoheader

Download the latest version of PHP 4.4 from http://www.php.net and decompress as usual. Download the RADIUS PECL extension from http://m0n0.ch/wall/downloads/freebsd-4.11/radius-1.2.5.tgz. Unpack it in php-4.4.x/ext and rename the resulting directory (whose name includes the version number) to 'radius'. Go back into the php-4.4.x directory and build/install as follows:

rm configure
./buildconf --force
./configure --without-mysql --with-pear --with-openssl --enable-discard-path --enable-radius --enable-sockets --enable-bcmath
make
install -s sapi/cgi/php $MWROOT/usr/local/bin

Put the following in $MWROOT/usr/local/lib/php.ini:

magic_quotes_gpc = Off
magic_quotes_runtime = Off
max_execution_time = 0
max_input_time = 180
register_argc_argv = Off
file_uploads = On
upload_tmp_dir = /ftmp
upload_max_filesize = 8M
post_max_size = 10M
html_errors = Off
include_path = ".:/etc/inc:/usr/local/www:/usr/local/captiveportal"

1.7.2. mini_httpd

Download mini_httpd 1.19 from http://www.acme.com/software/mini_httpd/mini_httpd-1.19.tar.gz and extract. A custom patch needs to be applied that:

  • adds a limit to the number of concurrent connections to prevent DoS attacks (since mini_httpd is a forking web server), and since m0n0wall 1.21: an optional per-client-IP-address limit

  • adds captive portal mode support (including httpd support for the file manager since m0n0wall 1.21)

  • sets SCRIPT_FILENAME (for php)

  • fixes a bug where mini_httpd would exit if a TCP connection was closed before mini_httpd had a chance to handle it (ECONNABORTED)

  • adds "index.php" to the list of index file names

  • makes the error pages much more plain

  • doesn't add a Server: header

  • patches the Makefile to build mini_httpd with SSL support

Apply the patch, compile and install as follows:

patch < $MWPATCHDIR/packages/mini_httpd.patch
make
install -s mini_httpd $MWROOT/usr/local/sbin

1.7.3. ISC DHCP server and relay

The FreeBSD ports system will be used to compile these.

1.7.3.1. DHCP server

cd /usr/ports/net/isc-dhcp3-server
make

Don't choose any options in the dialog box (i.e. deselect them all). When it's compiled, install as follows:

install -s work/dhcp-*/work.freebsd/server/dhcpd $MWROOT/usr/local/sbin

1.7.3.2. DHCP relay

cd /usr/ports/net/isc-dhcp3-relay
make
install -s work/dhcp-*/work.freebsd/relay/dhcrelay $MWROOT/usr/local/sbin

1.7.4. Dnsmasq

This requires GNU getopt, so it's easier to use the port:

cd /usr/ports/dns/dnsmasq
make
install -s work/dnsmasq-*/src/dnsmasq $MWROOT/usr/local/sbin

1.7.5. MSNTP

cd /usr/ports/net/msntp
make
install -s work/msntp-*/msntp $MWROOT/usr/local/bin

1.7.6. wol

Get the latest version of wol from http://sourceforge.net/project/showfiles.php?group_id=8895. Extract, then compile and install as follows:

./configure --disable-nls
make
install -s src/wol $MWROOT/usr/local/bin

1.7.7. ez-ipupdate

Get ez-ipupdate 3.0.11b8 from http://dyn.pl/client/UNIX/ez-ipupdate/ez-ipupdate-3.0.11b8.tar.gz. A patch needs to be applied that:

  • fixes interface IP address determination under FreeBSD

  • writes out the cache file before running the post-update command (instead of after) so that the command can use the information from the cache file

  • fixes a security issue (syslog() call)

patch < $MWPATCHDIR/packages/ez-ipupdate.c.patch
./configure
make
install -s ez-ipupdate $MWROOT/usr/local/bin

1.7.8. bpalogin

Get the latest source code version of bpalogin from http://bpalogin.sourceforge.net/index.php?page=download#source. Extract, then compile and install as follows:

./configure
make
install -s bpalogin $MWROOT/usr/local/sbin

1.7.9. MPD

Install MPD 3.x using the ports system:

cd /usr/ports/net/mpd
make
install -s work/mpd-*/src/mpd $MWROOT/usr/local/sbin

1.7.10. OpenVPN

Get the latest release version of OpenVPN from http://openvpn.net/download.html. Extract, then compile and install as follows:

setenv CFLAGS "-DLOG_OPENVPN=LOG_LOCAL6 -O2"
./configure --disable-lzo --disable-plugins --disable-management --disable-socks --disable-http --disable-debug
make
install -s openvpn $MWROOT/usr/local/sbin

1.7.11. racoon

As of version 1.21, m0n0wall uses the ipsec-tools version of racoon, as the KAME racoon version isn't being developed anymore and has been removed from the FreeBSD ports tree.

Install racoon from ipsec-tools using the ports system:

cd /usr/ports/security/ipsec-tools

Edit CONFIGURE_ARGS in the Makefile:

  • remove the --enable-debug and --enable-ipv6 options (saves space)

  • add the --without-readline option (removes the dependency on libreadline)

make
install -s work/ipsec-tools-*/src/racoon/.libs/racoon $MWROOT/usr/local/sbin
install -s work/ipsec-tools-*/src/libipsec/.libs/libipsec.so.0 $MWROOT/usr/local/lib

1.7.12. ucd-snmp

Get ucd-snmp 4.2.x (not net-snmp as it's much bigger) from http://sourceforge.net/project/showfiles.php?group_id=12694. Extract, then compile and install as follows:

./configure  --without-openssl --disable-debugging --enable-static \
--enable-mini-agent --disable-privacy --disable-testing-code \
--disable-shared-version --disable-shared --disable-ipv6 \
'--with-out-transports=TCP Unix' \
'--with-mib-modules=mibII/interfaces mibII/var_route ucd-snmp/vmstat_freebsd2'

Just press enter at all questions.

make
install -s agent/snmpd $MWROOT/usr/local/sbin

1.8. Installing supplementary tools

Apart from the various (relatively) big software packages like PHP or isc-dhcp that make m0n0wall what it is, there's a range of small helper tools, usually written in C by one of the m0n0wall developers, that need to be compiled. These tools handle things that either can't be done in PHP at all, or only inefficiently. This section explains how to compile and install them.

Download the supplementary tools from http://m0n0.ch/wall/downloads/freebsd-4.11/tools.tgz.

1.8.1. choparp

choparp is an ARP proxy daemon. m0n0wall uses a customized version that supports address ranges, rather than just single addresses and subnets. Install as follows:

gcc -o choparp choparp.c
install -s choparp $MWROOT/usr/local/sbin

1.8.2. atareinit

This simple tool reinitializes ATA drive controllers and is required for the IDE HD standby feature.

gcc -o atareinit atareinit.c
install -s atareinit $MWROOT/usr/local/sbin

1.8.3. stats.cgi

The SVG-based traffic and CPU load graphs in the webGUI call a CGI through the web server to get the information. For efficiency, this CGI is written in C rather than PHP since it's invoked once every second while the graph is being displayed. Install as follows:

gcc -o stats.cgi stats.c
install -s stats.cgi $MWROOT/usr/local/www

1.8.4. minicron

minicron is a very simple daemon that invokes a command at regular intervals. The captive portal uses this to prune expired user sessions. Install as follows:

gcc -o minicron minicron.c
install -s minicron $MWROOT/usr/local/bin

1.8.5. verifysig

This tool uses the OpenSSL libraries to check the digital signature of an uploaded firmware image using /etc/pubkey.pem. Install as follows:

gcc -o verifysig -lcrypto verifysig.c
install -s verifysig $MWROOT/usr/local/bin

1.8.6. runmsntp.sh

MSNTP has the bad habit of exiting sometimes (e.g. when it can't contact a time server for awhile), so we need a simple shell script to restart it if it dies. Install as follows:

install runmsntp.sh $MWROOT/usr/local/bin

1.8.7. linkup scripts

A few shell scripts need to be provided for MPD to invoke when a PP(T)P connection is established or torn down:

install ppp-linkup vpn-linkdown vpn-linkup $MWROOT/usr/local/sbin

1.9. Building the boot loader

The boot loader in official m0n0wall images is configured to read only one block at a time to work around incompatibilities between certain brands of CF cards and BIOSes. Also, it's compiled not to use fancy terminal emulation, as that usually messes up the output on a serial console. If you're going to build an image for a generic PC, you may be able to do without these modifications and just use the boot loader from /boot that was installed with FreeBSD. Otherwise, read on.

Edit /sys/boot/i386/libi386/Makefile and comment out the line that says CFLAGS+= -DTERM_EMU. Now apply the single-sector-read patch, if desired:

cd /usr/src
patch < $MWPATCHDIR/boot/boot.patch

When building a boot loader for WRAP boards, boot1.s needs to be modified not to fiddle with gate A20, since the WRAP doesn't have a keyboard controller:

patch < $MWPATCHDIR/boot/boot-wrap.patch

Compile the boot loader:

cd /sys/boot
make clean && make obj && make

Make a directory somewhere to hold the boot files until you're ready to create the image, and copy the necessary files to it:

mkdir $BOOTDIR
cp /usr/obj/usr/src/sys/boot/i386/loader/loader $BOOTDIR
cp /usr/obj/usr/src/sys/boot/i386/boot2/{boot1,boot2} $BOOTDIR
cp /usr/obj/usr/src/sys/boot/i386/cdboot/cdboot $BOOTDIR

1.9.1. Preparing loader.rc

loader.rc is the file that the loader reads and interprets. For m0n0wall, it is used to disable ATA DMA (to increase compatibility with odd hardware, especially since CF/HD performance isn't very important in m0n0wall) and load the kernel and the MFS root file system.

Create the file $BOOTDIR/loader.rc with the following contents:

set hw.ata.atapi_dma="0"
set hw.ata.ata_dma="0"
load /kernel
load -t mfs_root /mfsroot
boot

If you're building for a platform that only has a serial console, add the flag "-h" (without the quotes) to the "boot" command on the last line.

1.10. Adding the libraries

None of the dynamically linked libraries that are needed have been added yet. This is because a Perl script, mklibs.pl, can be run on the root file system to create a list of libraries that are actually needed (mklibs.pl does this by running ldd on each binary). Get the script from http://m0n0.ch/wall/downloads/freebsd-4.11/mklibs.pl and run it as follows:

perl mklibs.pl $MWROOT > m0n0wall.libs
perl mkmini.pl m0n0wall.libs / $MWROOT

1.11. Adding the PHP-based configuration scripts

Now it's time to add the actual guts that make m0n0wall what it is! Get the latest phpconf scripts from the Subversion repository at http://svn.m0n0.ch/wall. If you want the latest development version, use the 'trunk' subdirectory, or else if you want to go with a specific branch or a tagged release version, see the 'branches' and 'tags' subdirectories, respectively. Assuming you want the latest version and have Subversion installed on your machine, just run the following command in a temporary directory:

svn export http://svn.m0n0.ch/wall/trunk

Copy the contents of the phpconf subdirectory to $MWROOT/etc. Make sure that all rc.* files go into $MWROOT/etc and all *.inc files into $MWROOT/etc/inc. Also, make sure that the rc.* files have the execute permission bits set (chmod 755).

1.12. Adding the webGUI

Copy the webGUI files (webgui directory in the repository) to $MWROOT/usr/local/www. Copy the captive portal files (captiveportal directory in the repository) to $MWROOT/usr/local/captiveportal. Make sure that all *.php files have the execute permission bits set (chmod 755).

1.13. Creating mfsroot

First of all, a zero-filled image file needs to be created. The current size (as of m0n0wall 1.21) is 11 MB, which should be enough to fit all the files in the root file system. If you add big components, you may need to increase this. However, don't make it much bigger than necessary, as the MFS will take up as much space as you specify here, even if the file system is not full. Put mfsroot somewhere outside of $MWROOT!

dd if=/dev/zero of=mfsroot bs=1k count=11264

Now we'll use vnconfig to attach the file to /dev/vn0:

vnconfig -s labels -c vn0 mfsroot

After that, it's possible to use disklabel and newfs to create an UFS file system, and mount it:

disklabel -rw vn0 auto
newfs -b 8192 -f 1024 -o space -m 0 /dev/vn0c
mount /dev/vn0c /mnt

The root file system that you prepared in $MWROOT can now be transferred into /mnt.

tar should be used for this task, as it correctly preserves hard links (unlike cp) and thereby saves space:

cd /mnt
tar -cf - -C $MWROOT ./ | tar -xvpf -

Cleaning up and gzipping the image (step out of /mnt first):

umount /mnt
vnconfig -u vn0
gzip -9 mfsroot

1.14. Putting it all together: creating the image

This is very similar to creating the mfsroot. Again, a zero-filled image file needs to be created first. This needs to be big enough to accomodate the kernel, mfsroot, boot loader and config.xml. As of 1.21, 7 MB is used.

dd if=/dev/zero of=image.bin bs=1k count=7168
vnconfig -s labels -c vn0 image.bin

This time, the first and second stage boot loaders need to be written into the special reserved area at the beginning of the image too.

disklabel -Brw -b $BOOTDIR/boot1 -s $BOOTDIR/boot2 vn0 auto
disklabel -e vn0

You will be dropped into your favorite text editor. Duplicate the line that starts with c:, and change the c: to a: and the fstype from unused to 4.2BSD. Now create and mount the root file system:

newfs -b 8192 -f 1024 -o space -m 0 /dev/vn0a
mount /dev/vn0a /mnt

Copy kernel.gz, mfsroot.gz, the boot loader and the default config file (which is going to be the initial configuration of your new image):

cp /sys/compile/M0N0WALL_[PLATFORM]/kernel.gz /mnt
cp mfsroot.gz /mnt
mkdir /mnt/boot
cp $BOOTDIR/{loader,loader.rc} /mnt/boot
mkdir /mnt/conf
cp $MWROOT/conf.default/config.xml /mnt/conf

Time to finalize the image (step out of /mnt first):

umount /mnt
vnconfig -u vn0
gzip -9 image.bin

Et voilà! Rename the image as per standard m0n0wall conventions to make sure that the webGUI firmware upgrade works (which insists that the filename starts with the platform name followed by a dash):

mv image.bin.gz [platform]-[version].img

1.15. Creating a bootable CD-ROM version

mkisofs needs to be installed first (use /usr/ports/sysutils/cdrtools).

Create a new directory to hold the contents of the CD-ROM in $CDROOT. Copy your kernel.gz and mfsroot.gz to it. Add the boot loader as follows:

mkdir $CDROOT/boot
cp $BOOTDIR/{cdboot,loader,loader.rc} $CDROOT/boot

Now use mkisofs to create the bootable ISO image:

mkisofs -b "boot/cdboot" -no-emul-boot -A "m0n0wall CD-ROM image" \
        -c "boot/boot.catalog" -d -r -publisher "foo.com" \
        -p "Your Name" -V "m0n0wall_cd" -o "m0n0wall.iso" \
        $CDROOT

Chapter 2. m0n0wall Hacker's Guide

Note

This chapter is somewhat dated, though much of it remains applicable. The most recent m0n0wall development documentation is the image building guide chapter.

This chapter is based on Rudi van Drunen's m0n0wall Hackers Guide, used with permission.

m0n0wall is open-source software: if it doesn't quite do what you want, you can change it yourself, or have someone else of your choice change it for you. This chapter describes how to do that.

Note that the instructions in this guide are meant as guidelines, your mileage may vary. Also, hacking low level things may seriously mess up your development system, target system, or other systems, please take care.

2.1. Setting Up a Host Development Environment

In order to get a development environment for m0n0wall you will need the target hardware and a host system. typically this will be a soekris NET45xx board and a FreeBSD 4.8-RELEASE (Intel x86) system.

You will be booting the m0n0wall from the host system, so no CF card is needed. The boot image is located on the host and changes in the kernel or root filesystem are easily made by hacking them on the host machine. Once ready, the root filesystem or kernel image can be created and put in the /tftpboot directory (as described below) and the target (soekris) can be booted from this image over the network.

Now you can test the image, make your desired changes on the host, rebuild the root and / or kernel and test again. If you are confident with the changes you made, you will be able to build an image for the CF card. This image can then be used to "firmware upgrade" the m0n0wall (when running from CF) as usual though the WEB user interface.

As you will be network (PXE (Preboot Execution Environment)) booting the soekris from your host system and be able to generate image files to load you will have to set up some environment on the host to support just that. First of all, be sure that you have a kernel running that contains the vnode driver. You will need this feature to be able to turn a file into a device. Add the following to your kernel config and rebuild the kernel:

pseudo-device   vn              # Vnode driver (turns a file into a device)

Instructions for rebuilding the kernel can be found in chapter 9 of the FreeBSD handbook.

2.2. PXE Booting

The PXE boot environment needs a DHCP server on the subnet and a tftp server on the subnet, reachable by the soekris. The pxeboot provided with FreeBSD (at least the 4.8 build) is not compiled with TFTP support. If you're running NFS this is not a problem as you can put the files in a NFS accessible directory. To use TFTP (which seems easier) you need to do the following:

First edit your make file config (/etc/make.conf) and add the following line:

LOADER_TFTP_SUPPORT=YES

Then, rebuild the pxeboot file:

cd /usr/src/sys/boot
make clean
make depend
make

And copy the just created pxeboot file to /tftpboot:

cp /usr/src/sys/boot/i386/pxeldr/pxeboot /tftpboot

2.2.1. DHCP server

Check if you have a (running) DHCP server on the network where the soekris (and the development system) are on. If not, build isc-dhcp from the ports tree with:

cd /usr/ports/net/isc-dhcp3; make all install clean

PXE boot needs some extra entries in the dhcp record for the (soekris) target. It needs the (tftp)server address from which to retrieve the bootfile (denoted by the next-server keyword) and the actual name of the bootfile (denoted by filename). An example of the dhcp record for the soekris is shown below:

host soekris {                
     hardware ethernet 0:0:24:ad:bc:ac;                                     
     fixed-address soekris.example.net;
     filename "pxeboot"; 
     next-server 192.168.1.2;
     option root-path "/tftpboot";
 }

Here we assume the SOEKRIS having the ip number refrenced by the name soekris.example.net. You could put the ip address here (when not running DNS). The host system (the machine where the SOEKRIS is booted from) is assumed to have the address 192.168.1.2 in this example. Please change these addresses to reflect your network setup.

2.2.2. tftp server

To enable the tftp server to be started from inetd (you should be running inetd, did I mention this ??) uncomment the following line in /etc/inetd.conf:

tftp  dgram  udp wait root /usr/libexec/tftpd  tftpd -s  /tftpboot

Now, restart inetd:

kill -HUP `cat /var/run/inetd.pid`

2.3. Setting up m0n0wall on the host

2.3.1. The Kernel

In order to get m0n0wall to build we have to compile the kernel for m0n0wall using the kernel config file as found on http://m0n0.ch/wall/downloads. Place this config file (M0N0WALL_NET45XX) in /usr/src/sys/i386/conf. Now build the kernel:

cd /usr/src/sys/i386/conf; config M0N0WALL_NET45XX
cd /usr/src/sys/compile/M0N0WALL_NET45XX; make depend all          
strip kernel         
strip --remove-section=.note --remove-section=.comment kernel
gzip -9v kernel

Copy the kernel to /tftpboot:

cp kernel.gz  /tftpboot

2.3.2. The Modules

cd /usr/src/sys/compile/M0N0WALL_NET45XX; make modules

Then, move the needed modules to the modules directory in the m0n0wall root filesystem. In the pb8 version of m0n0wall the following modules are needed:

dummynet.ko
ipfw.ko

These newly-built modules can be found in /usr/src/sys/compile/M0N0WALL_NET45XX/modules/usr/src/sys/modules. modules directory).

2.3.3. The root filesystem

Fetch the root filesystem tar file from the m0n0wall web site to a directory, uncompress and untar. The contents of this directory will be in the root of the target system later on. In this just created directory you will be making the changes you like. As we will not not have mounted compact flash card on-line (under /cf), (you could, just put it in, but make sure it boots from the net instead of the flash) we will have to relocate the (default) config file in the root directory:

mkdir cf/conf ; cp conf.default/config.xml cf/conf

Now make a tarfile again to be put onto the to imagefile:

tar cfz ./rootfs.tgz <path to your rootfs-dir>

Now, you can create an imagefile (mfsroot) from this rootfilesystem. This imagefile has to be put into /tftpboot to be downloaded during boot.

dd if=/dev/zero of=./mfsroot.bin bs=1k count=10240
vnconfig -s labels -c vn0 ./mfsroot.bin
disklabel -rw vn0 auto
newfs -b 8192 -f 1024 /dev/vn0c

Now mount this file as device and copy the m0n0wall root filesystem in:

mount /dev/vn0c /mnt
cd /mnt
tar xfzP rootfs.tgz
cd /
umount /mnt
vnconfig -u vn0

Now your file mfsroot.bin file is the rootfilesystem image. When this image is put into /tftpboot it will be loaded and unpacked in memory once the kernel boots.

mv mfsroot.bin /tftpboot

2.3.4. For the impatient

Another way to get the kernel.gz file without compiling is extracting it from the net45xx-pbxrxxx.bin.gz image. To do just that, uncompress the image file and mount it as device under /mnt.

The net45xx-pbxrxxx.img files have also to be uncompressed first (check with file < filename >) . Just append a .gz at the filename and gzip -d the resulting file.

gzip -d net45xx-pbxrxxx.bin.gz
vnconfig -s labels -c vn0 ./net45xx-pbxrxxx.bin
mount /dev/vn0a /mnt
cp /mnt/kernel.gz /tftpboot
umount /mnt
vnconfig -u vn0

The root file system is also in the abovementioned image as the file mfsroot.gz. You can use this file to reconstruct the root file system by uncompressing and mounting it as device /dev/vx0c under /mnt.

gzip -d mvfsroot.gz
vnconfig -s labels -c vn0 ./mfsroot
mount /dev/vn0c /mnt
cd /mnt
tar cvf /tmp/mfs.tgz .
umount /mnt
vnconfig -u vn0
cd 
tar xvfzP /tmp/mfs.tgz

2.3.5. The loader and pxe config

The bootloader has to be available in the /tftpboot directory and has to be configured to load kernel.gz and the mfsroot.bin file. To do that make the following changes to the loader and configure pxeboot: create the following files: loader.conf:

rootfs_load="YES"
rootfs_name="mfsroot.bin"
rootfs_type="mfs_root"
autoboot_delay=1

loader.rc:

include /boot/loader.4th
start

and populate the /tftpboot directory:

mkdir -m 0755 -p /tftpboot/boot/defaults
cp -p /boot/loader /tftpboot/boot/
cp -p /boot/*.4th /tftpboot/boot/      
cp -p /boot/defaults/loader.conf /tftpboot/boot/defaults/
cp -p loader.conf loader.rc /tftpboot/boot/
chown -R root:wheel /tftpboot

2.3.6. GO

Now boot the stuff....

Remember to turn on dhcp (if needed):

/usr/local/sbin/dhcpd

Now you can test you m0n0wall system. If you edit / cange something in the root filesystem, or build a new kernel, do not forget to update your mfsroot.bin or kernel.gz file in the /tftpboot directory. Also remember that you have a virtual read-only memory filesystem, (nothing will be written back to the mfsroot.bin file on the host) and no flash, so changes in configuration will not be stored.

2.4. NFS

2.5. Unpacking/Editing/Packing the Source

The next thing is, if you are really confident with your system to create a new image that you can use to upgrade tha m0n0wall flash. The best way to do that is build an image like Manual has on his web site, so you can update the m0n0wall using the GUI tool. First reconstruct the root filesystem to its initial state with respect to the link and the location of the config file: (not really needed, the CF card will be mounted "over" the /cf directory even when not empty) but to keep everything as clean as possible you might do that

rm -rf cf/conf

Creating a flash image works about the same way as creating a rootfs file, but we will need a disklabel that suits the flash card. After creating a file device, we will be putting the kernel.gz file, the rootfilesystem file, the conf directory and the default configuration on the card (file). Also needed is the /boot directory, containing the boot loader files A suitable disklabel (put it in the label.proto file) might be:

# /dev/vn0c:
type: unknown
disk: amnesiac
label: 
flags:
bytes/sector: 512
sectors/track: 32
tracks/cylinder: 64
sectors/cylinder: 2048
cylinders: 5
sectors/unit: 10240
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # milliseconds
track-to-track seek: 0  # milliseconds
drivedata: 0
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a:    10240        0    4.2BSD     1024  8192    26   # (Cyl.    0 - 4)
  c:    10240        0    unused        0     0         # (Cyl.    0 - 4)

First you might compress the rootfilesystem to save space:

gzip -9 mfsroot.bin; mv mfsroot.bin mfsroot.gz

Create the boot directory for inclusion on the flash image, and populate it with the appropriate files:

mkdir -m 0755 -p boot; cd boot
cp /boot/boot? .
cp /boot/loader .
cp /boot/loader.help .
cp /boot/loader.4th .
cp /boot/mbr .
cp /boot/support.4th .
mkdir -m 0755 -p defaults
cp /boot/defaults/loader.conf defaults

Now create the custom files for the loader:

loader.conf:

kernfs_load="NO"                # Kernel filesystem

loader.rc:

load /kernel
load -t mfs_root /mfsroot
autoboot 0

Now you might start building the actual memory filesystem image ....

dd if=/dev/zero of=image.bin bs=1k count=5120
vnconfig -s labels -c vn0 image.bin
disklabel -BR vn0 label.proto
newfs -b 8192 -f 1024 /dev/vn0a

mount /dev/vn0a /mnt
cp -Rp boot /mnt
cp -p mfsroot.bin kernel.gz /mnt
mkdir /mnt/conf
cp -p /conf.default/config.xml /mnt/conf
umount /mnt
vnconfig -u vn0
gzip -9 image.bin

Now your new-and-improved-with-your-most-wanted-feature m0n0wall image is ready to be loaded !!

2.6. Writing a CF Image

2.7. Submitting Changes

To submit any changes you've made to m0n0wall, email them to the development list or email them to Manuel directly.

It's best to send to the development list, as that way if they do not get included in the base m0n0wall for some reason, they are available for others to use who might find them helpful.

Chapter 3. Other Documentation

There are several people who have written additional documentation for m0n0wall development which are beyond the scope of this manual, or which have not yet been incorporated into this manual. This chapter provides a reference to some of those sources to help you when you find yourself in a situation not covered in detail in this manual.

An Introduction to m0n0wall Development - Michael Iedema

Custom m0n0wall images howto - Jean-Francois Theroux

If you have written something on m0n0wall development, please email Chris Buechler to have it listed here.

Chapter 4. Development Frequently Asked Questions

This chapter contains development-related FAQ's. For non-development related FAQ's, see the main m0n0wall FAQ.

4.1. I don't have a FreeBSD box to develop new features for m0n0wall, is there an easier way?

You can use m0n0wall's built in features to test newly developed features. With this method you are not able to compile new binaries on the box, but are able to add php pages into the environment, as well as pre-compiled binaries.

  1. Develop the page

  2. Open a browser and point it to http://m0n0wall-ip/exec.php

  3. Upload the page and any binaries needed for it to function properly via exec.php.

  4. Execute the following to place php pages into the www root.

    cp /tmp/*.php /usr/local/www 
  5. If you need any binaries for the page to work, copy them into the appropriate places.

  6. Type in the address of your new page to try out your experiment.

Using this method, the image will revert after a reboot in case you messed something up. You may wish to write a script to move everything into its proper place and upload that along with your pages / binaries if the feature requires many files. Then simply execute the script in exec.php. After you've successfully tested the feature, and are happy with the result, submit your work to or the m0n0wall development list for it to be added to the project.

4.2. What patches have been applied to the FreeBSD source that is used in m0n0wall, and are they available?

The following patches have been applied to the FreeBSD kernel source that is used to build m0n0wall releases:

  • clock.c: fix writing the day of week back to the RTC (Soekris BIOS resets the date to 1-1-1970 if it's invalid - 0 is not a valid weekday as per the AT specification)

  • if_ethersubr.c: disable multicast warning in bridge code

  • if_xl.c: disable hardware TX checksumming

  • ip_input.c: fix problem with packets from dummynet pipes getting NATed again

  • ip_nat.c: fix ipfilter bug (only required for 3.4.31)

  • ip_output.c: reverse ipfilter/ipfw processing order

  • ng_pptpgre.c: reduce ACK timeout to 1 second, disable PPTP windowing

  • subr_diskslice.c: disable warning about partition size

They are available to download from http://m0n0.ch/wall/downloads/kernel-patches.tgz.

4.3. Where can I find the m0n0wall source code?

The m0n0wall source code can be found in rootfs-version.tar.gz on the downloads page.

4.4. Is there any information available on the unsupported extensions support?

The following mailing list posts have information about this unsupported feature of m0n0wall which is available as of version 1.1.

Extensions

Re: [m0n0wall] Announcement: Package functionality underserious consideration!

Re: [m0n0wall] Announcement: Package functionality underserious consideration!