trusted root file system verification for Debian related distros like Ubuntu, EasyPeasy and others

debcheckroot v1.0
new improvements: see changelog
(at the beginning of the debcheckroot file)
Authors Email
Elmar Stellnberger

basic procedure / general hints

Basically there are two ways to verify your system: online and via BD-DL (Blue Ray Double Layer). Which choice may be the better will depend on what threat you want to shield against. If you are a company which wants to shield against criminal endeavor of other parties then we would recommend you online verification and a well configured and updated system. However if your suspected enemies are foreign intelligence services then just updating and configuring your system well can not shield you as these services have their own intelligence to implant and spot backdoors in oss and non-oss software. There are plenty of such weaknesses in any kind of software written in C and just focusing on publicly known weaknesses by updating is ridiculous.

Consequently my suggestion for the second threat scenario is not to update but to install from BD-DL only; that even if your computer is held online. Note that this tool can in its current state not give you an ultimate security as it uses an overhauled checksuming algorithm: md5sums. However manipulating a file by keeping its size and md5sum may be hard enough as the targeted files would effectively have to be smaller than their originals because trailing garbage needs to be added to keep the md5sum. Nonetheless if this tool does not yield any result but you have a reason to be suspicious then simply install the same system with exactly the same packages in an empty partition from scratch and compare both root file systems file by file. This will be most effective.

Unfortunately no 'silver bullet' is given yet though there is an ongoing endeavour to make systems more secure. Checking all binaries requires a deterministic build process which is not given yet for *.pyc the python bytecode files. They may look different for every execution and can thus not be checked yet. Fortunately Debian is currently working on reproducible builds. If you do not have reason to think that your system has been cracked you may however delete these files after checking to make sure that everthing is as it should be.

The first thing to do when you experience your system being under attack is of course to put your machine offline and to keep it offline. Verify in a next step with BD or online after booting from a clean medium. Note that starting the verification out of an infected system will at best be useless (of course it is). debcheckroot has been designed the way that you can boot any system that provides perl, gzip, bzip2 and tar. These are the only required tools debcheckroot depends upon. Booting from CD or DVD is recommended as these media are read-only. debcheckroot does not do a chroot or use executables out of the infected system as would be required for debsums because this would infect your freshly booted system. Note that in todays world even the BIOS has become an attack vector. Consequently you would have to boot a fresh system, verify or flash the BIOS and then reboot a fresh system (see: flashrom/coreboot) to start the verification. You will need to check the boot sector i.e. at least the first 512 bytes of the MBR as well.

Alternatively you may want to unmount the hard disk and verify it in a computer which you can trust. Note that we can not guarantee your success as also the firmware of your hard disk could have been under attack. That way malware can be hidden either in additional backup blocks for use instead of bad sectors or simply on any place where your hdd returns read errors. Such errors can at best be distinguished from real hardware errors by doing a low level format of your drive with a vendor specific tool (like f.i. packed on the Ultimate boot CD): If the errors go away after flashing the HDD firmware or resetting the drive that may be suspicious.

read something more about the technical possibilities of NSA, MI6 and Mossad

current issues of interest:

i.e. debsums can be used to check for hardware failures or unmeant changes done by accident but not to assert the security of an installation. debsums gives you the hazardous illusion of protecting your security as long as you do not know what the tool really does.
i.e. debcheckroot has many advantages also with regards to usability

usage hints:

Basically for an online verification there is nothing more to do than to enter debcheckroot /rootdir; for a verification from BD you need to specify the mount point where you have mounted or wanna mount your installation image like debcheckroot --cdmp /media/cdrom0 /rootdir. Note that it will look for the mount point inside /rootdir unless you specify --absreporoot.

If the arch can not be determined automatically from /etc/motd you need --arch i386/amd64/ia64. If your local package database has been destroyed for some reason then you may want to specify the location of a backup database by --dpkgdir /media/usbstick/var/lib/dpkg where the status and the diversions files need to reside. Use dpkg-divert if you need to rename a file that has been installed from a .deb package because otherwise the package database and thus debcheckroot will not know about it.

You may even want to use a different sources.list f.i. when you have installed from DVD but now want to verify against a freshly purchased and downloaded BD-DL: Simply use the --srclis parameter to achieve this. You may also add individual repos manually by the --repo switch which takes exactly the same command line format as seen in the lines of /etc/[apt/]sources.list. f.i. you could have removed a repo from sources.list but still have some packages installed from there. It may be a good practice to have an own sources-debchckroot.list which lists all repsoitories ever used.

Finally you may get an additional list of all installed packages called pkgs.lis and a list of modified packages with download URL called 2download.lis with the --installinfo switch which may be practical for creating an exact mirror installation or for manually restoring an altered installation.

It is also possible to do a quick verification of a set of core packages only with the --core switch where you can add additional packages to be checked by the --packages=pkg1,pkg2,... switch.

In order to check the MBR (Master Boot Record) you may simply read it out with dd: dd if=/dev/sda of=my.mbr bs=512 count=1

ouptut of debcheckroot:
verified-interesting.annot .... files which have changed; and their package (same format as for checkroot)
unchecked-packages.lis ... packages which are either provided by no repo or have no built-in md5sum list
unchecked.lis ... unchecked files not covered by the md5sum list in the package header (f.i. config files, autogenerated files): need to be checked manually. missing-files.lis ... files that should have been installed but which are no more there & the package to be reinstalled
pkgs.lis ... list of all installed packages & their version and arch separated by space (--installinfo switch only)
2download.lis ... list of all packages that needed to be reinstalled to get an unaltered system along with the URL from where to download the package
get informed about site updates via rss!  (right click: add with Akregator)