trusted root file system verification for Debian related distros like Ubuntu, EasyPeasy and others

Short Description:

debcheckroot is a tool comparable to checkroot that retrieves file checksums online and therefore can allow a trusted verification of your root file system at least as far as you can trust your internet connection or your verification medium (DVD, BD, etc.) respectively. This tool does not use signatures yet. However with regards to most private keys being systematically stolen by the NSA and other services this is a minor infirngement. On the contrary debcheckroot is more simple than checkroot, consequently less error prone and should thus currently be considered more trustworthy in comparence to its rpm-based counterpart.

Usage of debsums instead of Debian-checkroot is strongly discouraged because debsums uses locally stored md5sums which can be modified by an attacker along with the files themselves. It has been meant for integrity checking not for security issues! Debsums furthermore does not provide an output as clean and neatly structured as checkroot and does not spot files additionally added to your system by someone else.

debcheckroot v1.0

new improvements: see changelog
(at the beginning of the debcheckroot file)
Authors Email
Elmar Stellnberger

cross reference: checkroot for openSUSE

basic procedure / general hints

      Basically there are two ways to verify your system: online and via BD-DL (Blue Ray Double Layer). Which choice may be the better will depend on what threat you want to shield against. If you are a company which wants to shield against criminal endeavor of other parties then we would recommend you online verification and a well configured and updated system. However if your suspected enemies are foreign intelligence services then just updating and configuring your system well can not shield you as these services have their own intelligence to implant and spot backdoors in oss and non-oss software. There are plenty of such weaknesses in any kind of software written in C and just focusing on publicly known weaknesses by updating is ridiculous.

Consequently my suggestion for the second threat scenario is not to update but to install from BD-DL only; that even if your computer is held online. Note that this tool can in its current state not give you an ultimate security as it uses an overhauled checksuming algorithm: md5sums. However manipulating a file by keeping its size and md5sum may be hard enough as the targeted files would effectively have to be smaller than their originals because trailing garbage needs to be added to keep the md5sum. Nonetheless if this tool does not yield any result but you have a reason to be suspicious then simply install the same system with exactly the same packages in an empty partition from scratch and compare both root file systems file by file. This will be most effective.

Unfortunately no 'silver bullet' is given yet though there is an ongoing endeavour to make systems more secure. Checking all binaries requires a deterministic build process which is not given yet for *.pyc the python bytecode files. They may look different for every execution and can thus not be checked yet. Fortunately Debian is currently working on reproducible builds. If you do not have reason to think that your system has been cracked you may however delete these files after checking to make sure that everthing is as it should be.

The first thing to do when you experience your system being under attack is of course to put your machine offline and to keep it offline. Verify in a next step with BD or online after booting from a clean medium. Note that starting the verification out of an infected system will at best be useless (of course it is). debcheckroot has been designed the way that you can boot any system that provides perl, gzip, bzip2 and tar. These are the only required tools debcheckroot depends upon. Booting from CD or DVD is recommended as these media are read-only. debcheckroot does not do a chroot or use executables out of the infected system as would be required for debsums because this would infect your freshly booted system. Note that in todays world even the BIOS has become an attack vector. Consequently you would have to boot a fresh system, verify or flash the BIOS and then reboot a fresh system (see: flashrom/coreboot) to start the verification. You will need to check the boot sector i.e. at least the first 512 bytes of the MBR as well.

Alternatively you may want to unmount the hard disk and verify it in a computer which you can trust. Note that we can not guarantee your success as also the firmware of your hard disk could have been under attack. That way malware can be hidden either in additional backup blocks for use instead of bad sectors or simply on any place where your hdd returns read errors. Such errors can at best be distinguished from real hardware errors by doing a low level format of your drive with a vendor specific tool (like f.i. packed on the Ultimate boot CD): If the errors go away after flashing the HDD firmware or resetting the drive that may be suspicious.

read something more about the technical possibilities of NSA, MI6 and Mossad

current issues of interest:

    Why you should not use debsums in case of security issues
  • debsums requires you to chroot into the infected target root (inadmissible)
  • debsums uses locally stored md5sums which can be altered by an attacker rather than freshly downloaded checksums
  • debsums requires a full Debian infrastructure; it is much more hard or impossible to revise all these helper programs for possible vulnerabilities than just to allow perl, gzip, bzip2 and tar
i.e. debsums can be used to check for hardware failures or unmeant changes done by accident but not to assert the security of an installation. debsums gives you the hazardous illusion of protecting your security as long as you do not know what the tool really does.

    What makes debcheckroot the better tool
  • debcheckroot has full built-in support of repos; you can verify online and do not need to specify the individual location of each package to test against locally
  • debcheckroot fetches fresh package headers from the verification medium instead of local md5sum lists
  • it also lists additional/spurious files that should not be here / that are not under control of the package management system
  • debcheckroot stores the results in nicely formatted output files for later analysis; i.e. you do not have to pipe the screen output
  • you can easily spot all your manual alterations of a system which you wanna reinstall
  • debsums is much slower because it unpacks the package headers into temporary files; this however only where there are no locally stored md5sums
  • debsums can not download package headers via the internet or a local network
i.e. debcheckroot has many advantages also with regards to usability

    Future Improvements
  • give an option to create and use sha-256 lists instead of the md5sums provided by Debian
  • allow usage of multiple changeable media: i.e. DVD & BD-SL verification rather than just BD-DL verification
  • support file only repos
  • generate md5sums on the fly for packages that do not include such a list
  • Debian: the reproducible builds project will increase the range of file-coverage (e.g. *.pyc)

In questions of support you may write me a letter; if you wanna discuss debcheckroot on a mailing list it would be nice if you advised me.

*** other interesting content from elstel ***

usage hints:

Basically for an online verification there is nothing more to do than to enter debcheckroot /rootdir; for a verification from BD you need to specify the mount point where you have mounted or wanna mount your installation image like debcheckroot --cdmp /media/cdrom0 /rootdir. Note that it will look for the mount point inside /rootdir unless you specify --absreporoot.

If the arch can not be determined automatically from /etc/motd you need --arch i386/amd64/ia64. If your local package database has been destroyed for some reason then you may want to specify the location of a backup database by --dpkgdir /media/usbstick/var/lib/dpkg where the status and the diversions files need to reside. Use dpkg-divert if you need to rename a file that has been installed from a .deb package because otherwise the package database and thus debcheckroot will not know about it.

You may even want to use a different sources.list f.i. when you have installed from DVD but now want to verify against a freshly purchased and downloaded BD-DL: Simply use the --srclis parameter to achieve this. You may also add individual repos manually by the --repo switch which takes exactly the same command line format as seen in the lines of /etc/[apt/]sources.list. f.i. you could have removed a repo from sources.list but still have some packages installed from there. It may be a good practice to have an own sources-debchckroot.list which lists all repsoitories ever used.

Finally you may get an additional list of all installed packages called pkgs.lis and a list of modified packages with download URL called 2download.lis with the --installinfo switch which may be practical for creating an exact mirror installation or for manually restoring an altered installation.

It is also possible to do a quick verification of a set of core packages only with the --core switch where you can add additional packages to be checked by the --packages=pkg1,pkg2,... switch.

In order to check the MBR (Master Boot Record) you may simply read it out with dd: dd if=/dev/sda of=my.mbr bs=512 count=1

ouptut of debcheckroot:
verified-interesting.annot .... files which have changed; and their package (same format as for checkroot)
unchecked-packages.lis ... packages which are either provided by no repo or have no built-in md5sum list
unchecked.lis ... unchecked files not covered by the md5sum list in the package header (f.i. config files, autogenerated files): need to be checked manually. missing-files.lis ... files that should have been installed but which are no more there & the package to be reinstalled
pkgs.lis ... list of all installed packages & their version and arch separated by space (--installinfo switch only)
2download.lis ... list of all packages that needed to be reinstalled to get an unaltered system along with the URL from where to download the package

get informed about site updates via rss!  (right click: add with Akregator)