Collectd Graph Panel v0.3

It has been a while and many people were already using one of the development versions of CGP. Time to release a new version of CGP: v0.3.

Special thanks for this version go to Manuel CISSÉ, Jakob Haufe, Edmondo Tommasina, Michael Stapelberg, Julien Rottenberg and Tom Gallacher for sending me patches to improve CGP. Also thanks to everyone that replied after the previous release.

In this version there are a couple of minor improvements and some new settings to configure. But the most important: Support has been added for 13 Collectd plugins. CGP now supports 26 Collectd plugins. Also a couple of plugins have been updated. Please read the changelog or git log for more information about the changes.

New plugins:

Download the .tgz package, the patch file to upgrade from v0.2 or checkout the latest version from the git repository.

Download: http://pommi.nethuis.nl/…/cgp-0.3.tgz
Patch: http://pommi.nethuis.nl/…/cgp-0.2-0.3.patch.gz
Git: http://git.nethuis.nl/pub/cgp.git

Collectd Graph Panel v0.2

Version 0.2 of Collectd Graph Panel (CGP) has been released. This version has some interesting new features and changes since version 0.1.

A new interface is introduced, based on Daniel Von Fange’s styling. A little bit of web 2.0 ajax is used to expand and collapse plugin information on the host page. The width and heigth of a graph is configurable, also the bigger detailed one. UnixSock flush support is added to be able to let collectd write cached data to the rrd file, before an up-to-date graph of the data is generated. CPU support for Linux 2.4 kernel and Swap I/O support is added. And some changes under the hood.

Download the .tgz package, the patch file to upgrade from v0.1 or checkout the latest version from my public git repository.

CGP Overview Page CGP Server Page CGP Detailed Page

Download: http://pommi.nethuis.nl/…/cgp-0.2.tgz
Patch: http://pommi.nethuis.nl/…/cgp-0.1-0.2.patch.gz
Git: http://git.nethuis.nl/pub/cgp.git

Collectd Graph Panel v0.1

Collectd Graph Panel (CGP) is a graphical web front-end for Collectd written in php. I’ve developed CGP because I wasn’t realy happy with the existing web front-ends. What I like to see in a front-end for Collectd is clear overview of all the hosts you are managing and an easy way to get detailed information about a plugin.

This first release is a very basic version of what I have in mind. Not all plugins are supported yet and the user interface can be done better. Still a lot to work on!

Please give it a try! It should work out-of-the-box. Download the tgz package or checkout the latest version from my public git repository.

Download: http://pommi.nethuis.nl/…
Git: http://git.nethuis.nl/pub/cgp.git

Postgrey p0f patch

Postgrey is a Postfix policy server implementing greylisting. This patch adds p0f support to Postgrey based on p0fq.pl from p0f, like the patch by Fedux. P0f is a fingerprint tool, to identify operating systems. The difference between Fedux’s patch and mine is that his patch requires p0f-analyzer. See the image below.

p0f-analyse-postgrey

My patch uses p0f’s socket created with the ‘-Q’ option. Because Postgrey doesn’t know what the source port is of the sender, p0f must also be configured with the ‘-0’ option.

my patch

Example usage:

# p0f
p0f -u postgrey -Q /var/run/p0f-sock -0 -N -i eth0 'tcp dst port 25'

# postgrey options
--p0f --p0f-service=/var/run/p0f-sock --p0f-ip=<ip-of-eth0>

Postgrey version: 1.32
Download: pommi.nethuis.nl/..

ext3 overhead on 1TB storage

Recently I bought a portable harddrive from Western Digital. The Western Digital Elements (WDE1U10000E) carries 1 Terabyte of space and can be connected via USB 2.0. 1TB, for just 99,00 Euro (2008-12-27). According to Wikipedia and the SI standard the drive must contain 1,000,000,000,000 bytes (1TB). fdisk shows us:

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes

So that is correct.

The disk is preformatted FAT32. After mounting the disk, df shows me there is actually 976283280 KiB (932GiB) available. This is about 999714078720 bytes. It looks like 490807296 bytes (468MiB) is gone, but it must be used for File Allocation Tables.

Because FAT32 is crappy old and I use Linux and want a journaling filesystem, I will reformat the device with ext3. After setting the right partition table type via fdisk.

$ mkfs.ext3 -m0 /dev/sda1
mke2fs 1.41.3 (12-Oct-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
61054976 inodes, 244190000 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
7453 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000, 214990848

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 36 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

After a minute or 10 (yeah! USB 2.0) it was all done. First of all 2646016 bytes (2584 KiB) is not formatted (1000204886016 – (244190000 * 4096)). After mounting the disk, df shows me this time there is 961432072 KiB (917 GiB) available. This is less then FAT32, but we have a journaling filesystem now. 15327928 (97676000 – 961432072) KiB is used for that. But why and how?

dumpe2fs /dev/sda1 shows us:

dumpe2fs 1.41.3 (12-Oct-2008)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          0bdd2888-06fc-4b22-a6e5-987ac65236ee
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Filesystem flags:         signed_directory_hash
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              61054976
Block count:              244190000
Reserved block count:     0
Free blocks:              240306876
Free inodes:              61054965
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      965
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Filesystem created:       Sun Dec 28 14:22:22 2008
Last mount time:          Sun Dec 28 14:35:20 2008
Last write time:          Sun Dec 28 14:35:20 2008
Mount count:              1
Maximum mount count:      36
Last checked:             Sun Dec 28 14:22:22 2008
Check interval:           15552000 (6 months)
Next check after:         Fri Jun 26 15:22:22 2009
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      0bda5622-6cc2-4a1a-8135-c3f810580d43
Journal backup:           inode blocks
Journal size:             128M
  • /dev/sda1 has 7453 block groups.
  • Inode size is 256 bytes.
  • 8192 inodes for each block group.

7453 * 256 * 8192 makes 15630073856 bytes (15263744 KiB) for inode space.

15327928 – 15263744 = 64184 unexplained KiB left

Besides one primary superblock, 18 extra backup superblocks are stored on the disk. A superblock is 256 bytes, though it is stored in a 4 KiB block. 19 * 4096 makes 77824 bytes (76 KiB).

64184 – 76 = 64108 unexplained KiB left

If someone has an explanation for it, please leave a reply.

61054976 inodes means there can be stored over 61 million files on the formatted 917 GiB. This is way too much for me. 10% of it is enough too, so there will also be less space needed for storing the inodes. Formatting the disk with option -i 131072 sould better fit me.

TorrentFlux quota patch

For downloading torrents I use TorrentFlux on a Debian Linux machine. All users on this system have a quota, so they don’t mess things up. TorrentFlux shows a storage bar on the main page with information about how much space is left on the device. Unfortunately it shows information about the complete partition, not about how much of your quota is left. This patch adds quota support and is backwards compatible.

Download: pommi.nethuis.nl/..