Did you ever want to sync your Majordomo mailing list with another address database? Or make it easy for users to unsubscribe using a web interface? Then you will have noticed that Majordomo only offers you a mail interface or an admin tool that you have install on your mailserver. So what if you don’t have access to the mailserver?

Then I got the solution for you: introducing Majordomo Web API

I needed to sync an address database in eGroupware with different Majordomo mailing lists (every night). I did this by building a small API that looks like a normal asynchronous API – just with the difference that in be background a mail request to Majordomo is sent and the results are retrieved from an IMAP or POP3 account.

Get the code, or look at egroupware_update_mailinglists.php script for an example. It should be easy to build a modern AJAX web interface on top of the JSON-based RPC interface.

As it always is, our mailing list provider switched from Majordomo to Mailman only a couple weeks after I finished the scripts. So the work didn’t really pay off, but I hope it’s useful to somebody out there.

After a (too) long time, we’ve finally got the XForms release 0.8.7 for Firefox 3.6 out the door. Get it from addons.mozilla.org while it’s fresh! It fixes many bugs, especially visible will be the better-looking select1 dropdown boxes on Linux and Mac.

Please consult the release announcement for a complete list of changes.

Known bugs:
Inline schemas don’t work. As a workaround, use an external schema. (bug #567628)

I have several PCs mounting its home directory as well as other data directories from a NFS server. Until now I was using good (?) old NFSv3. Today I tried a switch to NFSv4 in order to decrease locking the problems Firefox has with its sqlite databases being on a NFS filesystem. Quick notes:

• if you use (and need) ACLs (the default POSIX ACLs), do not switch to NFSv4. It probably will take ten more years to be fully working (the speed of NFS development seems to be close to GNU Hurd).
• if you modify the “Domain” entry in /etc/idmapd.conf, modify it on all clients and the server, otherwise all users and groups will be mapped to “nobody”.
• I don’t know yet if the Firefox locking problems are gone (I hope so …)

As a developer you’re used to deal with strange bugs on a daily basis. The one today was one of that kind. The Python script I use for building the XPIs uploads the finished files using HTTP POST. For building the POST data it uses MultipartPostHandler which I found on the web.

But for some time now I got a Python exception: gaierror: (-2, 'Name or service not known').  And no, it’s not my name servers, those are up and running. Causing the problem was the following line:

 boundary = mimetools.choose_boundary()

Somehow gethostbyname() screwed up (we were switched to new DHCP servers recently) and looking up our own hostname failed. At least it took me some time to look at the right place for that exception. When opening an URL you expect such an exception usually not inside a function giving you a MIME boundary :-)

After all, the XForms builds for Linux are up and running again, now for mozilla-1.9.2 and mozilla-central. Please give these builds a try and report any regressions you find as we plan to release a version for Firefox 3.6 around March.

Firefox 3.6 will be released today, Januar 21st. Unfortunately the XForms extension for this version is not yet ready. We’re planning an official release for Firefox 3.6 within this quarter, see bug 539184 for details. The nightly builds for 3.6 are currently broken as well, I hope to get this fixed until the end of the week.

As a side note, there will be no official release for XForms for Firefox 3.5. The XPI package available at my builds page has been tested by many people and works well, but no further development efforts will go into that version.

I was finally able to restore the nightly builds for Firefox 3.5 (the switch to Mercurial required a rewrite of the build scripts as well). It should be working now, have a look at the build page for the details.

Another quick note: XForms for Firefox 3.6 is on its way.

Yeah, documentation in an unstable wiki is great. I wonder if MDC uses HTTPS only to make it even slower …

Damit alle, die kein KDE4 benutzen (oder sich noch nicht von Ubuntu trennen können), dennoch in Genuss des Mensa-Speiseplans bekommen, habe ich die RSS-Feeds aktualisiert (die gab es schon länger, aber eher so als Service unter Freunden):

http://philipp.wagner.name/cafeteriamenu/

Der Quellcode ist ebenfalls im Mercurial Repository verfügbar (nein, Code zum Parsen von HTML-Seiten ist nicht schön).

Have a lot of fun…

The semester starts again and I took the chance to get back to the cafeteria plasmoid and fix some old issues. Changes include:

• Fix the cmake build files to work with newer KDE versions.
• Add caching and network status detection to the data engine. The menu for the next week is cached if you are online and available without network connection. The cached data is only used if no network connection is available and refreshed as soon as you’re online again.

Download the source packages:
cafeteria_engine-0.2.tar.bz2
cafeteria_plasmoid-0.2.tar.bz2

Build instructions are included in the README files.

I’ve added the code to my Mercurial repository as well.

Today I had to copy several LVM partitions to another machine. The first tools I thought of were dd and netcat (nc), the swiss army knife for every sysadmin. But the following two lines didn’t make me happy:

source$ ssh root@192.168.66.2 -- nc -l 3333 \| dd of=/dev/VolGroupData/data1 &
source$ dd if=/dev/VolGroupData/data1 | nc 192.168.66.2 3333

dstat on the target host shows:

----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
[... removed first few lines until memory caches are full ...]
 0   1  88  10   0   0|  15M   15M|  16M  493k|   0     0 |  15k 9177
 0   1  88  10   0   0|  16M   17M|  17M  538k|   0     0 |  16k 9597
 0   1  89  10   0   0|  16M   16M|  17M  547k|   0     0 |  16k 9767
 0   1  88  10   0   0|  16M   15M|  17M  541k|   0     0 |  16k 9413

Only 17 MB/s over Gigabit ethernet? And where are the disk reads coming from? Well, after some more experimenting, the answer was not that hard: block size. dd uses a default block size of 512 Bytes, increasing it to 4 KB shows an entirely different picture:

source$ ssh root@192.168.66.2 -- nc -l 3333 \| dd obs=4K of=/dev/VolGroupData/data1 &
source$ dd ibs=4K if=/dev/VolGroupData/data1 | nc 192.168.66.2 3333

----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
[... removed first few lines until memory caches are full ...]
1   6  92   0   0   0|   0    74M|  77M 1821k|   0     0 |  57k   85k
1   7  92   0   0   0|1638B   73M|  77M 1819k|   0     0 |  59k   88k
1   8  91   0   0   0|   0    70M|  74M 1724k|   0     0 |  49k   72k
1   7  92   0   0   0|   0    70M|  74M 1737k|   0     0 |  56k   82k
1   6  92   0   0   0|   0    73M|  76M 1795k|   0     0 |  57k   86k
1   6  93   0   0   0|   0    72M|  75M 1779k|   0     0 |  56k   85k

Now, how about that? Average was in the end around 65 MB/s, which is much closer to what I expected.