Then I got the solution for you: introducing Majordomo Web API

I needed to sync an address database in eGroupware with different Majordomo mailing lists (every night). I did this by building a small API that looks like a normal asynchronous API – just with the difference that in be background a mail request to Majordomo is sent and the results are retrieved from an IMAP or POP3 account.

Get the code, or look at egroupware_update_mailinglists.php script for an example. It should be easy to build a modern AJAX web interface on top of the JSON-based RPC interface.

As it always is, our mailing list provider switched from Majordomo to Mailman only a couple weeks after I finished the scripts. So the work didn’t really pay off, but I hope it’s useful to somebody out there.

• if you use (and need) ACLs (the default POSIX ACLs), do not switch to NFSv4. It probably will take ten more years to be fully working (the speed of NFS development seems to be close to GNU Hurd).
• if you modify the “Domain” entry in /etc/idmapd.conf, modify it on all clients and the server, otherwise all users and groups will be mapped to “nobody”.
• I don’t know yet if the Firefox locking problems are gone (I hope so …)

As a side note, there will be no official release for XForms for Firefox 3.5. The XPI package available at my builds page has been tested by many people and works well, but no further development efforts will go into that version.

Yeah, documentation in an unstable wiki is great. I wonder if MDC uses HTTPS only to make it even slower …

http://philipp.wagner.name/cafeteriamenu/

Der Quellcode ist ebenfalls im Mercurial Repository verfügbar (nein, Code zum Parsen von HTML-Seiten ist nicht schön).

Have a lot of fun…

• Fix the cmake build files to work with newer KDE versions.
• Add caching and network status detection to the data engine. The menu for the next week is cached if you are online and available without network connection. The cached data is only used if no network connection is available and refreshed as soon as you’re online again.

Download the source packages:
cafeteria_engine-0.2.tar.bz2
cafeteria_plasmoid-0.2.tar.bz2

Build instructions are included in the README files.

I’ve added the code to my Mercurial repository as well.

source$ ssh root@192.168.66.2 -- nc -l 3333 \| dd of=/dev/VolGroupData/data1 &
source$ dd if=/dev/VolGroupData/data1 | nc 192.168.66.2 3333

dstat on the target host shows:

----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
[... removed first few lines until memory caches are full ...]
 0   1  88  10   0   0|  15M   15M|  16M  493k|   0     0 |  15k 9177
 0   1  88  10   0   0|  16M   17M|  17M  538k|   0     0 |  16k 9597
 0   1  89  10   0   0|  16M   16M|  17M  547k|   0     0 |  16k 9767
 0   1  88  10   0   0|  16M   15M|  17M  541k|   0     0 |  16k 9413

Only 17 MB/s over Gigabit ethernet? And where are the disk reads coming from? Well, after some more experimenting, the answer was not that hard: block size. dd uses a default block size of 512 Bytes, increasing it to 4 KB shows an entirely different picture:

source$ ssh root@192.168.66.2 -- nc -l 3333 \| dd obs=4K of=/dev/VolGroupData/data1 &
source$ dd ibs=4K if=/dev/VolGroupData/data1 | nc 192.168.66.2 3333

----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
[... removed first few lines until memory caches are full ...]
1   6  92   0   0   0|   0    74M|  77M 1821k|   0     0 |  57k   85k
1   7  92   0   0   0|1638B   73M|  77M 1819k|   0     0 |  59k   88k
1   8  91   0   0   0|   0    70M|  74M 1724k|   0     0 |  49k   72k
1   7  92   0   0   0|   0    70M|  74M 1737k|   0     0 |  56k   82k
1   6  92   0   0   0|   0    73M|  76M 1795k|   0     0 |  57k   86k
1   6  93   0   0   0|   0    72M|  75M 1779k|   0     0 |  56k   85k

Now, how about that? Average was in the end around 65 MB/s, which is much closer to what I expected.

First the good things:

• The speaker and microphone work perfectly out-of-the-box with zero configuration using the snd_usb_audio Linux kernel module.
• The same thing is true for the trackball and scrollwheel as well as the mouse buttons. They are recognized as regular USB mouse and instantly usable.

Not working out-of-the-box are the buttons on top and the status LEDs.

Now there are a couple different layers where you could start writing a driver for these buttons:

1. the kernel. Make it work like a keyboard device.
2. X11. Make it work like a keyboard device, too.
3. create a library that an application could like if it wants to provide support for the SpeechMike.

After some discussion on the linux-usb mailing list it became clear that the SpeechMike is not a usual USB HID device that could be easily supported by the existing code.

I then thought about the idea of the device being a keyboard a bit more: which keyboard events would you assign to the buttons? Use e.g. KEY_PLAY for the “Play” button? That would mean pressing the “Play” button on the SpeechMike would have the same meaning as pressing the “Play” button on your multimedia keyboard. Didn’t sound like a good idea to me. Use the more general events like F23? Doesn’t look like a good solution either.

I also found out that you can press an arbitrary number of buttons at the same time, like modifier keys (CTRL, …) on your normal keyboard. I wasn’t sure how that would work with a kernel driver or a X11 driver, so I finally decided to write a simple library which I could use in my application.

So here it is: libspeechmike. You can get the source code from my Mercurial (hg) repository at

http://philipp.wagner.name/hg/libspeechmike.

The source code includes a example program, speechmike_dump, that shows you which keys you pressed and can serve as a reference if you want to build your own application with it. It uses the hiddev interface, which is really easy to use without the need of another library (like libhid or libusb). You only need to make sure you have read permissions for the device file.

The LEDs are currently not working, I guess I’ll need to do some USB sniffing with the original Philips driver on Windows to get those working.

1. Use lighttpd (doesn’t give that much of a performance gain, but at least a bit). This requires PHP to run as FastCGI.
2. For PHP: use XCache (makes a huge difference as all the classes don’t need to be recompiled for each request)
3. Use lighttpd’s X-Sendfile support to make the webserver send the pictures to the client and get the load away from PHP.

For point 3 I needed to modify Gallery2, the patch (for Gallery 2.3) is available here:
http://philipp.wagner.name/sw/g2-sendfile.patch

To enable sendfile support, add this to your config.php:
$gallery->setConfig('useSendFile', true);

And don’t forget to enable X-Sendfile support in your webserver’s configuration!

This patch does not require Lighttpd, I’ve tested it with Apache and mod_xsendfile, too. Please note that Sendfile (as well as XCache) do not work with lighttpd 1.4 as soon as mod_cgi is enabled (they silently stop working, error reporting is not one of the strengths of lighttpd).

Of course I did no benchmarks (*gg*) but things feel much snappier now, and that’s all that counts.

That was actually half a year ago, but back then I didn’t find time to look more closely at it. First I thought I’d be able to write the Plasmoid using ECMA-Script (JavaScript). But it seems that the bindings are still under heavy development (like everything in the Plasma:: namespace) and not really usable yet. So I found a tutorial on how to write a Plasmoid using C++. Doing further reading, I also found out that I’d need to write a DataEngine to get the data, too.

It took me a while to fight my way throught the documentation *cough* ([1], [2], [3]) and understand the basic concepts, I think I got it mostly working now :-)

Before I continue with more talking, have a look:

The data comes from a webservice I operate, which currently gives the menus of all cafeterias of the Studentenwerk München. Drop me a note if you like it and would like to get your cafeteria’s menu in there, too.

So finally, here are the sources:
The data engine: cafeteria_engine-0.1.tar.bz2
The plasmoid: cafeteria_plasmoid-0.1.tar.bz2

I did development on openSUSE 11.0 using the Factory KDE repository (which currently contains KDE 4.1.2).

This is getting long, so I’ll postpone my rant about missing documentation to the next blog post ;-) I’d love to get some freeback on this, so feel free to drop me a note.

Update: I just noticed that sometimes after adding the plasmoid to the desktop for the first time and clicking on “Configure”, you’ll get a segfault with a backtrace like in bug #168278. To work around that, add the plasmoid to the desktop, log off and on again and configure it then (and possibly do the same again?). That way it should work. I guess this is a bug in Qt, but that’s not yet for sure (the QGraphics* stuff is pretty new and known to be buggy).

Update 2: There is a fix in qt-copy now, which will be part of Qt 4.4.4 and 4.5 (see here).