Today I had to copy several LVM partitions to another machine. The first tools I thought of were dd and netcat (nc), the swiss army knife for every sysadmin. But the following two lines didn’t make me happy:

source$ ssh root@192.168.66.2 -- nc -l 3333 \| dd of=/dev/VolGroupData/data1 &
source$ dd if=/dev/VolGroupData/data1 | nc 192.168.66.2 3333

dstat on the target host shows:

----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
[... removed first few lines until memory caches are full ...]
 0   1  88  10   0   0|  15M   15M|  16M  493k|   0     0 |  15k 9177
 0   1  88  10   0   0|  16M   17M|  17M  538k|   0     0 |  16k 9597
 0   1  89  10   0   0|  16M   16M|  17M  547k|   0     0 |  16k 9767
 0   1  88  10   0   0|  16M   15M|  17M  541k|   0     0 |  16k 9413

Only 17 MB/s over Gigabit ethernet? And where are the disk reads coming from? Well, after some more experimenting, the answer was not that hard: block size. dd uses a default block size of 512 Bytes, increasing it to 4 KB shows an entirely different picture:

source$ ssh root@192.168.66.2 -- nc -l 3333 \| dd obs=4K of=/dev/VolGroupData/data1 &
source$ dd ibs=4K if=/dev/VolGroupData/data1 | nc 192.168.66.2 3333

----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
[... removed first few lines until memory caches are full ...]
1   6  92   0   0   0|   0    74M|  77M 1821k|   0     0 |  57k   85k
1   7  92   0   0   0|1638B   73M|  77M 1819k|   0     0 |  59k   88k
1   8  91   0   0   0|   0    70M|  74M 1724k|   0     0 |  49k   72k
1   7  92   0   0   0|   0    70M|  74M 1737k|   0     0 |  56k   82k
1   6  92   0   0   0|   0    73M|  76M 1795k|   0     0 |  57k   86k
1   6  93   0   0   0|   0    72M|  75M 1779k|   0     0 |  56k   85k

Now, how about that? Average was in the end around 65 MB/s, which is much closer to what I expected.

It took a bit longer than expected, but thanks to the great patience Doron, Aaron and bz had reviewing the last critical bug, it is finally here:

The XForms AddOn for Firefox 3.5. Download it now while it’s fresh :-)

The XPI contains binary components for Windows (32bit) and Linux (32 and 64bit).

Remember: this is an experimental development snapshot, so don’t be surprised if it kills your cat. But it should still be more stable than the previous release version, as we fixed some crash bugs.

I also added a bit of documentation on how to build Mozilla XForms from source at the Mozilla Developer Center.

Enjoy!

Firefox 3.5 is released and I’ve heard a couple people asking when XForms will be ready for Firefox 3.5. The answer is: soon. There is one critical bug to be resolved (bug #490983). As soon as this one is commited, I’ll start building nightly snapshots again and we’ll see if we can release an official version for Firefox 3.5. Stay tuned!

I recently got a Philips Speechmike device. It seems Philips has a Linux SDK for it, but I was unable to obtain it. I thought it wouldn’t be impossible to write a Linux driver for those couple buttons, so I gave it a try.

First the good things:

• The speaker and microphone work perfectly out-of-the-box with zero configuration using the snd_usb_audio Linux kernel module.
• The same thing is true for the trackball and scrollwheel as well as the mouse buttons. They are recognized as regular USB mouse and instantly usable.

Not working out-of-the-box are the buttons on top and the status LEDs.

Now there are a couple different layers where you could start writing a driver for these buttons:

1. the kernel. Make it work like a keyboard device.
2. X11. Make it work like a keyboard device, too.
3. create a library that an application could like if it wants to provide support for the SpeechMike.

After some discussion on the linux-usb mailing list it became clear that the SpeechMike is not a usual USB HID device that could be easily supported by the existing code.

I then thought about the idea of the device being a keyboard a bit more: which keyboard events would you assign to the buttons? Use e.g. KEY_PLAY for the “Play” button? That would mean pressing the “Play” button on the SpeechMike would have the same meaning as pressing the “Play” button on your multimedia keyboard. Didn’t sound like a good idea to me. Use the more general events like F23? Doesn’t look like a good solution either.

I also found out that you can press an arbitrary number of buttons at the same time, like modifier keys (CTRL, …) on your normal keyboard. I wasn’t sure how that would work with a kernel driver or a X11 driver, so I finally decided to write a simple library which I could use in my application.

So here it is: libspeechmike. You can get the source code from my Mercurial (hg) repository at

http://philipp.wagner.name/hg/libspeechmike.

The source code includes a example program, speechmike_dump, that shows you which keys you pressed and can serve as a reference if you want to build your own application with it. It uses the hiddev interface, which is really easy to use without the need of another library (like libhid or libusb). You only need to make sure you have read permissions for the device file.

The LEDs are currently not working, I guess I’ll need to do some USB sniffing with the original Philips driver on Windows to get those working.

It has been quiet around Mozilla XForms for a while now. But now Beta3 of Firefox 3.5 is out and it’s time to update the XForms plugin to get it released in time with the official Firefox release.

Mozilla switched from CVS to Mercurial (hg) with this release and removed the XForms extension from the core tree. We got a new hg repository, but it’s still empty (http://hg.mozilla.org/xforms). So what’s to do now?

1. Convert the extensions/xforms directory from CVS to hg.
2. Create a new hg repository on hg.mozilla.org for schema-validation and convert extensions/schema-validation to hg.
3. Make the XForms extension build with Firefox 3.5 (there has been some cleanup work towards Mozilla2).

To get started, I did the CVS to hg conversion for XForms and schema-validatio n.The results are available at http://philipp.wagner.name/hg. I mainly followed Gijs instructions for converting Venkman, so that’s the short version of how I did it:

1. Check out existing code from CVS
   $> cvs -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -d xforms-cvs mozilla/extensions/xforms
2. Make sure your ~/.hgrc file contains the following lines:
[extensions]
hgext.convert=
mq =

   [convert]
hg.usebranchnames=0
hg.clonebranches=1

3. Convert all branches
   $> hg --encoding utf8 convert xforms-cvs xforms-hg-initial
   This may take a while.
4. Remove the [convert] section from your ~/.hgrc file
5. Create the authors.txt file with a mapping of CVS usernames to names and e-mail addresses.
   For a list of authors use
   cd xforms-hg-initial/default && hg log | grep "user:" | sort | uniq | cut -c14- > ../../authors.txt
6. Convert the default repository to the final XForms repository:
   $> hg --encoding utf8 convert --authors authors-merged.txt xforms-hg-initial/default/ xforms-hg-final

I chose to “merge” authors, by mapping multiple e-mail addresses to one (the current one). For schema-validation I did exactly the same and the result is available  in my hg repository.

The final step is go get it all compiling. Using hg annotate I found the relevant bug reports that changed things and I made the required changes to XForms, which resulted in the patch attached to bug #485734.

Now the XForms extension compiles and works as it did before on Firefox 3.5. What’s left is to get all those outstanding patches into the code. To check it out, use

$> hg clone http://hg.mozilla.org/releases/mozilla-1.9.1 mozilla-1.9.1
cd mozilla-1.9.1
$> hg clone http://philipp.wagner.name/hg/xforms extensions/xforms
$> hg clone http://philipp.wagner.name/hg/schema-validation extensions/schema-validation
Then building works as it did before.

I just uploaded a Mozilla XForms XPI for OpenSolaris 2008.11 (x86). Give it a try!

http://philipp.wagner.name/mozilla-xforms

Building Mozilla on OpenSolaris is fairly straightforward, simply follow https://developer.mozilla.org/en/Solaris_Build_Prerequisites. I needed to disable the crash reporter (but that’s not needed for XForms anyways).

When I wrote

Of course I did no benchmarks (*gg*) but things feel much snappier now, and that’s all that counts.

I somehow knew that it would not take very long until the benchmarks would arrive. Actually, it took a whole day until it happened :-)

So, here are some quick benchmarks:

The gallery home page

This page contains several albums, a random image block and the default layout. The first chart contains requests per second, the second sets the values without X-Sendfile values to 100 % and compares them with the X-Sendfile enabled numbers.

As you can easily see, even on PHP-heavy pages with not that many pictures, the throughput increases by around 7 percent when using X-Sendfile. But even more important is the use of XCache, which gives you around 200 percent!

Now let’s have a look at a

Serving a single image (with security checks)

The tested image was inside a protected area which requires the user to log-in, so Gallery is not able to utilize its fast-path mechanism for pictures which don’t require any security checks.

Now that’s impressing, isn’t it? XCache gives you over 4 times as many requests/s! And X-Sendfile increases that number by around 30 percent.

All tests were conducted on my pretty slow (VMware) server, so the interesting stuff here is the percentages, not the absolute numbers. The numbers are from Apache’s ab with 20 concurrent requests and 100 requests total.

Conclusion: XCache (or some other bytecode cache) is a absolute must for Gallery (as well as for any heavily object-oriented PHP application, especially when using AJAX together with OO frameworks – but that’s another story).
X-Sendfile speeds the whole thing up even more and increases the responsiveness of your Gallery to give the user a smooth browsing experience (there’s nothing more annoying than to wait for pictures on a slide slow).

I have been using Gallery2 for my private picture collection for quite some time. It’s a great piece of software that shows how to write PHP code. Unfortunately, all the abstraction layers make it quite slow without additional help. I did three things to speed things up a bit:

1. Use lighttpd (doesn’t give that much of a performance gain, but at least a bit). This requires PHP to run as FastCGI.
2. For PHP: use XCache (makes a huge difference as all the classes don’t need to be recompiled for each request)
3. Use lighttpd’s X-Sendfile support to make the webserver send the pictures to the client and get the load away from PHP.

For point 3 I needed to modify Gallery2, the patch (for Gallery 2.3) is available here:
http://philipp.wagner.name/sw/g2-sendfile.patch

To enable sendfile support, add this to your config.php:
$gallery->setConfig('useSendFile', true);

And don’t forget to enable X-Sendfile support in your webserver’s configuration!

This patch does not require Lighttpd, I’ve tested it with Apache and mod_xsendfile, too. Please note that Sendfile (as well as XCache) do not work with lighttpd 1.4 as soon as mod_cgi is enabled (they silently stop working, error reporting is not one of the strengths of lighttpd).

Of course I did no benchmarks (*gg*) but things feel much snappier now, and that’s all that counts.

If you’re working in a company or go to university or college, that’s probably close to the most important question of the day. Wouldn’t it be great to have that information right on your desktop? While I was playing around with KDE4, I thought it would be great to have a Plasmoid (like a Dashboard Widget for all the Mac users out there) displaying today’s menu.

That was actually half a year ago, but back then I didn’t find time to look more closely at it. First I thought I’d be able to write the Plasmoid using ECMA-Script (JavaScript). But it seems that the bindings are still under heavy development (like everything in the Plasma:: namespace) and not really usable yet. So I found a tutorial on how to write a Plasmoid using C++. Doing further reading, I also found out that I’d need to write a DataEngine to get the data, too.

It took me a while to fight my way throught the documentation *cough* ([1], [2], [3]) and understand the basic concepts, I think I got it mostly working now :-)

Before I continue with more talking, have a look:

The data comes from a webservice I operate, which currently gives the menus of all cafeterias of the Studentenwerk München. Drop me a note if you like it and would like to get your cafeteria’s menu in there, too.

So finally, here are the sources:
The data engine: cafeteria_engine-0.1.tar.bz2
The plasmoid: cafeteria_plasmoid-0.1.tar.bz2

I did development on openSUSE 11.0 using the Factory KDE repository (which currently contains KDE 4.1.2).

This is getting long, so I’ll postpone my rant about missing documentation to the next blog post ;-) I’d love to get some freeback on this, so feel free to drop me a note.

Update: I just noticed that sometimes after adding the plasmoid to the desktop for the first time and clicking on “Configure”, you’ll get a segfault with a backtrace like in bug #168278. To work around that, add the plasmoid to the desktop, log off and on again and configure it then (and possibly do the same again?). That way it should work. I guess this is a bug in Qt, but that’s not yet for sure (the QGraphics* stuff is pretty new and known to be buggy).

Update 2: There is a fix in qt-copy now, which will be part of Qt 4.4.4 and 4.5 (see here).

Last weekend the patch for bug #437483 got committed (it looks like it went to CVS head and branch only, not to mozilla-central — no idea how that works now). This means that the install.rdf now contains a line like

<em:targetPlatform>Linux_x86-gcc3</em:targetPlatform>

The extension manager uses this information to determine if the addon will work on your platform. Unfortunately, this broke our build scripts and the nightly XPI refused to install.

A similar issue came up with the Firefox 3 release build, which didn’t work with Firefox 3.0.1 any more.

Both issues are fixed. Please re-download the XForms XPI and you should be ok.