How missing features sell stuff...

Developing software on and for Solaris can be real fun, considering all the tools available. OTOH, one is permanently confronted with the situation that there is a large installation base of Solaris 8 and 9. So people keep asking "Does this run on Solaris 8?".

Well, how could I be able to answer this question without testing or guessing? So what can I do? I don't really want to guess! Should I install Solaris 8 on a separate disk or get another machine? What about the idea installing Solaris 8 in a zone in Solaris 10? Sounds great and convenient, doesn't it? But unfortunately, I got the answer on opensolaris.org that this won't work. Too sad!

To come back to the topic: I just bought another SPARC, because Solaris is _missing_ this feature. What a weird world!

Let's look at this situation once more. Could it be that Sun has already made Solaris too good to be able to sell new machines and get customers to install the new Solaris 10, which is actually free of charge? Looking at the software vendor with the biggest market share, my theory seems to be confirmed. They can easily sell their new products, as people keep hoping that everything gets better in the next release. How sad.

BTW: the same seems to be true for all kinds of products. When did you buy new HIFI equipment? I don't think of MP3 players or something the like, but of real quality! Was it 10 years ago? 20? Today you even have a hard time buying real good quality Stereo amplifiers without tuner and lot's of things most people don't use and which certainly are no good for sound quality.

Now I sit and wait for Christmas - erh - for my Ultra 60 to arrive. Somehow I am happy although I spent money for something that could have been achieved in a more economic way. But what did I buy? An about five year old machine - you wouldn't dare doing this with a PeeCee. Again, probably to good quality...

To Sun: find out what people are willing to pay for at regular intervals and get it sold to them. Lower the quality if needed. It would be too sad if you went bankrupt, because the quality is too good. Think of HP's printers. They were once too good (my father still has an about 8 years old HP5MP).

Just my $0.02...

all key metrics at a glance

When one wants to know what is happening on a system, most people start top. Solaris users have prstat, which has some advantages, but misses some values listed by top. So what do you do? I tell you what I have been doing all the time: start all of the following in seperate xterms: vmstat, iostat, prstat.

This is very informative, but presents values I don't really care about and misses some I would like to have. So I wrote sysstat that presents all metrics I want at a single glance. Look here:



If you like this output, you can get the sources of the software here.

xjobs & DTrace

After having integrated all the features that I thought to be most important for xjobs, I considered giving it some tuning. Some things came right to my mind (object caching, more efficient token passing, and so on). But then I wanted to see, what DTrace could tell me.

Unfortunately, I forgot to log my tests directly, so I reran the tests against the previous release for this blog. The old release misses the object cache and token tunings that I included before considering DTrace, but the numbers should be roughly the same, as the D script only looks at syscall times.

I used the following, fairly simple script that sums up the times spent in syscalls:

#!/usr/sbin/dtrace -s
syscall:::entry
/ execname == $$1 /
{
self->ts = timestamp;
}
syscall:::return
/ self->ts /
{
@[probefunc] = sum(timestamp - self->ts);
self->ts = 0;
}

Then I did the following:

maierkom@aquila:~$ systime xjobs
dtrace: script '/home/maierkom/bin/sun4/systime' matched 460 probes
^Z
Angehalten
maierkom@aquila:~$ bg
[1] systime xjobs &
maierkom@aquila:~$ /usr/bin/ls -1 /usr/sbin | src/xjobs-20060412/xjobs -v2 echo >> & /tmp/12
maierkom@aquila:~$ fg
systime xjobs
^C

issetugid 8500
fstat64 11800
setcontext 14700
sysconfig 18400
ioctl 21400
read 30300
getcwd 43400
brk 62600
stat 95700
resolvepath 110900
memcntl 158400
schedctl 2193000
lwp_self 2504800
getrlimit 3253400
fstat 4133900
getpid 4888400
lwp_sigmask 4929800
setpgrp 5193200
unlink 6338700
fcntl 7303600
lstat64 10207400
mmap 12543500
munmap 12776900
write 15068100
open 19000800
exece 432717100
access 787848500
fork1 1721425000
close 87198287700
waitsys 103746109400

OK, waitsys is big. I guess that's OK. But WTF is the matter with close? xjobs has to close all open filedescriptors that are not needed by the jobs forked. So I changed the filedescriptor handling, and set all
descriptors to be closed on exec. Additionally, access uses a lot of
time. When the utility to execute is given on the command line of xjobs,
one only needs to search the PATH once. OK, then let's do this caching.

After adding these changes and some the ones mentioned above concerning
the user code, I get the following results:

dtrace: script '/home/maierkom/bin/sun4/systime' matched 460 probes
^Z
Angehalten
maierkom@aquila:~$ bg
[1] systime xjobs &
maierkom@aquila:~$ /usr/bin/ls -1 /usr/sbin | src/xjobs-20060413/xjobs -v2 echo > & /tmp/13
maierkom@aquila:~$ fg
systime xjobs
^C

getrlimit 7100
issetugid 8700
fstat64 13900
setcontext 17400
sysconfig 19800
ioctl 23700
read 36500
getcwd 57200
brk 73300
resolvepath 106500
stat 110700
memcntl 185800
schedctl 1784700
lwp_self 2381700
fstat 2850800
setpgrp 3608200
lwp_sigmask 4380400
getpid 5376800
unlink 5641200
access 5886200
munmap 6634300
mmap 7548900
lstat64 7577200
fcntl 12572000
write 16043800
open 17121400
close 19644800
waitsys 32769800
fork1 119443300
exece 368008400

That looks much better. Now, let's take a look at the overall numbers:

maierkom@aquila:~$ /usr/bin/ls -1 /usr/sbin | timex src/xjobs-20060412/xjobs -v2 echo > /tmp/12

real 6.86
user 17.18
sys 4.81

maierkom@aquila:~$ /usr/bin/ls -1 /usr/sbin | timex src/xjobs-20060413/xjobs -v2 echo > /tmp/13

real 0.68
user 0.13
sys 0.48

WOW! DTrace you made my day! Factor ten for real and system time. xjobs is now really fast.

Car stereo requirements

Having a new car stereo system that plays MP3 CDs is really a Nice Thing(TM). But what's odd about it, are the new requirements that come up.

It started, when I began encoding some of my favorite CDs, so I could listen to them when driving. First I did everything by hand using cdda2wav and lame. Thinking, I would be probably doing it again and again, I wrote a shell script to make it a little bit more convenient and automate the process of looking up the song title, interpret and album name and encode it into the filename and directory.

Fine, but then I thought: my machine has two processors, I could start two lame processes at once and get the songs encoded to mp3 in half the time. Looking at the Solaris board utilities, I couldn't find anything apropriate. Here was my odd requirement. So I wrote xjobs and published it under GPLv2. Get it here.

Now my shell scripts pipes its output directly to xjobs, and xjobs takes care that there are always two lame processes running. Convenient and fast.