There is a change happening around here. I posted this on my business blog a couple of days ago, and I thought I would share some of the changes with you dear reader.
We have renamed the business, or to be closer to the truth, we have given it a real name. No longer will we be operating just under my name, instead, we will be using the name Collaborynth. Yes it's a made up name, but I think it works to show how the main aim of our business is to help our clients through the sometimes very confusing maze of collaboration options.
We're in the process of developing our own hosted collaboration products as well as continuing the development of our more general hosting stuff.
This means that jamespurser.com.au has now returned to it's normal function as my personal blog.
The future looks exciting, let's go there!
Dear lazyweb number 3.
So far, I've asked:
high latency net simulations - great answers.
python friendly back-end accessible search engines - many answers, none that fit the bill. So I wrote my own :).
Today, I shall ask - is there a python-accessible persistent b+tree(or hashtable, or ...) module. Key considerations:
- scaling: millions of nodes are needed with low latency access to a nodes value and to determine a nodes absence
- indices are write once. (e.g. a group of indices are queried, and data is expired altered by some generational tactic such as combining existing indices into one larger one and discarding the old ones)
- reading and writing is suitable for sharply memory constrained environments. ideally only a few 100KB of memory are needed to write a 100K node index, or to read those same 100K nodes back out of a million node index. temporary files during writing are fine.
- backend access must either be via a well defined minimal api (e.g. 'needs read, readv, write, rename, delete') or customisable in python
- easy installation - if C libraries etc are needed they must be already pervasively available to windows users and Ubuntu/Suse/Redhat/*BSD systems
- ideally sorted iteration is available as well, though it could be layered on top
- fast, did I mention fast?
- stable formats - these indices may last for years unaltered after being written, so any libraries involved need to ensure that the format will be accessible for a long time. (e.g. python's dump/marshal facility fails)
sqlite, bdb already fail at this requirements list.
snakesql, gadfly, buzhug and rbtree fail too.
OpenOffice Writer.
No, I kid you not. It quells all of the riotous JavaScript etc, gives you a plain layout, & best of all you & your friend the Delete key can have fun culling useless rubbish, making font sizes meaningful, readable, & you can save the most successful results as Writer (ODT) files for later.
Oddly enough, I’m starting to look forward to my email — even wading through a hundred or so spam a day — thanks to SMS (/ME WAVEs across 12 megameters).
I've blogged about this before but I'm going to do it again.
Why, oh why do people insist on building their own TZ databases and ignoring the Operating Systems TZ dataset? Is this just a really stupid case of NIH? I've hit this in drupal (specifically the Events Module), Plone (inherited from zope), and I've been told about java having it's own tz set as well.
I might even do a t-shirt, this has got me that annoyed!
For those people who like living on the bleeding edge of development, you can now update your system to include the latest builds coming out of the OpenSolaris sausage factory. Alan has been posting updates on indiana-discuss when they are available. Those who have been used to SXCE should be pretty comfortable updating, and both IPS, ZFS and beadm have made this a breeze.
The current update is build 91, and here’s the steps to clone your ZFS file-system, and download the latest packages -
# Refresh the package catalog gman@rampage:~$ pfexec pkg refresh # Install an updated version of SUNWipkg (which avoids a few bugs) gman@rampage:~$ pfexec pkg install pkg:/SUNWipkg@0.5.11,0.5.11-0.86 # Refresh the package catalog again (to get fixed up content) gman@rampage:~$ pfexec pkg refresh # Image Update for the win! (sit back and watch the progress) gman@rampage:~$ pfexec pkg image-update # Now you need to ensure to active your new boot environment (temporary step) gman@rampage:~$ pfexec mount -F zfs rpool/ROOT/opensolaris-1 /mnt gman@rampage:~$ pfexec /mnt/boot/solaris/bin/update_grub -R /mnt # Now reboot and enjoy!In the works is build 92, that should be available in a week or so - including GNOME 2.22. Hopefully by the time 2008.11 rocks around, we’ll have 2.24 included!
For best system security you want to apply kernel security patches ASAP. For an attacker gaining root access to a machine is often a two step process, the first step is to exploit a weakness in a non-root daemon or take over a user account, the second step is to compromise the kernel to gain root access. So even if a machine is not used for providing public shell access or any other task which involves giving user access to potential hostile people, having the kernel be secure is an important part of system security.
One thing that gets little consideration is the overall effect of applying security updates on overall uptime. Over the last year there have been 14 security related updates (I count a silent data loss along with security issues) to the main Debian Etch kernel package. Of those 14, it seems that if you don’t use DCCP, NAT for CIFS or SNMP, IA64, the dialout group, then you will only need to patch for issues 2, 3 (for SMP machines), 4, 5, 7 (sound drivers get loaded on all machines by default), 9, 10, 11, 12, 13, and 14.
This means 11 reboots a year for SMP machines and 10 a year for uni-processor machines. If a reboot takes three minutes (which is an optimistic assumption) then that would be 30 or 33 minutes of downtime a year due to kernel upgrades. In terms of uptime we talk about the number of “nines”, where the ideal is generally regarded as “five nines” or 99.999% uptime. 33 minutes of downtime a year for kernel upgrades means that you get 99.993% uptime (which is “four nines”). If a reboot takes six minutes (which is not uncommon for servers) then it’s 99.987% uptime (”thee nines”).
While it doesn’t seem likely to affect the number of “nines” you get, not using SMP has the potential to avoid future security issues. So it seems that when using a Xen (or other virtualisation technology) assigning only one CPU to the DomUs that don’t need any more could improve uptime for them.
For Xen Dom0’s which don’t have local users or daemons, don’t use DCCP, NAT for CIFS or SNMP, wireless, CIFS, JFFS2, PPPoE, bluetooth, H.323 or SCTP connection tracking, then only issue 11 applies. However for “five nines” you need to have 5 minutes of downtime a year or less. It seems unlikely that a busy Xen server can be rebooted in 5 minutes as all the DomUs need to have their memory saved to disk (writing out the data to disk and reading it back in after a reboot will probably take at least a couple of minutes) or they need to be shutdown and booted again after the Dom0 is rebooted (which is a good procedure if the security fix affects both Dom0 and DomU use), and such shutdowns and reboots of DomU’s will take a lot of time.
Based on the past year, it seems that a system running as a basic server might get “four nines” if configured for a fast boot (it’s surprising that no-one seems to be talking about recent improvements to the speed of booting as high-availability features) and if the boot is slower then you are looking at “three nines”. For a Xen server unless you have some sort of cluster it seems that “five nines” is unattainable due to reboot times if there is one issue a year, but “four nines” should be easy to get.
Now while the 14 issues over the last year for the kernel seems likely to be a pattern that will continue, the one issue which affects Xen may not be representative (small numbers are not statistically significant). I feel confident in predicting a need for between 5 and 20 kernel updates next year due to kernel security issues, but I would not be prepared to bet on whether the number of issues affecting Xen will be 0, 1, or 4 (it seems unlikely that there would be 5 or more).
I will write a future post about some strategies for mitigating these issues.
Here is my summary of the Debian kernel linux-image-2.6.18-6-686 (Etch kernel) security updates according to it’s changelog, they are not in chronological order, it’s the order of the changelog file:
Dear Google,
I am not sure how to quantify the exact amount of evil involved in unsolicited bulk email (I guess I could argue that it's even commercial email, because you are a company promoting a product, even if it is a coding competition), but let me assure you, the amount of evil is exactly the same in 2008 as it was either time in 2005, and for that matter, in 2003.
So, knock it off already.
Today I received a Dell PowerEDGE T105 for use by a client. My client had some servers for development and testing hosted in a server room at significant expense. They also needed an offsite backup of critical data. So I suggested that they buy a cheap server-class machine, put it on a fast ADSL connection at their home, and use Xen DomU’s on that for development, testing, and backup. My client liked the concept but didn’t like the idea of having a server in his home.
So I’m going to run the server from my home. I selected a Dell PowerEDGE tower system because it’s the cheapest server-class machine that can be purchased new. I have a slight preference for HP hardware but HP gear is probably more expensive and they are not a customer focussed company (they couldn’t even give me a price).
So exactly a week after placing my order I received my shiny new Dell system, and it didn’t work. I booted a CentOS CD and ran “memtest” and the machine performed a hard reset. When it booted again it informed me that the event log had a message, and the message was “Uncorrectable ECC Error” with extra data of “DIMM 2,2“. While it sucks quite badly to receive a new machine that doesn’t work, that’s about the best result you can hope for when you have a serious error on the motherboard or the RAM. A machine without ECC memory would probably just randomly crash every so often and maybe lose data (see my previous post on the relative merits of ECC RAM and RAID [1]).
So I phoned up Dell (it’s a pity that their “Packing Slip” was a low quality photocopy which didn’t allow me to read their phone number and that the shipping box also didn’t include the number so I had to look them up on the web) to get technical support. Once we had established that by removing the DIMMs and reinserting them I had proved that there was a hardware fault they agreed to send out a technician with a replacement motherboard and RAM.
I’m now glad that I bought the RAM from Dell. Dell’s business model seems to revolve around low base prices for hardware and then extremely high prices for extras, for example Dell sells 1TB SATA disks for $818.40 while MSY [1] has them for $215 or $233 depending on brand.
When I get the machine working I will buy two 1TB disks from MSY (or another company with similar prices). Not only does that save some money but it also means that I can get different brands of disk. I believe that having different brands of hard disk in a RAID-1 array will decrease the probability of having them both fail at the same time.
One interesting thing about the PowerEdge T105 is that Dell will only sell two disks for it, but it has four SATA connectors on the motherboard, one is used for a SATA DVD player so it would be easy to support three disks. Four disks could be installed if a PCIe SATA controller was used (one in the space for a FDD and another in the space for a second CD/DVD drive), and if you were prepared to go without a CD/DVD drive then five internal disks could probably work. But without any special hardware the space for a second CD/DVD drive is just begging to be used for a third hard disk, most servers only use the primary CD/DVD drive for installing the OS and I expect that the demand for two CD/DVD drives in a server is extremely low. Personally I would prefer it if servers shipped with external USB DVD drives for installing the OS. Then when I install a server room I could leave one or two drives there in case a system recovery is needed and use the rest for desktop machines.
One thing that they seem to have messed up is the lack of a filter for the air intake fan at the front of the case. The Opteron CPU has a fan that’s about 11cm wide which sucks in air from the front of the machine, in front of that fan there is a 4cm gap which would nicely fit a little sponge filter. Either they messed up the design or somehow my air filter got lost in transit.
Incidentally if you want to buy from Dell in Australia then you need to configure your OS to not use ECN (Explicit Congestion Notification [2] as the Dell web servers used for sales rejects all connections from hosts with ECN enabled. It’s interesting that the web servers used for providing promotional information work fine with ECN and it’s only if you want to buy that it bites you.
But in spite of these issues, I am still happy with Dell overall. Their machine was DOA, that happens sometimes and the next day service is good (NB I didn’t pay extra for better service). I expect that they will fix it tomorrow and I’ll buy more of their gear in future.
Update: I forgot to mention that Dell shipped the machine with two power cables. While two power cables is a good thing for the more expensive servers that have redundant PSUs, for a machine with only one PSU it’s a bit of a waste. For some time I’ve been collecting computer power cables faster than I’ve been using them (due to machines dying and due to clients who want machines but already have spare power cables). So I’ve started giving them away at meetings of my local LUG. At the last meeting I gave away a bag of power cables and I plan to keep taking cables to the meetings until people stop accepting them.
We recently joined The Club of Mountain View, because it just happens to be around the corner, and we both want to try and get back into some semblance of shape.
The Club has pretty reasonable facilities. It has a large cardio room, a large weights room, a group exercise room where they do a number of classes, and a spinning room. It also has three racquetball/handball courts, as well as an indoor basketball court.
Pretty much all of the facilities are included in the monthly membership, so Sarah went and booked a racquetball court this evening, and we had a bit of a whack.
We didn't have a clue how to play, so it was just basically half an hour of belting the ball around, and I have to say, I really liked it.
I haven't played squash since high school, but didn't enjoy it, because I was more used to tennis, and constantly misjudged where the racquet head was, hence missing the ball. I never really liked the characteristic of a squash ball either.
I really like tennis, but I have habit of getting carried away and belting the tripe out of the tennis ball, either sending it completely out of the court (over the fence) and far away, or at the very best well and truly past the back line of the court and into the fence.
Racquetball seems perfect for me. I can belt the living daylights out of the ball. It can't get lost. The racquet is short and reasonably large. There's a very satisfying thwack sound to boot. In short: I like it.
I wonder what the people in Civic thought of me today watching me walk back to uni from target carrying four 100 litre bags of polystyrene beans on a very windy day. Mikal was somewhat annoyed with me that I did not give him a heads up so he could come and laugh at me doing the walk here. He did however help me fill the bag and take the photo, activities that were in his words either annoying or strangely satisfying at times.
The bean bag is a Jumbo sized denim bag purchased from Blob Beanbags, it seems well made, the zip can be pushed up inside a little sleeve so kids can not easily open the bag, also I purchased a liner with it so it is easy to take the beans out and clean the bag. Rather comfortable really, maybe I should actually get one for my office.
Today, I completed my migration of my personal machine to one that runs OS X. For those not following Twitter, I picked up a MacBook Air last week, and have slowly been moving my stuff off from the Dell. The Dell can now serve as a full development machine, and I can start running “unstable” Linuxes on it now (”unstable” like Rawhide).
But I digress. This is about how I moved Thunderbird and Firefox over to my new box.
Thunderbird:
Copy ~/.thunderbird over, and place it in ~/Library/Thunderbird on OS X. Only problem I found was with the Lightning plugin, which managed to grab itself an update, and all was dandy.
Firefox:
Copy ~/.mozilla/firefox over, and place it in ~/Library/Application Support/Firefox. All the plugins I had, just ran fine.
Only snag? I couldn’t find a copy of Firefox 2 online. Good thing I had a copy on another Mac… Why did I need Firefox 2? Google Browser Sync. Though I suspect that in the very near future, I’ll move over to Mozilla Weave, and get all my systems up to speed with Firefox 3.
Next up, lets see how long I run OS X on the Air… or do I replace it with Linux if it annoys me significantly enough?