Sunday 14 August 2005

Goodbye apt-get

So it's finally time to leave apt-get, at least on my desktop machine, and go down the path of bsd and ports. I gave up on fink for my software packaging needs and switched to OpenDarwin instead. The main reason for the choice was that OpenDarwn has a larger and more updated package list, so it requires less manual downloading and package dependency resolving. The small price of learning a slightly different controlling program (port vs apt-get) was more than outweighed by the ease of typing 'sudo port install x' and just walking off and letting it do all the work. The lack of a central repository (OpenDarwin downloads the tarballs directly from the original site, not from a central repository) makes it difficult to use a repository cache, but I guess that's irrelevant since I am the only one in the office that uses OpenDarwin anyway.

Also took care of the clunky OpenOffice issue by installing NeoOfficeJ. My colleages have not had good experiences with the older versions but the latest version does not seem too bad. Fires up relatively quickly and seems stable, but I must admit that I have not used it too extensively. Will know more later.

One minor annoyance with OSX are the emacs keybindings, or the lack of two of them. Almost all the commonly used ones are here, even the obscure ones. However, there are two that I use very frequently that are not found in the text editor programs - Meta-f and Meta-b. These go forward and back one word, respectively. I have ctrl-f and ctrl-b, which move one letter at a time, but I have gotten very used to the one word movements that I seriously feel the lack without them. In terminal, all the Meta-x keys are done via option-x, and in text editors, some of this is there. option-backspace is backspace one whole char, the same as Meta-backspace in emacs. However, the option-f and option-b keys produce some strange characters, instead of replicating their Meta-x behaviours. Ah well. At least option-arrow key still moves on word at a time, but I'm too lazy to move my entire hand the few cm required to hit them :)

One other note - never use transparencies on anything you want a fast response time from. I had a slight transparency going on my terminals and was wondering why text-mode emacs was moving dog-slow. Once the transparency was shut off, it was back the good old speeds you have come to expect and love from text terminals. Even with the high-end graphics cards that all these machines come with, transparency is still very expensive on the CPU

Thursday 11 August 2005

Interviewing Thoughts

Just some thoughts to trends/things that some people say when I am doing interviews. Some I strongly disagree with, others ... I am curious what other peoples thoughts and experiences are with them.

  1. You can't keep up with the young 'uns
  2. You have to change jobs every year, or you will never advance
  3. If you learn on your own, without 'real' experience, it is worthless
  4. To really become an expert, you must focus on one specific field and ignore the others
  5. Details are not important. The compiler will catch it
1. You can't keep up with the young 'uns

This was first related to me by an applicant in her late twenties, with only a little working experience but lots of degrees. In the interview, she said that her goal was to move into project management because, "as you get older, your thought processes slow and you cannot code as fast as those young kids."

The sad part was, all the senior applicants I interviewed that day shared exactly the same viewpoint. All wanted to move out of coding and into project management. They wanted to get into management not because they enjoyed it or because it had to be done, but because they felt it was their only escape from obsolescence.

This is silly. This ignores all the experience and knowledge that older, more experienced coders have gained through painful experience. This assumes that a person's mental faculties begin deteriorating at such a young age (not true!) and that quantity of code equates to productivity. I know I certainly produced a lot more volume of code in my early programming years, but experience has taught me to produce more effective code. It takes fewer lines, has far fewer defects and is far easier to maintain later. If anything, the years have taught me that quality is far more critical than quantity. And woe to anyone that even thinks I'm close to being an old fart ...

2. You have to change jobs at least once a year, or you will never advance

This particular one really annoys me. It feels like the get-rich-quick days of the dotcom boom again. Demand for good technical people is high in China, especially so for those with FOSS skills. Because of this, opportunities abound and everyone is jumping from one job to another to get more money.

I suppose it does give a person more experience and probably a higher salary in the short run. But do you truly gain any skills when you jump around this quickly? By moving so much, you miss the big picture. Most major projects have lifecycles longer than a year.

An eye opening experience I find for most people is to be still around during the maintenance phase of a project. It is then that you realize that those fancy coding hacks you did, or that /* This will never happen */ section of code, will come back and bite you. You get to see the long-term price of taking shorts cuts and skimping on quality. It gives you an appreciation for doing things _right_, especially if you have been through two project lifecycles, where you have seen on where it went right and one where it went wrong.

3. If you learn on your own, without 'real' experience, it is worthless

One thing we always make a point to ask for is whether a person tinkers with technologies on their own, outside of what is required for work or class. More than one applicant has justified not doing so by saying, "if it was not done for work, it is not real world experience. It is worthless."

Is it? The best technical people I know are always voraciously learning. Some deliberately learn technologies far removed from their daily work, to stretch their brain to new ways of thinking. The diverse knowledge base they draw on seems to serve them well.

Yes, "real world" experience is valuable. You never truly appreciate the value of caches until you have been slashdotted, or when you max out all the CPUs on your server farm serving dynamic content. It is hard to really appreciate the value of distributed databases until you have so many transactions that the fastest hard disks in existence cannot write the transactions to disk as fast as they pour in.

However, they are not everything. There are many people with "real world" experience who still have not really learned the best practices and techniques. Unless you are really lucky, you cannot learn this unless you are constantly looking for it. The chances of you working with the best in your field are extremely limited, unless you're lucky enough to be working for someplace like google.

4. To really become an expert, you must focus on one specific field and ignore the others

Partially related to #3. I meet a lot of people with only Java experience. They know every single Java library, framework and technique there is. But nothing else. They have never touched another language, framework or system. They sneer at all others as inferior, but have never tried them.

Depth is valuable, but so is breadth. I have watched Java experts sink their systems as they tried to employ lower performance Java web caches, when squid outperforms the faster available system by an order of magnitude. I have seen people try to squeeze J2ME into embedded systems when an ncurses interface is the only thing that would have given a halfway decent response time.

As they say, when you only have a hammer, everything in the world looks like a nail. If you don't know anything else, you will always use the tools you know, even if they are not the most appropriate for the system. You miss the best of breed tools.

5. Details are not important. The compiler will catch it

Final peeve. Some applicants make careless and copious mistakes while answering their coding question. They explain this away as, "oh, the compiler will catch it when I run this on a computer."

The compiler is not your nanny. It is not your QA department. Even if you had a QA staff to back you up, the most qualified person to find errors in your code is you. There are insidious logic errors that no compiler will catch and even good code reviews and QA people may miss. The best and easiest time to catch errors is right after you have written the code, when all the variables, requirements and interfaces are fresh in your head. Spend the time to review your code and catch everything you can.

Attention to detail is really important. If you lack it, it comes back to haunt you. If you have it, your users may not be able to explain why, but they will like your work a lot more. It is like the Apple products which I rave about - the attention to detail and usability have given their products an incredible competitive advantage. Even with commodity, even sometimes inferior hardware, they can still command per unit profit margins that are the envy of other PC manufacturers.

Wednesday 10 August 2005

Powerbook day 5

(Originally posted 20050810)

So it has been 5 days with this nice little bit of hardware and I am liking it greatly. It occurs to me though, that my blogs do not reflect this. I am more likely to point out its shortcomings and difficulties than all the things I love about it. So I thought I'd spend this blog writing about it. I'll try to gloss over the stuff that most other people rave about and focus on what is special about it to me as a full-time developer.

Ok, the usual stuff people rave about:
  1. Beautiful, well thought out design
  2. It Just Works!
  3. Consistent UI
  4. Sleeps and recovers in less than 4 seconds
  5. High quality hardware

The other stuff that makes me use it as my primary workstation, despite its currently lower performance:

1. Power users welcome

While Apple may have been one of the first to properly implement a GUI and has one of the slickest interfaces around, the mouse is almost totally optional! People like me who like to work primarily through the keyboard interface can actually navigate around purely with the keyboard, using the mouse very rarely. For me, this is a significant speed increase as keyboard shortcuts are far faster than using the mouse. Additionally, because the key bindings are very similar to emacs (my main development environment), everything is comfortable and familiar.

One other cool thing about emacs key bindings is the very limited hand movement. You can reach just about all the keys without moving your hand more than a centimeter from standard rest position. You do not even have to move your hands the ~10 cm required to hit the arrow keys (3-4 cm on a laptop), thereby not disrupting your typing speed. Yes, I am anal about these things. But it does make a serious productivity difference to me.

2. *nix base

While the flashy GUI protects new users from having to deal with the guts and internals of the OS, you can still pop open the hood as necessary. Fire up the terminal program and you can see which processes are the memory/cpu hogs, what crazy programs are running riot, etc. You can fire up your trusty shell scripts to perform routine tasks, run all the *nix applications which we know and love, etc. Get down to the OS level and figure out exactly why that peripheral is not working, if you can.

Granted, this is no different from any *nix environment, but it is a *nix environment with few of the warts. In many ways, it feels like a *nix desktop done _right_.

3. It Just Works!

Ok, so I ran out of other points. But this is a point I cannot emphasize enough. Almost everything works without problems, in exactly the way you would expect it to in a perfect world. One click/keypress often produces the results you are looking for. Things run effortlessly. You waste so little fighting the GUI/OS and so much time just being productive. All the little annoyances that bugged you, be they in the Linux, BSD or Windows world, seem to be magically gone here. No need to spend days or weeks tweaking/configuring it.

Ultimately, the powerbook and osx is what I consider the ultimate combination - the simplicity that Windows promised, the power and security of *nix, all combined in one very well designed package. It is a testament to what attention to detail and usability can do. The hardware, while of high quality, is commodity. Any other company can manufacture materials of comparable quality. However, no one can put it together, hardware and software, the way Apple has done with their computers or iPod.

Tuesday 9 August 2005

Powerbook Day 4

(Originally posted 20050809)

It is closing in to my 96th hour with the powerbook and it has been a fun experience so far. The first 48 hours was when I had to do the bulk of my learning - getting used to the "Mac way", installing applications and waiting for stuff to download. After that, it has been mostly smooth sailing.

The only issues I've encountered so far is the relative slowness of the ppc chip and OpenOffice.org clunkiness. The OpenOffice.org clunkiness is just because the standard installation uses X11 and is not well integrated with the rest of the desktop at all. Things like one click in email to open a document just do not work. It is a cumbersome multi step process. I'm currently downloading NeoOfficeJ (based on OOo) but the slow download speeds in China really takes a while. Still, not a big deal.

The more serious issue is the performance of the system for number crunching. After I had set up my development environment, I was surprised to find that CPU intensive unit tests in my projects took twice as long as they did on my old laptop. Functional tests (fire up all the servers, do black box testing using simulated browsers) took 4-5 times longer. This makes the powerbook substandard as a development station.

A simple test of this is to run code like this:

a = 1
for i in range(1,1000000):
    a = (a + i)/i

On my laptop and workstation (1.5Ghz AMD Athlon, underpowered by today's standards), this takes 0.8 seconds to run. On the default python program on OS X, this takes 1.3 seconds! Using the version of python obtained from fink, this drops to 1.1 seconds, but still far slower than the Athlon systems. This is all using python 2.3.5.

Have not looked into detail for the reasons behind this, but I suspect that it is in part because OS X is tuned to be a desktop, not a server. Responsiveness of the GUI is paramount.

At any rate, this is disappointing, but not a deal breaker for me. While it means I have to run the intensive functional tests on a separate, development machine, I can still use the powerbook for most of my daily work. I always have a workstation somewhere within ssh distance of me anyway, and sadly enough, most of my daily work does not involve CPU intensive activity anymore. Things such as responsive email, web browsing and document editing are more important.

Sunday 7 August 2005

It's Powerbook Time!

(Originally posted 20050807)

I finally gave in to the dark side this saturday. Could not resist it any more. I finally broke down and ... got me a Mac. A powerbook to be exact. Retired my old x86 laptop and went down the path of OS X and the PPC (temporarily). It has been an interesting 48 hours as I tinker with my machine, getting it ready for duty as my development machine on Monday.

There are quite a few reasons for getting the powerbook. They range from the superior quality of the hardware itself, to the "It Just Works" experience that my colleagues were experiencing with their Apple laptops, to sheer lust for all the nifty features that came fully functional on a standard OS X install. All these great features and I would not be giving up the *nix environment that I have grown so comfortable on after all these years on Linux desktops.

My greatest challenges in the last 24 hours have been adjusting myself to the "Mac" way of doing things and configuring the laptop for development. I have until Monday, as there will be certainly no time for me to do any fiddling with my development system once the workday begins. 

Navigating around an OS X system is relatively easy using a mouse, but for a variety of reasons (speed, looming RSI and the unwillingness to carry an extra mouse peripheral), I prefer to use the keyboard for most of my functions. The average linux (KDE/Gnome) desktop allows you to do this rather well, but with a rather different set of key bindings. The biggest thing to get used to is the Alt/Command difference. Coming from an emacs background is both a blessing and a curse. Most of the OS X apps support emacs bindings with the control key, especially cursor movement (^f, ^b, ^p, ^n, etc). Unfortunately, they do not support the Alt keys, as the equivalent of the Alt on OS X is the Command key and keys like Cmd-b and Cmd-f are already used, being Bold and Find, respectively. These are keys I use a lot in navigation (move forward one word, move back one word), so I find myself having to use a lot more keystrokes than I used to. I'm sure that I'll get used to this in time though.

After spending a few hours learning the system and adjusting it to my preferences, I realized that I would have to repartition the system. I wanted to reserve HD space to put a linux ppc distribution on it later, just in case something came along that OS X could not do. Unfortunately, repartitioning a HFS filesystem is a destructive process at present, so I was forced to wipe the system and reinstall. Fortunately, this process is relatively painless. You pop in the disk, select a few options (less than even an ubuntu install) and you can almost leave it alone till it is finish about 30 minutes later. It was the easiest system install I have ever done in my life.

After that, the next challenge was simply setting up a development environment on OS X. OS X is meant to be a consumer desktop, not a development workstation, despite its BSD/*nix roots. It is quite a bit more effort to install the necessary applications I need to develop on. Fortunately, I could draw upon the experience of several of my colleages, particularly Adam. Following his advice, I went with fink to provide most of the resources I needed. I suppose I could have tried OpenDarwin instead, but fink gave me the familiar apt-get tools from Debian.

After downloading the necessary .dmg file from the site, installation and usage of most applications was relatively easy. The major applications such as python2.3 and postgresql 7.4.x was installed with "apt-get install x". The smaller, lesser known applications were more of a pain though. Things such as psycopg, twisted and clearsilver required source installs as they were not in the repository. I even played with DarwinPorts for a while as it had most of those packages. Unfortunately, the DarwinPorts versions were newer than what I needed, so it was back to source installs.

Finally, a little under 48 hours after I got the laptop, my development station was set up, all systems were running and I was ready to develop. It has been an interesting experience so far and the laptop has been a joy to work with.