cd/dvd drives not working under linux (2.6.18)

At some point over the last few months my dvd drives stopped working on Linux Centos 5.0 (kernel 2.6.18). Having spent half the day googling to find a fix, looking at udev, hal, fstab, bios settings, ata/piix/ide modules, jumper settings on the drive... I found a fix to the problem. Having tried to execute

modprobe ide-cd

I got an error message about not understanding what the dma option is. Which led me onto editing /etc/modprobe.conf and commenting out the the line:

options ide-cd dma=1

This fixed the problem.

The problem with too many software layers

This article summaries the problems with too many software layers: In Relation To... From component model failure to layer fetishism

A kdb tick/q demo

1) Install q and kdb tick, including k4.lic file

2) In install dir

mkdir data
mkdir logs

create file tick/schema.q containing:
test:([]time:`time$();sym:`symbol$();size:`float$();price:`float$())

3) create a start.sh script for kdb
#!/bin/bash
export QHOME=/usr/local/bin/q
export PATH=$PATH:/usr/local/bin/q

$QHOME/l32/q tick.k schema $QHOME/data -o 0 -p 30000 >> $QHOME/logs/tp.log 2>&1 &
$QHOME/l32/q tick/r.k :30000 -p 30001 >> $QHOME/logs/rtdb.log 2>&1 &
$QHOME/l32/q $QHOME/data/schema -p 30002 >> $QHOME/logs/hdb.txt 2>&1 &

4) run start script

5) In a seperate q session insert some data
h:hopen `:localhost:30000
h ".u.upd[`test](`ibm; 10000.0; 98.0)"

Subversion auto-props default

Its a shame that subversion's config for auto-props dont have some sensible defaults or allow me to set them on a per repository basis. Here are some that I use in my ${home}/.subversion/config.

[miscellany]
enable-auto-props = yes

[auto-props]
*.q = svn:eol-style=LF;svn:mime-type=text/x-q
*.sh = svn:eol-style=LF;svn:executable=ON;svn:mime-type=text/x-sh
*.java = svn:keywords=Id;svn:mime-type=text/x-java-source
*.png = svn:mime-type=image/png
*.jpg = svn:mime-type=image/jpeg

An interesting tool for database synchronization

SymmetricDS

Boolean parameters can be evil

Alex Ruiz's Weblog

Maven Help

mvn --help

Lists the profiles which are currently active for the build:
mvn help:active-profiles

Displays the effective POM for the current build, with the active profiles factored in:
help:effective-pom

Prints out the calculated settings for the project, given any profile enhancement and the inheritance of the global settings into the user-level settings:
help:effective-settings

To get help on a plugin:
mvn help:describe -Dplugin=xxx -Dfull
mvn help:describe -DgroupId=xxx -DartifactId=xxx -Dfull=true

Apache Archiva Maven Repository Server

Seems like a pretty decent Maven Repository server. I've set it up to manage my own Maven repositories, including my own "external-public" repository which proxies access to a number of the big external Maven repositories.

Its easy to install and all configuration is done through the web interface. As well as configuring repositories and connect repositories so one can proxy another, it adds user management to secure access via various roles. The docs on the home site aren't great but google and you will find other good config howtos.

For more detail on functionality see http://www.devzuz.org/blogs/oching/2007/11/05/1194233400000.html

Creating a default Maven project

mvn archetype:generate

This will ask multi-choice questions regarding creating a new project/module.

Or you can directly specify all options as:

mvn archetype:create -DgroupId=com.parwy.testproj -DartifactId=testproj -DarchetypeArtifactId=maven-archetype-quickstart

Archetypes include:
maven-archetype-archetype
maven-archetype-bundles
maven-archetype-j2ee-simple
maven-archetype-marmalade-mojo
maven-archetype-mojo
maven-archetype-plugin-site
maven-archetype-plugin
maven-archetype-portlet
maven-archetype-profiles
maven-archetype-quickstart
maven-archetype-simple
maven-archetype-site-simple
maven-archetype-site
maven-archetype-webapp

The default is maven-archetype-quickstart

Web frameworks

I not had to do any web development for years but thought I'd archive this link away in case I need to look at them again. Hopefully, by the time I ever come across the need to do web development, the tools and frameworks will have evolved much further.

What belongs on the domain model and what belongs on service classes

This article hit home, I have often tried to balance what belongs on the domain model and what belongs on "even more domain-specific" service classes. Poor old Gavin took some flak though... I totally agree with his follow up to the criticism.

Bulk search and replace using sed

This little snippet replaces occurrences of oldString with newString in all files named *.txt in the current directory and below, keeping a backup of the original files.

for i in $(find . -name "*.txt"); do sed 's/oldString/newString/g' $i > $i-tmp; mv $i $i-backup; mv $i-tmp $i; done

Interesting product range for algo trading

Portware Strategy Server

Not used it, but could make Subversion branch management easier

Svnmerge.py

Something to look at for mapping javabean properties

Dozer

Java libraries for building network applications

Apache MINA
JGroups

Java library for measurements

JScience

Classic discussion on problems with fulltime architects

Ground Control to Architect Tom...

A nice firefox addon for searching in a page

Search Marker :: Firefox Add-ons

Skipping tests in maven

mvn -Dmaven.test.skip=true install

Deleting lots of files

If you are trying to delete a directory with lots of files in, rm will complain about to many files and refuse to delete. You can get round this with xargs:

find . -name "log-200709*.gz" -print0 | xargs -0 -t -r rm -f

Unit testing and documentation

When I left my last job, some of the big lessons I learnt in software development were:

1) Rely on dependency injection, from a design perspective, I don't mean using an IoC container like Spring. This meant avoiding singletons at all costs.

2) Compile a large system out of many small modules (jars) rather than compile the system into a monolithic jar. This was vital to prevent circular dependencies.

On my current project I would add a few more lessons:

1) Write unit tests, these may slow you down earlier, but they become a god-send three years down the line, when some of the original team members have left and been replaced by others. You want new members of the team to be able to get into and refactor any part of the system. At this point you must rely on unit tests, not on the memory of an old developer on the team telling them, "you'll have to watch out in that package, there is this issue you must remember or that issue you must ensure you don't break".

2) In the same vain as testing more documentation of packages, classes, methods and fields. You don't need much, as good naming of artifacts is often enough. But I probably could have documented a bit more.

3) Use more composition to change the behaviour of objects rather than inheritance. Using composition more occurs naturally through a dependency-injection approach, however it is still possible to overuse inheritance, which reduces reusability.

4) Be more strict about ensuring most classes are immutable, rather than making them mutable for performance reasons. I got some big gains in performance but I think I sacrificed more in terms of "gotchas" other developers face with mutable classes, e.g. storing an object in a set and then wondering why it is different later. Anyway some of the performance gains disappeared by the need to clone mutable objects.

5) Using libraries for units right from the beginning and pervasively. For units I mean something like Joda from dates/times/periods and classes for other measures such as monetary amounts.

Introduction to FX algo trading from a ECNs perspective

http://www.hedgeweek.com/specialreports/detail.jsp?content_id=176474

API design issues

ACM Queue - API: Design Matters: Should the authors of lousy APIs be held accountable for their crimes? Why changing APIs might become a criminal offense

Offline access to Google Reader through Google Gears

Google Gears (BETA)

Downloads Google Reader entries, lets you read them offline and resyncs read items when you connect. Lovely.

Google Calendar on your mobile phone

Official Google Blog: Calendar for mobile devices

Brendon in print - How the mobile phone biz lost the plot - The Register

Wow, Brendon gets printed in The Register: How the mobile phone biz lost the plot

Oops I had an open http proxy

I accidentally configured my apache web server to be an open proxy. I only discovered this when my own web browsing became very slow. First thing I did was reboot my server, that didnt help, so next I rebooted my netgear router, that didnt help either. I waited a day in case it was a problem at the ISP (even though their website was not showing any incidents). So the next day I ran:

netstat -a --inet --notrim

and found that a huge number of hosts were connecting to my http port. I followed this up by looking at my apache access logs and true enough, hundreds of hosts were connecting and asking for URLs that were external sites.

First things I did was access the admin site of my router and disable http/https. I then fixed my apache configuration and restarted apache, before renabling http/https on my router. After this I watched my apache access logs, as expected many hosts were still hitting my webserver, although at least now it was returning 404, rather than proxying the http request. So I installed ntop and got some statistics on which ip addresses were the biggest culprits. Armed with a list of ip addresses I blacklisted any ip traffic from them using iptables, i.e:

iptables -v -A INPUT -s 213.240.225.101 -j DROP
iptables -v -A INPUT -s 200.155.23.28 -j DROP
iptables -v -A INPUT -s 219.110.175.160 -j DROP
iptables -v -A INPUT -s 125.92.99.94 -j DROP
iptables -v -A INPUT -s 201.236.88.90 -j DROP
iptables -v -A INPUT -s 201.211.89.201 -j DROP
iptables -v -A INPUT -s 125.65.112.35 -j DROP

A place to buy posters

AllPosters.co.uk

GroupLayout

GroupLayout looks quite useful for creating forms. It came out of Netbean's Matisse GUI builder and is making its way into Java 6. Here is an introduction: Tomas Pavek's Blog: Getting to know GroupLayout, part 1.

Herman Miller Aeron Chair

Must buy a Herman Miller Aeron Chair at some point.

Google Code - Project Hosting

Google Code - Project Hosting is a Sourceforge like software management application that allows you to create a subversion repository for your code, a file repository for releases and a wiki for documentation.

Importing a new subversion project

Say you want to import a project called contacts into subversion repository called main. First we create a dummy project to import, this creates the structure (trunk/branches/tags) for the project. Second we import the real project into the trunk area.

C:\sekhonp\it\builds>ls
drwx------+ 4 sekhonp None 0 Jan 1 15:24 contacts

C:\sekhonp\it\builds>mkdir emptyproject
C:\sekhonp\it\builds>mkdir emptyproject\trunk
C:\sekhonp\it\builds>mkdir emptyproject\branches
C:\sekhonp\it\builds>mkdir emptyproject\tags

C:\sekhonp\it\builds>svn import emptyproject http://svn.parwy.net/repos/main/contacts
Adding emptyproject\trunk
Adding emptyproject\branches
Adding emptyproject\tags

C:\sekhonp\it\builds>ls -al
drwx------+ 4 sekhonp None 0 Jan 1 15:24 contacts
drwx------+ 5 sekhonp None 0 Jan 1 15:26 emptyproject

C:\sekhonp\it\builds>svn import contacts http://svn.parwy.net/repos/main/contacts/trunk

We now have a repository structure that looks like:

main ->
contacts ->
    trunk ->
        project files...
    branches ->
    tags ->

You can now clean up and checkout the trunk of the project:

C:\sekhonp\it\builds>rm -rf emptyproject
C:\sekhonp\it\builds>rm -rf contacts
C:\sekhonp\it\builds>svn co http://svn.parwy.net/repos/main/contacts/trunk contacts
C:\sekhonp\it\builds>ls -al
drwx------+ 5 sekhonp None 0 Jan 1 15:43 contacts