28 December 2007

cd/dvd drives not working under linux (2.6.18)

At some point over the last few months my dvd drives stopped working on Linux Centos 5.0 (kernel 2.6.18). Having spent half the day googling to find a fix, looking at udev, hal, fstab, bios settings, ata/piix/ide modules, jumper settings on the drive... I found a fix to the problem. Having tried to execute

modprobe ide-cd

I got an error message about not understanding what the dma option is. Which led me onto editing /etc/modprobe.conf and commenting out the the line:

options ide-cd dma=1

This fixed the problem.

25 November 2007

A kdb tick/q demo

1) Install q and kdb tick, including k4.lic file

2) In install dir

mkdir data
mkdir logs

create file tick/schema.q containing:
test:([]time:`time$();sym:`symbol$();size:`float$();price:`float$())

3) create a start.sh script for kdb
#!/bin/bash
export QHOME=/usr/local/bin/q
export PATH=$PATH:/usr/local/bin/q

$QHOME/l32/q tick.k schema $QHOME/data -o 0 -p 30000 >> $QHOME/logs/tp.log 2>&1 &
$QHOME/l32/q tick/r.k :30000 -p 30001 >> $QHOME/logs/rtdb.log 2>&1 &
$QHOME/l32/q $QHOME/data/schema -p 30002 >> $QHOME/logs/hdb.txt 2>&1 &

4) run start script

5) In a seperate q session insert some data
h:hopen `:localhost:30000
h ".u.upd[`test](`ibm; 10000.0; 98.0)"

24 November 2007

Subversion auto-props default

Its a shame that subversion's config for auto-props dont have some sensible defaults or allow me to set them on a per repository basis. Here are some that I use in my ${home}/.subversion/config.

[miscellany]
enable-auto-props = yes

[auto-props]
*.q = svn:eol-style=LF;svn:mime-type=text/x-q
*.sh = svn:eol-style=LF;svn:executable=ON;svn:mime-type=text/x-sh
*.java = svn:keywords=Id;svn:mime-type=text/x-java-source
*.png = svn:mime-type=image/png
*.jpg = svn:mime-type=image/jpeg

18 November 2007

Maven Help

mvn --help

Lists the profiles which are currently active for the build:
mvn help:active-profiles

Displays the effective POM for the current build, with the active profiles factored in:
help:effective-pom

Prints out the calculated settings for the project, given any profile enhancement and the inheritance of the global settings into the user-level settings:
help:effective-settings

To get help on a plugin:
mvn help:describe -Dplugin=xxx -Dfull
mvn help:describe -DgroupId=xxx -DartifactId=xxx -Dfull=true

Apache Archiva Maven Repository Server

Seems like a pretty decent Maven Repository server. I've set it up to manage my own Maven repositories, including my own "external-public" repository which proxies access to a number of the big external Maven repositories.

Its easy to install and all configuration is done through the web interface. As well as configuring repositories and connect repositories so one can proxy another, it adds user management to secure access via various roles. The docs on the home site aren't great but google and you will find other good config howtos.

For more detail on functionality see http://www.devzuz.org/blogs/oching/2007/11/05/1194233400000.html

Creating a default Maven project

mvn archetype:generate

This will ask multi-choice questions regarding creating a new project/module.

Or you can directly specify all options as:

mvn archetype:create -DgroupId=com.parwy.testproj -DartifactId=testproj -DarchetypeArtifactId=maven-archetype-quickstart

Archetypes include:
maven-archetype-archetype
maven-archetype-bundles
maven-archetype-j2ee-simple
maven-archetype-marmalade-mojo
maven-archetype-mojo
maven-archetype-plugin-site
maven-archetype-plugin
maven-archetype-portlet
maven-archetype-profiles
maven-archetype-quickstart
maven-archetype-simple
maven-archetype-site-simple
maven-archetype-site
maven-archetype-webapp

The default is maven-archetype-quickstart

13 November 2007

Web frameworks

I not had to do any web development for years but thought I'd archive this link away in case I need to look at them again. Hopefully, by the time I ever come across the need to do web development, the tools and frameworks will have evolved much further.

12 November 2007

What belongs on the domain model and what belongs on service classes

This article hit home, I have often tried to balance what belongs on the domain model and what belongs on "even more domain-specific" service classes. Poor old Gavin took some flak though... I totally agree with his follow up to the criticism.

11 November 2007

Bulk search and replace using sed

This little snippet replaces occurrences of oldString with newString in all files named *.txt in the current directory and below, keeping a backup of the original files.

for i in $(find . -name "*.txt"); do sed 's/oldString/newString/g' $i > $i-tmp; mv $i $i-backup; mv $i-tmp $i; done

14 October 2007

Deleting lots of files

If you are trying to delete a directory with lots of files in, rm will complain about to many files and refuse to delete. You can get round this with xargs:

find . -name "log-200709*.gz" -print0 | xargs -0 -t -r rm -f

Unit testing and documentation

When I left my last job, some of the big lessons I learnt in software development were:

1) Rely on dependency injection, from a design perspective, I don't mean using an IoC container like Spring. This meant avoiding singletons at all costs.

2) Compile a large system out of many small modules (jars) rather than compile the system into a monolithic jar. This was vital to prevent circular dependencies.

On my current project I would add a few more lessons:

1) Write unit tests, these may slow you down earlier, but they become a god-send three years down the line, when some of the original team members have left and been replaced by others. You want new members of the team to be able to get into and refactor any part of the system. At this point you must rely on unit tests, not on the memory of an old developer on the team telling them, "you'll have to watch out in that package, there is this issue you must remember or that issue you must ensure you don't break".

2) In the same vain as testing more documentation of packages, classes, methods and fields. You don't need much, as good naming of artifacts is often enough. But I probably could have documented a bit more.

3) Use more composition to change the behaviour of objects rather than inheritance. Using composition more occurs naturally through a dependency-injection approach, however it is still possible to overuse inheritance, which reduces reusability.

4) Be more strict about ensuring most classes are immutable, rather than making them mutable for performance reasons. I got some big gains in performance but I think I sacrificed more in terms of "gotchas" other developers face with mutable classes, e.g. storing an object in a set and then wondering why it is different later. Anyway some of the performance gains disappeared by the need to clone mutable objects.

5) Using libraries for units right from the beginning and pervasively. For units I mean something like Joda from dates/times/periods and classes for other measures such as monetary amounts.

29 April 2007

Oops I had an open http proxy

I accidentally configured my apache web server to be an open proxy. I only discovered this when my own web browsing became very slow. First thing I did was reboot my server, that didnt help, so next I rebooted my netgear router, that didnt help either. I waited a day in case it was a problem at the ISP (even though their website was not showing any incidents). So the next day I ran:

netstat -a --inet --notrim

and found that a huge number of hosts were connecting to my http port. I followed this up by looking at my apache access logs and true enough, hundreds of hosts were connecting and asking for URLs that were external sites.

First things I did was access the admin site of my router and disable http/https. I then fixed my apache configuration and restarted apache, before renabling http/https on my router. After this I watched my apache access logs, as expected many hosts were still hitting my webserver, although at least now it was returning 404, rather than proxying the http request. So I installed ntop and got some statistics on which ip addresses were the biggest culprits. Armed with a list of ip addresses I blacklisted any ip traffic from them using iptables, i.e:

iptables -v -A INPUT -s 213.240.225.101 -j DROP
iptables -v -A INPUT -s 200.155.23.28 -j DROP
iptables -v -A INPUT -s 219.110.175.160 -j DROP
iptables -v -A INPUT -s 125.92.99.94 -j DROP
iptables -v -A INPUT -s 201.236.88.90 -j DROP
iptables -v -A INPUT -s 201.211.89.201 -j DROP
iptables -v -A INPUT -s 125.65.112.35 -j DROP

27 January 2007

GroupLayout

GroupLayout looks quite useful for creating forms. It came out of Netbean's Matisse GUI builder and is making its way into Java 6. Here is an introduction: Tomas Pavek's Blog: Getting to know GroupLayout, part 1.

20 January 2007

Herman Miller Aeron Chair

Must buy a Herman Miller Aeron Chair at some point.

Google Code - Project Hosting

Google Code - Project Hosting is a Sourceforge like software management application that allows you to create a subversion repository for your code, a file repository for releases and a wiki for documentation.

01 January 2007

Importing a new subversion project

Say you want to import a project called contacts into subversion repository called main. First we create a dummy project to import, this creates the structure (trunk/branches/tags) for the project. Second we import the real project into the trunk area.

C:\sekhonp\it\builds>ls
drwx------+ 4 sekhonp None 0 Jan 1 15:24 contacts

C:\sekhonp\it\builds>mkdir emptyproject
C:\sekhonp\it\builds>mkdir emptyproject\trunk
C:\sekhonp\it\builds>mkdir emptyproject\branches
C:\sekhonp\it\builds>mkdir emptyproject\tags

C:\sekhonp\it\builds>svn import emptyproject http://svn.parwy.net/repos/main/contacts
Adding emptyproject\trunk
Adding emptyproject\branches
Adding emptyproject\tags

C:\sekhonp\it\builds>ls -al
drwx------+ 4 sekhonp None 0 Jan 1 15:24 contacts
drwx------+ 5 sekhonp None 0 Jan 1 15:26 emptyproject

C:\sekhonp\it\builds>svn import contacts http://svn.parwy.net/repos/main/contacts/trunk


We now have a repository structure that looks like:
main ->
contacts ->
trunk ->
project files...
branches ->
tags ->

You can now clean up and checkout the trunk of the project:

C:\sekhonp\it\builds>rm -rf emptyproject
C:\sekhonp\it\builds>rm -rf contacts
C:\sekhonp\it\builds>svn co http://svn.parwy.net/repos/main/contacts/trunk contacts
C:\sekhonp\it\builds>ls -al
drwx------+ 5 sekhonp None 0 Jan 1 15:43 contacts