Tuesday, 31 December 2013

In Our Day...


Another year gone! When I look at young people, I do feel old sometimes.

A colleague at work is trying to get rid of some old books. One such is about "The Internet", written in 1996.

Some of the things in there, boggle the mind of people who have not lived in that day and age.

Here're some thoughts on "When I was young...", "Back in our day...", things that you young whippersnappers have no clue about.

Let me know if you have more.

The Telephone

Back in our day, if we wanted to receive telephone calls, we had to stay at home! Because that's where the phone was. You couldn't take it with you, as it was attached to a telephonecord. That's right, your conversation went over a wire. And that is all the phone could do, make phone calls and receive phone calls, nothing else. You couldn't even send short text messages with it! As a matter of fact, you had to remember somebodies phonenumber or have it written down somewhere! The telephone was split into two parts, the part you could listen and talk to, and which was connected to the second part that you used to dial numbers. It had this circle with holes in it near the rim. Every hole was a number. If you wanted to dial a telephone number, you'd put your finger in the correct (first) hole, and turn this circle round until you couldn't anymore. Then you let go. The circle would turn back, and you could repeat this with the next number.

Ofcourse, we did have something called a "phonebooth", small compartments of glass and metal, that were put out in the street. Just big enough for one man to stand in, and it had a phone, that you could put quarters (money) in so you could call someone.

Some rich folk had a "car"phone (a mobile phone), which basically was shaped like a brick, with the same weight, and attached to a freakin' suitcase to carry the batteries and assorted technical crap around in.

Back in our day...

Back in our day, when you were lost, you were basically screwed. You could either ask some stranger or read the signs. If neither were here, and you were in the middle of ff-ing nowhere, you were simply out of luck. We got lost a lot, but, if it is any consolation, we always found the way again. Sometimes you'd bring a map along if you'd never been to the place before. It's kind of like a navigation system, only it's printed on paper, doesn't talk, is non-interactive, and doesn't automatically tell you where you are.

Back in our day, when we went to the supermarket, we'd have to pay for our groceries with cold, hard cash! At that time we had our own Dutch money, the Guilder (Gulden). None of this Euro stuff.

Back in our day, if we wanted to use the Internet, we'd use the telephonecord with a device called a modem. It would translate internet by making beeping sounds and sending that over the phoneline, to another computer that was listening to the beeping sounds to understand what you wanted.

Back in our day, we could write something called a letter on paper by hand, put it in an envelope, seal it and put a postage stamp on the envelope. We'd bring that letter to a post office and they had mailmen that would deliver these letters. It took at least two/three days for a letter to arrive. That was called mail.

Back in our day, if we wanted something calculated, we'd have to get a pen and paper and do it ourselves. Sure, shopkeepers had big bulky cash registers, but these were impractical for home use. My mom always told the story, that when she had her first calculator, she'd use it, and then, recalculate everything by hand, just to make sure.

Back in our day, when we wanted to change the channel on the television, we'd have to get up, off the couch, walk to television set, press the button on the television set for the other channel, and then walk back and sit down again.

Back in our day, when I went on vacation with my parents, my mom carried a special wallet with her. It had four (color-coded!) compartments that contained money of the different European countries we would be passing through/staying in. She'd order this money before the vacation at the local bank/exchange. If you were abroad, and were in need of money, you could use something called Travellers Cheques, basically a piece of paper you took to a foreign bank and you scribbled on it how much you wanted.

Back in our day, our car didn't have seatbelts, nor airbags or deformable zones. The two people in front got seatbelts, and that was it.

Back in our day, if you showed us a cassette tape and a pencil, we knew exactly what to do. Kids nowadays would have a big question mark on their face.

Back in our day, we could hear when the batteries of our cassette tape player, called a "Walkman", were running low.

Back in our day, we could record "software" from a radio station onto cassette tape and load it up onto our computer. Because we didn't have Internet.

Back in our day, if we wanted to see the pictures we took with our cameras, we'd have to bring something called "a negative" to a "photo development shop" and he'd have the pictures ready "on paper" the next week. And if some of the pictures were crap, that's too bad, because we had to pay for those too!


For All Those Born in the 70's and 80's!

We are the last generation that learned to play in the street, we are the first who've played video games, see cartoons in color and went to amusement parks. We were the last to record songs of the radio on cassettes and we are the pioneers of walk-mans and chat-rooms... We learned how to program the VCR before anyone else, play with the Atari, Super... Nintendo and believed that the Internet would be a free world all on a 56kbit modem. Traveled in cars without seat belts or air-bags & lived without cell phones. Rode our bicycles down the road without brakes. We never had phones but still kept in touch. We did not have Play stations, 99 television stations, flat screens, surround sound, mp3s IPods computers and broadband...

but nevertheless we had a GREAT Time

Wednesday, 25 December 2013

IoT - Internet of Things

A nice post on what the Internet of Things is. (Don't ask me, it's relatively new to me).

How I explained the IoT to my parents.

Back in the old days, when I was still working at a big Telecom company, we'd often heard something like it. It mentioned querying your Refridgerator, to see if you had enough ingredients for making Chicken Cacciatore, or if you had to go shopping. This is apparently the same thing in a new name.

Sunday, 8 December 2013

JSR 305: Annotations for Software Defect Detection

The NullPointerException is the most prevalent exception thrown in Java programs known to man. Dealing with it has always caused a great deal of Null checks in my (and other peoples) code.

At work we've recently started using the special annotations from JSR 305[1].

There are basically two. Nullable to indicate that a result of variable can be Null, and Nonnull to indicate that a result or variable will never be null.

The JSR 305 annotations are primarily intended for compilers or other tools that read source code. However, they are usually retained in the byte code for the benefit of static analysis tools such as FindBugs that inspect byte code rather than source.

An added benefit is that it forces me to think about what a method should return.

Eclipse, our work environment, has its own versions of those annotations, which we do not use.
  • org.eclipse.jdt.annotation.Nullable
  • org.eclipse.jdt.annotation.NonNull
  • org.eclipse.jdt.annotation.NonNullByDefault

The originals from JSR 305[1] are:
  • javax.annotation.Nullable
  • javax.annotation.Nonnull
  • javax.annotation.ParametersAreNonnullByDefault

In order to get the proper annotations, we do have to include a jar file. We use "jsr305-2.0.1.jar".

At the start of using these annotations, I've turned "Errors" off, regarding the Null annotations, as I was getting far too many errors.

The Executive Committee voted to list this JSR as dormant in May 2012. But the implementation of JSR 305 [1] seems to work in several IDEs.

Eclipse problems and workarounds

The tooling in Eclipse is of course not perfect in detecting where a Null is a possible problem. There are ways to help Eclipse make the right decision. In the examples below, the part of the code that Eclipse will indicate as being faulty is red underlined.
  1. The following code shows up Faulty in Eclipse. Eclipse/FindBugs cannot determine that the check for null negates the second possible NullPointer.
    // Potential null pointer access: The method getItem() may return null
    if (order.getItem() == null
        logger.debug("order has no item.");
    int price = compute(order.getItem());
    The workaround is as easy as making a temporary local variable.
    // no problem!
    Item item = order.getItem();
    if (item == null
        logger.debug("order has no more items.");
    int price = compute(item);
  2. Warning in the second line. Eclipse is not clever enough to figure out that assertNotNull will take care of it. Eclipse is smart, but not psychic.
    Replace it with another local variable and put some supressors on it.
    Item item = order.getItem();
  3. NullChecks are not being used in the original Java API, causing potential warnings. In the example, without the SuppressWarnings on the method, the last return would be underlined.
    public @Nonnull Set<Item> getItems() 
        if (items == null
            return Collections.emptySet();
        return Collections.unmodifiableSet(items);
  4. I've encountered an issue with Enums. Why does this happen? I have no clue.
       boolean result = hasProperty(name, StateEnum.MANDATORY);
       return result;

    private boolean hasProperty(@Nullable String name, @Nonnull StateEnum state)
    I can solve it in the obvious way, but am unable to find the cause of the problem.
       boolean result = hasProperty(name, StateEnum.MANDATORY);
       return result;

    private boolean hasProperty(@Nullable String name, @Nonnull StateEnum state)
  5. If the Null constraint annotations are not used in the Interface (specifically method parameters... it seems method return values don't matter), you are not allowed to add them in your implementation. You will receive the error message from eclipse.
    Multiple markers at this line
    - Illegal redefinition of parameter myParam, inherited method from SomeInterface does not constrain this parameter
    - implements mrbear.some.SomeInterfaceImpl.myMethod
  6. You are better off not using @Nonnull at the Hibernate Entity Level, as that is going to cause problems in some cases. Notable when using "Examples" for searching.

Note: Of course, it is a code smell to have to change your production code, to help Eclipse tooling deal with Null. This is probably the main reason, why it doesn't have the expected uptake in my company.

Mushroom cloud

Changing your existing code base to make use of this new check, is quite a job. If you start at a specific point (the point, to be more specific, where you were changing your code initially), it tends to mushroom cloud outwards. Somewhere along the line, you have to decide how far you wish to take this.

It would have been a great deal simpler, to have added this functionality straight from the beginning. Alas, hindsight is always 20-20.


I've not discovered any frameworks that inherently are using the same JSR305, except for one. Google Guava.


Recently had the issue that I had a List, of which I knew that each member of that List was not NULL. But I had no way of annotating this with @Nonnull.

It appears it is possible to (among other things) annotate generics once JSR 308[3] is added to Java. It is scheduled for Java 8 for now.[4]

The Checker Framework[2] makes it possible right now, apparently.

The syntax would look something like this:
protected @Nonnull List<@Nonnull String> getNames() {

Updated: 10/12/2013 added chapter on Generics and a problem-case I encountered.

Updated: 20/02/2014 added a critical note at the end of the chapter of "Eclipse problems and workarounds".


[1] JSR 305: Annotations for Software Defect Detection
Avoiding “!= null” statements in Java?
The Open Road: javax.annotation.
[2] Checker Framework
[3] JSR 308: Annotations on Java Types
[4] JSR 337: JavaTM SE 8 Release Contents

Sunday, 1 December 2013

Oracle Timestamp vs. Date

Recently have been hitting a brick wall due to a bug at work.

Query doesn't return any results if the date is "2009-06-01 00:00:00:01". But for the date "2009-06-01 00:00:00:00" it works fine.

The table is as follows:

And contains:
103581900/01/01 00:00:00:000000(null)

/* Formatted on 28/11/2013 17:02:58 (QP5 v5.227.12220.39754) */
FROM MUNICIPAL municipal1_  
WHERE municipal1_.code='0358'
AND (TO_TIMESTAMP('2009/06/01:00:00:01:01''yyyy/mm/dd:hh24:mi:ss:FF2'BETWEEN municipal1_.startdate
AND NVL (municipal1_.enddate - 1, TO_TIMESTAMP('2009/06/01:00:00:01:01''yyyy/mm/dd:hh24:mi:ss:FF2')))
The query above returns no rows. Why?

Some notes:
  1. the (null) for enddate means the timeperiod reaches into the future
  2. the sql function BETWEEN[5] checks to see if a value is within a certain range. The boundaries are inclusive.
  3. The enddate is a timestamp(6)
  4. enddate - 1 is no longer a timestamp, but a common date. An implicit datatype conversion takes place.[4]
  5. date doesn't store anything smaller than seconds [2]
  6. we lose the milliseconds
  7. NVL tries, if the first argument is null, to cast the second argument to the same type as the first argument (a date) [3]
  8. even though enddate is null, the data conversion still takes place
  9. ergo, the timestamp used now falls outside the range, and no results are returned.


There is one good solution to this. Don't use BETWEEN with timestamps if the range is exclusive the enddate. Use greater than/equals (>=) and lesser than (<) instead.

If you do wish to change a TIMESTAMP, use INTERVAL instead of arithmetic. [4]

/* Formatted on 28/11/2013 17:02:58 (QP5 v5.227.12220.39754) */
FROM MUNICIPAL municipal1_  
WHERE municipal1_.code='0358'
AND (TO_TIMESTAMP('2009/06/01:00:00:01:01''yyyy/mm/dd:hh24:mi:ss:FF2') >= municipal1_.startdate
AND TO_TIMESTAMP('2009/06/01:00:00:01:01''yyyy/mm/dd:hh24:mi:ss:FF2') < nvl(municipal1_.enddate, 


Do NOT use Arithmetic with Timestamps! You will get an automatic conversion to datatype Date and lose millisecond precision!

It is quite ugly to substract a day from a DATE, just so that you could use BETWEEN and not worry about the last boundary being included.

I guess it is my own fault for appropriating a bit of SQL code that was originally designed for DATES instead.


[1] Timestamp
[2] PL/SQL Data Type
[3] NVL
[4] Math with Timestamp

Sunday, 24 November 2013

What's the difference between Aggregation and Composition?

I've based this article heavily on information found in Wikipedia[1]. For more information, have a look there.
a family of links to different classes. For example: person-address
a specific kind of association between only two classes, parts can belong together but are things of themselves. For example: duck-pond, person-room
a specific kind of association between only two classes, but stronger than an aggregation, parent-child relationship, child cannot exist without the parent. For example: car-steeringwheel/house-room
a weak relationship, which indicates that one class depends on another because it uses it at some point in time. One class depends on another if the independent class is a parameter variable or local variable of a method of the dependent class. This is different from an association, where an attribute of the dependent class is an instance of the independent class. For example: AuthenticationService-User


In the Real World™ a car engine is part of a car, and if the car is scrapped (destroyed), so is the engine. In the realm of Software or Databases, it is often the case that an engine is suitable for more than one model car. So, matters are never black and white.[1]


[1] Wikipedia - Class diagram

Saturday, 9 November 2013

JFall 2013

I went to JFall 2013, the annual Dutch Java Conference, organised by the NLJUG on Wednesday, the 6th of November 2013 in Nijkerk, the Netherlands.

I found the Keynotes to be singularly exciting, a lot better compared to last year.

Main change was the main sponsor, ING, who have some interesting requirements for their software and who have undergone a change
from the traditional way of developing software to the DevOps way.

My chosen schedule:
  • Shootout! Template engines on the JVM (Jeroen Reijn)
  • Retro Gaming with Lambdas (Stephen Chin)
  • Code-driven introduction to the Java EE 7 Platform (Arun Gupta)
  • Hands-On Labs: Gradle (Mr. Haki)
  • Scratching the Surface with JavaFX (James Weaver)
  • ING Beleggen: how top quality software and state-of-the-art technology leads to continuous delivery (Jos Klompe)

I managed, according to my colleague, to miss out on the talk by Allard Buijze called 'Reactive applications: ready for the future'. But, if all is well, I should be able to view it at Parleys.com.

I hear the talk 'On the integrity of data in Java Applications' (Lucas Jellema) was also really good, that I didn't visit. There's only 24 hours in a day.

Shootout! Template engines on the JVM

The presentation was provided by Jeroen Reijn of Hippo (the Java CMS). He mentioned a few old Template engines, JSP, Freemarker and Velocity. Hippo seems to make use of Freemarker, at least at the moment.

Then he went into some (relatively) new Template engines with included benchmarks:
  • Thymeleaf (web+spring mvc+large memory footprint, a little slow)

    Has Natural Templating, which means that additional namespaces in the XHTML will cause the xhtml pages to render on a normal browser. (It just ignores the additional namespaces).

    Benchmark: 25,000 requests took 30 sec.
    For whom: html designers
  • Mustache (not just web+spring mvc addon)

    Logic-less, so they claim, has no control statements (if, for, while, else, etc). Called Mustache because Expressions are written between double curly brackets ({{), and those look like a mustache.

    Good IDE support.

    Benchmark: 25,000 requests took 12/13 sec.
    For whom: html designers
  • Jade (web)

    A Node.js based template engine. Indentation is part of the syntax, which means there's no need for closing tags. It's minimal, easy to read and small.

    Good IDE support.

    Benchmark: 25,000 requests took 20 sec.
    For whom: java developers
  • Scalate (Scala Template Engine, very slow due to the layout mechanism)

    Scala based, integrates well into Spring, Play, Lift, etc. Supports multiple template languages. The one language Jeroen went into deeper detail about is called SCAML.

    Benchmark: 25,000 requests took 100 sec.
    For whom: Scala enthousiasts

For the java developers that feel the need, the need for speed, stick to JSP and Freemarker.

Retro Gaming with Lambdas

Stephen Chen provided both an overview of lambdas in Java 8 and a practical example in the form of a JavaFX game called 'MaryHadALittleLambda'.

The latter part of which is available on github at https://github.com/steveonjava/MaryHadALittleLambda.

He talked about
  • lambdas
  • extension methods
  • the hamburger operator
  • method references
  • @FunctionalInterface

Code-driven introduction to the Java EE 7 Platform

It was given by the Arun Gupta. I was quite surprised to learn that he's been an employee of JBoss for the past three weeks.

I'm also still not used to the fact that their JBoss Application Server is now called ... Wildfly.

He did a quick code-driven introduction in the following topics:
  • WebSocket (URLs start with 'ws://')
  • JAX-RS (JS339)
  • JSON Client Api
  • Batch
  • JMS

JEE 7 samples are available at his github account at https://github.com/arun-gupta/javaee7-samples.

Hands-On Labs: Gradle

Gradle is sort of like Maven only with a Groovy DSL instead of XML and is more flexible than Maven. Where Maven required selfmade Plugins to get it to do the things you want, Gradle can just be adapted.

From what I hear it is quite easy, even automatic, to convert Maven build scripts into Gradle build scripts.

It seems that NetBeans has builtin support for Gradle, no plugins required. I gotta check that out!

Gradle helps in building a Project. The project consists of tasks. A task consists of actions.

The Lab was one of the easiest and simplest to do. I could have just as easily done that at home.

Scratching the Surface with JavaFX

I enjoyed the presentation of Jim (James) Weaver immensely. It started off with a Dutch Music Guessing Game to break the ice, presumably. After that he showed us the touch events interface in JavaFX. Then proceeded to dazzle us with simple 3D in JavaFX3D.

We should check out iotcommunity.net, and I should mention in JIRA that I want to have shadows from lightnodes.

JavaFX Ensemble application is very nice, as a starting point on what JavaFX makes possible.

Currently, there is some 'unofficial' versions of JavaFX that run on Android and IOS. Expect more news on this.

JavaFX can be lay outed by means of CSS. CSS is a good way to make it more 'finger friendly'. We're talking about touch screens after all.

The 3d parts showed things like shapes (ball, cube, triangle mesh, textures, bump maps), lights (ambient and directional), cameras.

One of the more awesome parts of the presentation was the fact that he wanted to play the guitar, when he was away from home, but without dragging his guitar along. So he programmed touch sensitive guitar strings in 3d on his Microsoft Touchpad. He could play quite well and it was very impressive to see a really good demo of all the touch events he had been talking about.

ING Beleggen: how top quality software and state-of-the-art technology leads to continuous delivery

Jos Klompe and his colleague provided a riveting and energizing talk about what they do for ING, to be specific the portal that customers and employees use to trade stocks and other, more complex, financial instruments.

The most interesting in the story is the complete turnaround from a Bank with a traditional waterfall model over to a dynamic Bank using Agile and Scrum and DevOps teams that is putting IT at the core of its business.

Thursday, 31 October 2013

System.out.println Debugging

Who hasn't done the following? Adding appropriate System.out.println statements to your code, in order to actually see what is going on.

I admit to using it once or twice. I must even admit that once or twice I've managed to accidentally check my printlns in.

I found the following Small Gem in the Tips and Tricks of Eclipse[1].

A picture of the properties of a breakpoint in Eclipse is available to the left.

I'm going to assume that a similar trick can be used in other IDEs.

Some special notes about the breakpoint:
  • it's conditional
  • it will only suspend execution when the condition evaluates to true
  • the condition will always evaluate to false
  • conclusion: the breakpoint will never suspend execution
  • it contains System.out.println statements

In effect, we have created a System.out.println statement, without actually changing the code.

This has certain advantages:
  • Debugging code isn't actually checked in by accident
  • You do not actually need to have the source code or to deploy anything to get the print statements
Already it is being used at my workplace by the Helpdesk for easy access to which SQL queries are sent to the database.


[1] Tips and Tricks (JDT) (Java Development Tools
Eclipse Java development user guide

Sunday, 27 October 2013

Trainwreck vs. Method Chaining

I thought it worth while to expound on the differences between a train wreck and method chaining.

Whereas a trainwreck is a bad idea, method chaining is an excellent idea. This blog will try to explain why these two are polar opposites on the good/bad scale, even though the syntax differs very little.


A trainwreck is a bad idea. It provides an Object A with in depth knowledge of several other objects. Knowledge that should be contained in the individual objects instead of Object A.

In code it would look like follows:

public class A
    public void someMethod(B b)

A Trainwreck breaks the Law of Demeter[1] (and everyone knows, if you break a law, you have to go to jail).

A solution would be to just tell B to do it, and let it figure it out.
public class A
    public void someMethod(B b)

Method Chaining

Method chaining is an excellent idea. The methods used always return the Object itself. It's a good way of constructing an Object without having to resort to multiple different Constructor methods with varying (large amount of) parameters.
public class Person

    private long id;
    private String firstName;
    private String lastName;
    private String address;
    private String telephone;
    private String title;

    public Person(long id)
        this.id = id;

    public Person setFirstName(String firstName)
        this.firstName = firstName;
        return this;


public static void main(String[] args)
    Person mrBear = (new Person(1l)).setFirstName("B.").

Method Chaining does not break the Law of Demeter[1]. It doesn't even talk to friends (in this case), but only to itself.

Here I have used a simplified example. Mostly you'd create a PersonBuilder to make Persons.[4]

For some great uses of method chaining check out [2] and [3].


[1] Glossary
[2] FluentInterface
[3] Expression Builders
[4] Too Many Parameters in Java Methods, Part 3: Builder Pattern

Thursday, 10 October 2013

Examples of a HashSet - Followup

See Examples of a HashSet for the original article.

It seems according to "Unique hashCodes is not enough to avoid collisions", even perfect hashcodes might not be enough for for example HashMap collections.

Wednesday, 2 October 2013

Red Soup

Recently had a real life experience that I could map to Software Design.

Occasionally, I or one of my colleagues gets beverages for everyone. The scheduling is: the one who wants to have a beverage gets some for the rest as well and queries their preference beforehand.

In this case, the job landed on one of my colleagues.

The conversation went something like:
Guy1: What would you like?
Me: I'd like some Red Soup.
Guy1: What kind of Red Soup?
Me: Red Soup! You're the Factory, figure it out.
Me: Convention Over Configuration!
Guy2: Should he draw you an UML diagram? As long as it implements or extends RedSoup.
Guy1: With Cream?
Me: That's an implementation detail. Not interested! Red Soup! Go!

It left me in stitches, I'm geeky like that.

(added an overview of the different arrows in UML. I always get confused.)

Wednesday, 25 September 2013

Architect Enterprise Applications with Java EE - Course

I had the opportunity to attend the Oracle course on "Architect Enterprise Applications with Java EE"[1] from Monday 09/09/2013 to Friday 13/09/2013. (after forking over a big sack of money)

The Course does have a number of Lab exercises, mostly based on UML or open questions (that created several discussions, which I enjoyed immensely). UML diagrams were drawn using UMLet[3].

I took the course as I wish to prepare for the 'Oracle Certified Master, Java EE 6 Enterprise Architect' certification.

This is the first step in the Certification Process [2].

I did receive some very interesting Course material along with it. However, it's been licensed to me specifically. (I could let you read it, but then I would have to kill you. Oracle licensing rules say so.)


[1] Architect Enterprise Applications with Java EE
[2] Oracle Certified Master, Java EE 6 Enterprise Architect
[3] UMLet

Tuesday, 20 August 2013

My Current Network

I just thought I'd create a diagram of what devices I have where in my network, with what names. It's beginning to get a little difficult to keep them apart.
Made a small batch script to check my network ips, just for fun.
for i in {1..150}
   ping -c 3 -w 100 10.0.0.$i | grep -e "icmp_req" -e "PING"
As you are no doubt aware, I've based the names of my devices, the naming scheme, on characters in Sir Arthur Conan Doyle's Sherlock Holmes.

Monday, 12 August 2013

Raspberry Pi


My current hardware setup seems to work okay with the Raspberry Pi.
  • Raspberry Pi model B
  • Pretty Adafruit Pi Box (Enclosure)
  • Belkin Gold Portable Audio Player-to-Stereo Cable (2xRCA male to 3.5 mm male)
  • Belkin USB 2.0 Micro B Cable
  • Sitecom USB 2.0 hub 4-port with power adaptor and weird USB port on top (ART. CODE CN-050 v1 002)
  • SanDisk Ultra SDHC Card 32 Gb class 10
  • Perimac USB 2 to Fast Ethernet Adapter
  • RCA cable (male to male) (I had one lying around)


  • mpd - Music Player Daemon (plays audio files in "/var/lib/mpd/music")
  • deluged - Deluge BitTorrent Client [2]
  • deluge-console - Deluge BitTorrent Client console
  • xbmc - X Box Media Center
  • Java JDK 8 - for the arm hf


  • wanted to have internet 'always on' without a lot of power/computers involved
  • wanted to be in control of the software
  • wanted to download stuff easily
  • wanted to be able to send my solar panels input to the internet during daylight hours
  • wanted to play music on my (very old) stereo
  • wanted to do stuff with our old TV
  • wanted new toys to play with

Trying Wheezy Raspbian

I've downloaded Wheezy Raspbian, nowadays with Hard Float, and put it on a 32 GB SD Card.
dd bs=4M if=2013-07-26-wheezy-raspbian.img of=/dev/sdb

Downloaded 2013-05-29-wheezy-armel.zip first, which is the Software Floating Point image. XBMC is compiled for the Hardware Floating Point image. So, it won't work. Originally, I had envisioned playing with Java, and Java SDK was not available for Hardware Floating Point at the time. This has now been remedied, luckily. So no reason not to use Hardware Floating Point.[5][6]

Installing XBMC

Now, I've already tried RaspBMC, and I didn't like the fact that my default linux network scripts were replaced by XBMC scripts. Especially, as I had a non-standard setup (two ethernet ports).

So I installed XBMC on a standard Wheezy Raspbian. [4]

My TV is Black & White?

My TV provided a black and white screen. Now, given I'm European, I assumed the TV to use PAL (Phase Alternating Line, 'Picture Always Lousy') and not the (American?) NTSC (National Television System Committee, 'Never The Same Colour'). NTSC is probably the default.

Added sdtv_mode=2 to /boot/config.txt (0 is NTSC, 1 is Japanese NTSC, 2 is PAL, 3 is Brazilian PAL.)[1]

Attaching a second Ethernet by means of USB

I'm wanting to use the thing (amongst other things) as my little firewall.

My second ethernet is detected. Yay! My /etc/network/interfaces file looks like:
auto lo eth0 eth1

iface lo inet loopback
iface eth1 inet dhcp
iface eth0 inet static

#allow-hotplug wlan0
#iface wlan0 inet manual
#wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
#iface default inet dhcp

I have my little firewall script in "/etc/network/if-up.d/" so it runs upon boot, after network setup. But I usually forget to turn on "net.ipv4.ip_forward" in /etc/sysctl.conf.

Root access

Root access to Raspbmc is disabled, and you should use 'sudo'. But you can turn it on. http://www.raspbmc.com/wiki/user/root-access/.

I was never a "sudo" guy, but I'm getting used to it.

Expanding the root partition by hand

My Second SD card provided the following problems, after indicating that the Raspberry Pi should expand the SD Card partition and reboot. As the other SD card I have did not have this problem, I surmise that the SD card is a bit dodgy. This is the message I saw (a lot) on the screen upon boot:

end_request: I/O error, dev mmcblk0, sector 137029
mmcblk0: error -110 transferring data, sector 137030, nr 185, cmd response 0x900, card statux 0xb00
So, I attempted to expand my SD Card partition on my labtop. That fixed the problem (for now).[7]


[1] RPiconfig
[2] How to Turn a Raspberry Pi into an Always-On BitTorrent Box
[3] RaspbianXBMC
[4] Michael Gorven's XBMC packages for the Raspberry Pi
[5] JavaFX on Raspberry Pi in 3 Easy Steps
[6] OpenJDK version for the Raspberry Pi
[7] RPi Resize Flash Partitions
Official Raspberry Pi Website

Tuesday, 6 August 2013

Old Gists

Recently found out I can refer directly to different revisions of a Gist, by adding the hashtag of a revision.

Instead of <script src="https://gist.github.com/maartenl/6048082.js"></script>.

This <script src="https://gist.github.com/maartenl/6048082/95c021aa4e2b1dc1570b5f70c2dcc95a26944fe1.js"></script> will show the revision with hashtag 95c021aa4e2b1dc1570b5f70c2dcc95a26944fe1.

It's very REST-y this way.

It provides an ideal way to show different refactorings in a blog.



Embedding a specific revision of a Gist on GitHub.com via JavaScript

Tuesday, 30 July 2013

Mis-Using Abstract for Static Classes

Recently came across this construction in the code we have at work.

Now, some notes:

  • nowhere was a subclass defined to implement the 'abstract methods' in StringUtilities.
  • as a matter of fact, there were no 'abstract methods'.
  • all other methods were defined 'static'

It seemed a weird way to defeat instantiation (badly, by the way, as a person could simply subclass the thing and instantiate that).

Proper defeating of instantiation is done by means of a private constructor.

But, if what you need is a Singleton, perhaps it's better to just define a single ENUM.

Nobody could quite explain to me in which cases the first example would be a "Good Idea"™.

Tuesday, 23 July 2013

Examples of a HashSet


Some Real Life Examples where we use a Hash and Buckets.

Let me first shortly introduce how a hash works by means of the first example, an AddressBook, and as a followup how it is implemented in Java's HashSet.


A HashSet is a specific implementation of a Set. It uses the Hash of the stored objects to determine the position in an array of what are called Buckets. Every bucket in the array is, if filled, filled with another collection. So a HashSet is actually a collection of collections.

Let's see some examples of hashes people use when finding addresses in an AddressBook:
perfect hash: returns number in a numeral system with radix 26
in this case each name in the address book has a unique hashcode. Every list contains exactly one entry. If you use this as an address book, each page contains at most one address. You are going to need a huge amount of pages, if you have a lot of friends and relatives. With this behaviour it would make more sense to store addresses via a key-value store (for example an implementation of Map).
average hash: return first letter of the last name, capitalised
in this case, the behaviour is the same as your average normal address book. Each list contains all address of people whos names start with the same letter.
bad hash: return 'A'
in this case, the behaviour is terrible! You could compare it to an address book, on which all addresses are written on the first page. This way, the hashset will behave like a normal linked list.
Let's assume the hash used is the middle one. The Hashcode of entries in an AddressBook takes the first letter of the last name (capitalised).

Java's HashSet

We assume a class Address, containing the hashCode() method that returns the first letter of the last name.

The HashSet<Address> is implemented by using a HashMap<Address, Object> internally.

Now, the HashMap has an underlying data structure consisting of a Entry<Address, Object>[] table. This is the array that is going to contain all the first elements of each linked list. In other words, this is the Array containing the buckets.

The index of a bucket in the array is almost equivalent to the hashCode of the Address instance.

When I say almost, I mean that some additional conversions take place on the original hashCode, for instance:
  • In the case of using Strings Sun (Oracle?) uses a special hashing function (sometimes)
  • Some bit twiddling takes place to prevent collisions due to hashCodes that do not use lower bits, and therefore would cause the start of the bucket array to never be used.
  • The result is "binary and"ed, ofcourse, with the table length (which is always a power of two) as a quick sort of modulo function to prevent ArrayIndexOutOfBounds problems.

For this reason, the buckets are not at the positions in the bucket array as you would expect, given the answers of the hashCode() function.

Every bucket contains the first Entry (Entry<Address, Object>) of a linked list.

A new entry is added to the top of the Linked list.

Where each Entry has a pointer to the next Entry in the bucket, or null if it is the only or last entry, effectively creating/ending the linked list.

So the contents should look like follows:


The Hashcode used in a supermarket is infinitely worse than my Addressbook. To start a search, accost an employee of said supermarket and inquire after the location of an article, say washing detergent. You'll receive the hashcode "aisle 14" or some such. When at the location of said hashbucket, you find all articles jumbled together. Your senses assailed by every colour swirl, bubble and geometric shape imaginable with letters and numbers in wild and disturbing images.

If you are lucky, the hashcode is maginally improved with additional information like "on the left", or "at the very end".


Apparently, in real life, there are HashSets in use as well. We can only come to the conclusion that any logical data structure that we use in real life without thinking, has it's corresponding implementation in the Software Realm.

I hope you've enjoyed this journey of discovery into the Java source code, as much as I did.


Why always override hashcode() if overriding equals()?
Java 6 HashMap JavaDoc
Java 6 HashSet JavaDoc
Java Generics and Collections
Maurice Naftalin & Philip Wadler

Tuesday, 16 July 2013


The Formula annotation is used on a property of a Hibernate entity. It is currently not a part of the Java Persistence Standard, and thusly is a Hibernate extension.

I thought I'd write some of my notes and bookmarks to other sites here, as the information in the Hibernate book seems very brief.


  • the SQL in the formula is inserted automatically whenever you retrieve the Entity, without you having to think about it.
  • the advantage compared to calculating the fields in the Getter or Setter, is that it is possible to use the property in HQL queries. For example, adding "ORDER BY existsFreeby" in HQL without problems.


  • HQL cannot be used in the Formula.
  • you have plain SQL in your Entity. I know this is normal according to the JPA spec, but at work they tend to frown on that. Don't ask me why.
  • whenever a keyword is unknown, it will prefix it with the name of the Entity table. This is a good feature, normally. In the example give, for example, the "id" will be expanded to "ORDERS0_.id". However, in the case where we used COALESCE, if we had a space between COALESCE and the opening bracket, like so "COALESCE (", Hibernate would expand this to "ORDERS0_.COALESCE (" causing an error, obviously.
  • a problem can occur when you are using ansi sql queries and using query.addEntity(Orders.class); It will break on that now.


Always use opening and closing brackets at the beginning and end of the actual SQL part. As it is inserted into the Hibernate created SQL, it might break the syntax if you forget.

References Formula
Hibernate derived properties
Calculated property with JPA / Hibernate
@Formula adds a this_. prefix to an SQL function - Hibernate HH-5265
Hibernate with JPA ignoring @Formula

Tuesday, 9 July 2013

Java Persistence with Hibernate, Second Edition

Recently bought "Java Persistence with Hibernate, Second Edition" from Manning Publications Co. It contains Hibernate 5 in detail along with the Java Persistence 2.1 standard (JSR 338).

It's the first time for me to try out MEAP, Manning's Early Access Program. I can read chapters as they are written.

Once the book is finished, I get the dead-tree-copy in the mail.

So far, I'm rather pleased.

Tuesday, 2 July 2013

Properly Accessing your Outer Class

Recently, during my work, a colleague indicated a way to make it obvious that a variable used in an Inner class, belonged to the outer class. In order to prevent being ambiguous.

I didn't know that could work.


Nested Classes - The Java Tutorials

Tuesday, 25 June 2013

Java EE 7 Launch

I have been unable to write about the Webcasts I have followed on the 12th of June regarding the release of Java EE 7. (See Java EE 7 Launch Webcast: Jun 12th)

So here's my attempt.

I'm going to put some points down as a reminder for me on what are the main points of JEE 7 indicated in the Webcast.
  • HTML 5
    • WebSockets
    • Servlet 3.1 NIO
    • JSF 2.2
    • JSON
      • Streaming API - searching and retrieving data from large JSON documents
      • Object Model API - creation and updating of JSON documents
    • REST
  • Developer Productivity
    • More Annotated POJOs
    • Less boilerplate code
    • Cohesive integrated platform
  • Meeting Enterprise Demands
    • Batch Processing
      • Item oriented : chunk (normally used)
      • Task oriented : batchlet
    • Concurrency
    • JMS


Your First Cup: An Introduction to the Java EE Platform
Release 7 for Java Platform, Enterprise Edition
Introduction to Java Platform, Enterprise Edition 7
An Oracle White Paper, June 2013

Thursday, 23 May 2013

Java EE 7 Launch Webcast: Jun 12th

Wednesday, June 12 and Thursday, June 13 (depending on time zone)

Event One

Event Two


9:00 a.m. PT

9:00 p.m. PT

Keynote Address

10:30 a.m. PT

10:30 pm PT

On-Demand Breakouts Available

10:30 a.m. PT

10:30 pm PT

Dynamic, Scalable HTML 5 – Chat Session

11:30 a.m. PT

11:30 p.m. PT

Meeting Enterprise Demands – Chat Session

12:30 p.m. PT

12:30 a.m. PT

Increasing Productivity – Chat Session


Java EE 7 Platform Completes the JCP Final Approval Ballot
Java EE 7 Launch Webcast: Jun 12th, Register Now
Time Zone Converter – Time Difference Calculator
System Test

Saturday, 6 April 2013

Mirroring a.k.a. RAID 1


I have recently bought 2 WD (Western Digital) 2 TB Elements external (USB 2.0) harddrives. I am using them for making Backups, therefore I considered a NAS to be overkill.

This article is pure for me to remember the steps I took to get RAID 1 working, and which devices I have, and what my logs said.

The steps followed are basically the steps as mentioned in [1].


# yum install mdadm
# apt-get update && apt-get install mdadm
# fdisk /dev/sdb
# fdisk /dev/sdc
# mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
# mkfs.ext3 -j -Lraid1 -v -m1 /dev/md0
-- -v = verbose
-- -m1 = percentage of disk used for Root, 1% is more than enough (=20gb)
-- -j = journaling
-- -L = label, in this case "raid1".
# mkdir /mnt/raid1
# mount /dev/md0 /mnt/raid1
# df -H
What my system said in "dmesg", once the drives were under power and plugged into USB ports.
[ 1128.092533] usb 2-1.3: new high-speed USB device number 8 using ehci_hcd
[ 1128.189507] Initializing USB Mass Storage driver...
[ 1128.189698] scsi10 : usb-storage 2-1.3:1.0
[ 1128.189820] usbcore: registered new interface driver usb-storage
[ 1128.189824] USB Mass Storage support registered.
[ 1129.190936] scsi 10:0:0:0: Direct-Access     WD       Ext HDD 1021     2021 PQ: 0 ANSI: 4
[ 1129.192184] sd 10:0:0:0: Attached scsi generic sg2 type 0
[ 1129.195482] sd 10:0:0:0: [sdb] 3907024896 512-byte logical blocks: (2.00 TB/1.81 TiB)
[ 1129.197278] sd 10:0:0:0: [sdb] Write Protect is off
[ 1129.197284] sd 10:0:0:0: [sdb] Mode Sense: 17 00 10 08
[ 1129.199146] sd 10:0:0:0: [sdb] No Caching mode page present
[ 1129.199153] sd 10:0:0:0: [sdb] Assuming drive cache: write through
[ 1129.204101] sd 10:0:0:0: [sdb] No Caching mode page present
[ 1129.204118] sd 10:0:0:0: [sdb] Assuming drive cache: write through   
[ 1129.220472]  sdb: sdb1
[ 1129.225647] sd 10:0:0:0: [sdb] No Caching mode page present
[ 1129.225653] sd 10:0:0:0: [sdb] Assuming drive cache: write through
[ 1129.225658] sd 10:0:0:0: [sdb] Attached SCSI disk
[ 1131.676246] usb 2-1.8: new high-speed USB device number 9 using ehci_hcd
[ 1131.770350] scsi11 : usb-storage 2-1.8:1.0
[ 1132.770974] scsi 11:0:0:0: Direct-Access     WD       Ext HDD 1021     2021 PQ: 0 ANSI: 4
[ 1132.772018] sd 11:0:0:0: Attached scsi generic sg3 type 0
[ 1132.773413] sd 11:0:0:0: [sdc] 3907024896 512-byte logical blocks: (2.00 TB/1.81 TiB)
[ 1132.775265] sd 11:0:0:0: [sdc] Write Protect is off
[ 1132.775274] sd 11:0:0:0: [sdc] Mode Sense: 17 00 10 08
[ 1132.777192] sd 11:0:0:0: [sdc] No Caching mode page present
[ 1132.777198] sd 11:0:0:0: [sdc] Assuming drive cache: write through
[ 1132.781952] sd 11:0:0:0: [sdc] No Caching mode page present
[ 1132.781958] sd 11:0:0:0: [sdc] Assuming drive cache: write through
[ 1132.800861]  sdc: sdc1
[ 1132.805944] sd 11:0:0:0: [sdc] No Caching mode page present
[ 1132.805950] sd 11:0:0:0: [sdc] Assuming drive cache: write through
[ 1132.805956] sd 11:0:0:0: [sdc] Attached SCSI disk
Originally, the drives came preformatted with NTFS, but that was easily fixed.
root@micemouse:~# fdisk /dev/sdb

Command (m for help): p

Disk /dev/sdb: 2000.4 GB, 2000396746752 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907024896 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000575ba

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048  3907024895  1953511424    7  HPFS/NTFS/exFAT

Command (m for help):

root@micemouse:~# mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
mdadm: /dev/sdb1 appears to be part of a raid array:
    level=raid0 devices=0 ctime=Thu Jan  1 01:00:00 1970
mdadm: partition table exists on /dev/sdb1 but will be lost or
       meaningless after creating array
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
mdadm: /dev/sdc1 appears to be part of a raid array:
    level=raid0 devices=0 ctime=Thu Jan  1 01:00:00 1970
mdadm: partition table exists on /dev/sdc1 but will be lost or
       meaningless after creating array
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
Time to grab a cup of something for the next part. Formatting 2 Terabytes takes a while.
root@micemouse:~# mkfs.ext3 -j -Lraid1 -v -m1 /dev/md0
mke2fs 1.42 (29-Nov-2011)
fs_types for mke2fs.conf resolution: 'ext3'
Filesystem label=raid1
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
122093568 inodes, 488345040 blocks
4883450 blocks (1.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
14904 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
    102400000, 214990848

Allocating group tables: done                           
Writing inode tables: done                           
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done


# watch -n 2 cat /proc/mdstat   
# tail -f /proc/mdstat
# mdadm --query /dev/md0

maartenl@micemouse:~$ tail -f /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdc1[1] sdb1[0]
      1953380160 blocks super 1.2 [2/2] [UU]
      [>....................]  resync =  0.1% (2340864/1953380160) finish=1801.8min speed=18046K/sec
unused devices: <none>
tail: /proc/mdstat: file truncated
Personalities : [raid1]
md0 : active raid1 sdc1[1] sdb1[0]
      1953380160 blocks super 1.2 [2/2] [UU]
      [>....................]  resync =  0.1% (2340864/1953380160) finish=1801.8min speed=18046K/sec
unused devices: <none>


[1] How to configure usb mybooks as RAID 1
Linux - creating software reaid one arrays
RAID - Wikipedia

Friday, 29 March 2013

Building a Paradox

According to Wikipedia[1]:
“A paradox is an argument that produces an inconsistency, typically within logic or common sense.”
Every once in a while, it helps to do something that is totally unrelated to the field of Software Design. In my case I like to make the occasional wooden thing. In this case, I decided to build a wooden paradox.

Just to elaborate, I wished to make a wooden version of Curry's Paradox[2]. If you wish to know how the trick works, youtube[3] has some very good explanations.

This little project is one of the easiest I've made. (Quickest too)


  • some wood, 1 cm thickness, made of glued-together-layers of wood
  • different colour paints
  • varnish
  • magnets

  • a fretsaw (I used an iron saw, because I'm short a fretsaw)
  • glue
  • sandpaper
  • tapemeasure (essential)
  • pencil
  • paper (I used cm2 paper)
  • scissors
  • eraser
  • brushes

The Making Of

Cut the shapes out of paper.

Mark the shapes on the wood.

Saw the shapes using the fretsaw.
Use sandpaper on the pieces.

Paint them.

Varnish them.

After drying, apply glue to the magnets and glue them to the shapes.


It's not rocket science!


I managed to make something appropriate using an iron saw instead of a fretsaw, and I do not recommend it. It's a pain to create 90 degrees corners.

Next time need to sandpaper more, as the edges were quite rough.


[1] Paradox - Wikipedia
[2] Missing square puzzle
[3] Curry's Paradox and the Notion of Area: Part I (Tanton)

Curry's Paradox

Friday, 15 March 2013

Natural Progression of Java for Statement


I find it fascinating the way Java is being developed, and accrues different new features from other "newer" programming languages.

One of the things that struck me forcefully recently, is the progression of our beloved for Statement. So forcefully, that I had to make a blog about it.

Output of the programs listed below is always:
James Gosling is 57 years old.
Charles Babbage is 221 years old.
Alan Turing is 100 years old.
Donald Knuth is 75 years old.
Edsger Dijkstra is 82 years old.
Anyone interested in the Person.java class used, can find it here.

Version 1

This is the way we did it back in the old days. A counter for 0 until we reached the size of the list we wanted to iterate over. And every time we had to cast what we got out of the list, as the list contained Objects.

Version 2

Finally! A Collections framework! With default naming, so we could always more or less assume what the exact Contract was that the Collection adhered to, once we knew the name of the Collection used.

Version 2.5

Yay! No more casting, no more 'counters' and we can add a variable number of arguments to some of our collection framework methods.

Version 7

Small change, due to project Coin. We have a simpler notation for Generics.

Version 8

Lambdas, here we come!


The for-loop, while still a very important tool in the arsenal of the Developer, seems to have been relegated to the internals of the Frameworks. I would not be surprised if the for-loop will become used less and less.

Small note

Whilst trying to get Lambda functions to work, I made the mistake to download the OpenJDK snapshot from Oracle that does not actually contain Lambda syntax yet. Try [1].


The for Statement
Maurice Naftalin's Lambda FAQ
JDK 8 Features
Java 8: The First Taste of Lambdas
[1] Java™ Platform, Standard Edition 8 Early Access with Lambda Support
Java Version History

Wednesday, 13 March 2013


So, after several of years in the business, I've had the pleasure to learn a new database server in each and every new job. I consider it a happy coincidence.

Here's a list:
1999 - 2009 : Sybase ASE
2009 - 2010 : Microsoft SQL Server
2011 - now : Oracle Database Server
2013 - now : recently followed the course to learn the workings of the Postgres Pro Advanced Database Server of EnterpriseDB, which is based on PostgreSQL

And, of course, I've dabbled at home with MySQL, but then, who hasn't?

So, in total:
  • Sybase ASE
  • Microsoft SQL Server
  • Oracle Database Server
  • PostgresPro Advanced Database Server
  • PostgreSQL
  • MySQL

I should be expanding this list with some NoSQL databases, but currently no projects that fit the bill on hand.

Friday, 8 March 2013

Postgres Plus 9.0 Advanced Server Database Administration

Recently I got into a tiff over the lack of CHECK CONSTRAINTS in MySql. The MySQL manual[1] says:
“The CHECK clause is parsed but ignored by all storage engines.”
They say it's best to use a Trigger to work around the problem. Luckily, I have the opportunity for my work to attend a course in Postgres Plus Advanced Server Database Administration.

Included in the course was a certification track. So, since the fourth of March 2013, I can now officially call myself an "EnterpriseDB Certified Postgres Plus 9.0 Associate". Yay!!!

I've made some notes during the course, and this here are some excerpts from them I feel were important enough to share.


EnterpriseDB [3] provides a version of PostgreSQL, called Postgres Plus Advanced Server[2], that is very similar to Oracle Database Server in a lot of ways.

The EnterpriseDB version requires a license, but you can download the trialversion to use for free for a couple of months. Downloading it is only possible after registering on their website.

In fact, the PostgreSQL Server will attempt to contact your profile info via the internet upon startup, but if there is no internet connection it starts up regardless.

Support for

Postgres Plus 9.0 has support for the following features:
PostgreSQL supports most of the major features of de SQL:2011 standard (ISO/IEC 9075:2011 "Database Language SQL").[5]
there is a schema search path per database/per user, by default set to ['$user',public], that indicates in what preference schemas are checked when a tablename is not prefixed with the schema
make it possible within a transaction to "save" part of the transaction, so the entire transaction need not be rollbacked upon failure, but can be rollbacked to the savepoint.
write ahead logging
log first, *then* the database
vertical table partitioning
divides different columns of one table across tables. Oracle syntax works, but can allow many more partitions
horizontal table partitioning
also called database sharding. Different rows of a table are stored in different tables. It's kind of a view of a couple of tables unioned together.
tables on different devices for concurrent io.
caching of everything in memory
materialized views
a view that is actually stored as a table, which provides the advantage of having indexes and being quicker, and the disadvantage of being possibly out of date
created aliases for database objects (tables, procedures, etc)
hierarchical queries
returns a resultset which is a flattened tree, based on a parent-child relationship defined in a CONNECT BY. It is possible to create queries that work on the node-level by using LEVEL (for example for indentation). ORDER SIBLINGS makes it possible to order all siblings under a common parent as you'd like.
database links
database links provide links in the current database to tables in other (remote) databases that otherwise could not be queried.
optimizer hints
hints provided to update/insert/delete/select statements for influencing which query plan the optimizer will choose. The APPEND keyword for this is interesting. It forces an insert to be appended at the end of the table, not at the location of a (couple of) vacuumed record(s). This is handy for efficient bulkloading of data.
send multiple requests of the database all at once, instead of each request individually. Minimizes network round trips.
lets you embed functions that perform actions within a regular SELECT query. A common use is with "SELECT pg_reload_conf();" which reloads the configuration.
define tables as inherited from other tables


DataBase Management System
free space map
Host Based Access
Multi-Statement Request
Multi Version Concurrency Control
Online analytical processing
Online transaction processing
QUEL [4]
old standard query language existing alongside SQL
Relational DBMS
"Shoot The Other Node In The Head", just to make sure that the master, which went down, stays down after the slave takes over.
tuple id, identification of a row in the table (a row in a table is called a tuple in Postgres)
visibility map
Write Ahead Logging
transaction log


[1] MySQL manual 5.6 - CREATE TABLE
[2] PostgreSQL
[3] EnterpriseDB/
[4] QUEL
[5] Appendix D. SQL Conformance

Wednesday, 27 February 2013


Recently had to transform an Oracle SQL Query into a Standard SQL Query. One of the three steps to take was changing the Oracle notation for outer joins into a "LEFT OUTER JOIN" and "RIGHT OUTER JOIN".

(+) is an Oracle specific function[1].

Let's go with the old and trivial example of the employees and the departments in a company, before we dive any deeper. It seems to work for everyone else, so it should work here.

As we see if the (+) is on the right of the expression, it is a "LEFT OUTER JOIN" and vice versa.

The example comes straight from [3]. It will show NULL for the department values for all employees that are not assigned to a department. Wikipedia has a nice article on the behaviour of outer joins.[4]

Multiple Outer Join

This is an example of how you could use left join multiple times.
In my work, what I usually find, is a query that selects from a master table that needs to provide all records, left joined with the rest of the world, just in case the records in other tables do not exist.

This won't work

You can write conditions in the "ON" clause, however, these conditions must be related to the SLAVE table. The following, for example, will not work.[2]

It will show all employees, even the ones that are no longer working for the company.

The solution here is to move the offending condition into the WHERE clause.

Odd one out

The example below, I encountered, is a bit odd, as it is not really an outer join between tables. It is used here to prevent having to write the 'or is null' part.


[1] Oracle (+)
[2] The ON condition
[3] Example Oracle Left Outer Join
[4] Wikipedia JOIN SQL
Outer joins in Oracle
A Visual Explanation of SQL Joins