Sunday, 8 December 2013

JSR 305: Annotations for Software Defect Detection

The NullPointerException is the most prevalent exception thrown in Java programs known to man. Dealing with it has always caused a great deal of Null checks in my (and other peoples) code.

At work we've recently started using the special annotations from JSR 305[1].

There are basically two. Nullable to indicate that a result of variable can be Null, and Nonnull to indicate that a result or variable will never be null.

The JSR 305 annotations are primarily intended for compilers or other tools that read source code. However, they are usually retained in the byte code for the benefit of static analysis tools such as FindBugs that inspect byte code rather than source.

An added benefit is that it forces me to think about what a method should return.

Eclipse, our work environment, has its own versions of those annotations, which we do not use.
  • org.eclipse.jdt.annotation.Nullable
  • org.eclipse.jdt.annotation.NonNull
  • org.eclipse.jdt.annotation.NonNullByDefault

The originals from JSR 305[1] are:
  • javax.annotation.Nullable
  • javax.annotation.Nonnull
  • javax.annotation.ParametersAreNonnullByDefault

In order to get the proper annotations, we do have to include a jar file. We use "jsr305-2.0.1.jar".

At the start of using these annotations, I've turned "Errors" off, regarding the Null annotations, as I was getting far too many errors.

The Executive Committee voted to list this JSR as dormant in May 2012. But the implementation of JSR 305 [1] seems to work in several IDEs.

Eclipse problems and workarounds

The tooling in Eclipse is of course not perfect in detecting where a Null is a possible problem. There are ways to help Eclipse make the right decision. In the examples below, the part of the code that Eclipse will indicate as being faulty is red underlined.
  1. The following code shows up Faulty in Eclipse. Eclipse/FindBugs cannot determine that the check for null negates the second possible NullPointer.
    // Potential null pointer access: The method getItem() may return null
    if (order.getItem() == null
    {
        logger.debug("order has no item.");
        continue;
    }
    int price = compute(order.getItem());
    The workaround is as easy as making a temporary local variable.
    // no problem!
    Item item = order.getItem();
    if (item == null
    {
        logger.debug("order has no more items.");
        continue;
    }
    int price = compute(item);
  2. Warning in the second line. Eclipse is not clever enough to figure out that assertNotNull will take care of it. Eclipse is smart, but not psychic.
    assertNotNull(order.getItem());
    assertNotNull(order.getItem().getDescription());
    Replace it with another local variable and put some supressors on it.
    assertNotNull(order.getItem());
    @SuppressWarnings("null")
    @Nonnull
    Item item = order.getItem();
    assertNotNull(item.getDescription());
  3. NullChecks are not being used in the original Java API, causing potential warnings. In the example, without the SuppressWarnings on the method, the last return would be underlined.
    @SuppressWarnings("null")
    public @Nonnull Set<Item> getItems() 
    {
        if (items == null
        {
            return Collections.emptySet();
        }
        return Collections.unmodifiableSet(items);
    }
  4. I've encountered an issue with Enums. Why does this happen? I have no clue.
       boolean result = hasProperty(name, StateEnum.MANDATORY);
       return result;
    }

    private boolean hasProperty(@Nullable String name, @Nonnull StateEnum state)
    {
       ....
    I can solve it in the obvious way, but am unable to find the cause of the problem.
       @SuppressWarnings("null")
       boolean result = hasProperty(name, StateEnum.MANDATORY);
       return result;
    }

    private boolean hasProperty(@Nullable String name, @Nonnull StateEnum state)
    {
       ....
  5. If the Null constraint annotations are not used in the Interface (specifically method parameters... it seems method return values don't matter), you are not allowed to add them in your implementation. You will receive the error message from eclipse.
    Multiple markers at this line
    - Illegal redefinition of parameter myParam, inherited method from SomeInterface does not constrain this parameter
    - implements mrbear.some.SomeInterfaceImpl.myMethod
  6. You are better off not using @Nonnull at the Hibernate Entity Level, as that is going to cause problems in some cases. Notable when using "Examples" for searching.

Note: Of course, it is a code smell to have to change your production code, to help Eclipse tooling deal with Null. This is probably the main reason, why it doesn't have the expected uptake in my company.

Mushroom cloud


Changing your existing code base to make use of this new check, is quite a job. If you start at a specific point (the point, to be more specific, where you were changing your code initially), it tends to mushroom cloud outwards. Somewhere along the line, you have to decide how far you wish to take this.

It would have been a great deal simpler, to have added this functionality straight from the beginning. Alas, hindsight is always 20-20.

Frameworks


I've not discovered any frameworks that inherently are using the same JSR305, except for one. Google Guava.

Generics


Recently had the issue that I had a List, of which I knew that each member of that List was not NULL. But I had no way of annotating this with @Nonnull.

It appears it is possible to (among other things) annotate generics once JSR 308[3] is added to Java. It is scheduled for Java 8 for now.[4]

The Checker Framework[2] makes it possible right now, apparently.

The syntax would look something like this:
protected @Nonnull List<@Nonnull String> getNames() {

Updated: 10/12/2013 added chapter on Generics and a problem-case I encountered.

Updated: 20/02/2014 added a critical note at the end of the chapter of "Eclipse problems and workarounds".

References

[1] JSR 305: Annotations for Software Defect Detection
http://jcp.org/en/jsr/detail?id=305
Avoiding “!= null” statements in Java?
http://stackoverflow.com/questions/271526/avoiding-null-statements-in-java
The Open Road: javax.annotation.
https://today.java.net/pub/a/today/2008/09/11/jsr-305-annotations.html
[2] Checker Framework
http://types.cs.washington.edu/checker-framework/
[3] JSR 308: Annotations on Java Types
http://jcp.org/en/jsr/detail?id=308
[4] JSR 337: JavaTM SE 8 Release Contents
http://jcp.org/en/jsr/detail?id=337

Sunday, 1 December 2013

Oracle Timestamp vs. Date

Recently have been hitting a brick wall due to a bug at work.

Query doesn't return any results if the date is "2009-06-01 00:00:00:01". But for the date "2009-06-01 00:00:00:00" it works fine.

The table is as follows:

And contains:
ID CODE STARTDATE ENDDATE
103581900/01/01 00:00:00:000000(null)

/* Formatted on 28/11/2013 17:02:58 (QP5 v5.227.12220.39754) */
SELECT *
FROM MUNICIPAL municipal1_  
WHERE municipal1_.code='0358'
AND (TO_TIMESTAMP('2009/06/01:00:00:01:01''yyyy/mm/dd:hh24:mi:ss:FF2'BETWEEN municipal1_.startdate
AND NVL (municipal1_.enddate - 1, TO_TIMESTAMP('2009/06/01:00:00:01:01''yyyy/mm/dd:hh24:mi:ss:FF2')))
The query above returns no rows. Why?

Some notes:
  1. the (null) for enddate means the timeperiod reaches into the future
  2. the sql function BETWEEN[5] checks to see if a value is within a certain range. The boundaries are inclusive.
  3. The enddate is a timestamp(6)
  4. enddate - 1 is no longer a timestamp, but a common date. An implicit datatype conversion takes place.[4]
  5. date doesn't store anything smaller than seconds [2]
  6. we lose the milliseconds
  7. NVL tries, if the first argument is null, to cast the second argument to the same type as the first argument (a date) [3]
  8. even though enddate is null, the data conversion still takes place
  9. ergo, the timestamp used now falls outside the range, and no results are returned.

Solution

There is one good solution to this. Don't use BETWEEN with timestamps if the range is exclusive the enddate. Use greater than/equals (>=) and lesser than (<) instead.

If you do wish to change a TIMESTAMP, use INTERVAL instead of arithmetic. [4]

/* Formatted on 28/11/2013 17:02:58 (QP5 v5.227.12220.39754) */
SELECT *
FROM MUNICIPAL municipal1_  
WHERE municipal1_.code='0358'
AND (TO_TIMESTAMP('2009/06/01:00:00:01:01''yyyy/mm/dd:hh24:mi:ss:FF2') >= municipal1_.startdate
AND TO_TIMESTAMP('2009/06/01:00:00:01:01''yyyy/mm/dd:hh24:mi:ss:FF2') < nvl(municipal1_.enddate, 
                            TO_TIMESTAMP('2009/06/01:00:00:01:01''yyyy/mm/dd:hh24:mi:ss:FF2')))

Conclusion


Do NOT use Arithmetic with Timestamps! You will get an automatic conversion to datatype Date and lose millisecond precision!

It is quite ugly to substract a day from a DATE, just so that you could use BETWEEN and not worry about the last boundary being included.

I guess it is my own fault for appropriating a bit of SQL code that was originally designed for DATES instead.

References

[1] Timestamp
http://docs.oracle.com/javase/7/docs/api/java/sql/Timestamp.html
[2] PL/SQL Data Type
http://docs.oracle.com/cd/B28359_01/appdev.111/b28370/datatypes.htm#CIHBCJEG
[3] NVL
http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions105.htm
[4] Math with Timestamp
https://blogs.oracle.com/knutvatsendvik/entry/math_with_timestamp
[5] BETWEEN
http://docs.oracle.com/cd/B28359_01/server.111/b28286/conditions011.htm#SQLRF52147