Creating a method reference on a null reference does not throw NullPointerException

We ran into a problem that the Unit tests ran perfectly on my local machine, but the same Unit tests would break in the continuous delivery pipeline.

The problem occurred when creating a method reference. Like so:

When running the test included above, the test failed with:

java.lang.AssertionError: Expected exception: java.lang.NullPointerException

After some research we found out that the difference between the two is the Eclipse compiler used by the IntelliJ IDE vs. the openjdk installed in the continuous delivery pipeline.

We found out that it is illegal to use a method reference on a null reference. Quoting from [1]:

First, if the method reference expression begins with an ExpressionName or a Primary, this subexpression is evaluated. If the subexpression evaluates to null, a NullPointerException is raised, and the method reference expression completes abruptly.

At first glance, this is a bit weird. After all, we want to have a method reference, in order to call it at a later time, which in fact may never occur. So why not have the NullPointerException when an attempt is made to actually call the method?

IntelliJ

IntelliJ comes equipped automatically with the Eclipse Java Compiler (ECJ), and as such it takes some effort to find out which version is installed along with the IDE.

Jar file ecj-4.6.1.jar was include in directory .local/share/JetBrains/Toolbox/apps/IDEA-U/ch-1/181.4445.78/lib/ecj-4.6.1.jar.

In IntelliJ it is possible to provide the path to the ecj jar to use, see [2].

Setting the path to the newly downloaded jar file ~/Downloads/ecj-4.7.3a.jar solved my problem.

It's been a long time since I encountered a bug in the compiler³, but other people have noticed it too⁴ and then it gets fixed.

Note

Bear in mind that, if we did not use a method reference, but an ordinary lambda, that this problem would not have occurred (immediately).

This means that, if you replace a Lambda with a method reference you may be introducing a NullPointerException earlier in the code without realising it⁵.

Bear this in mind.

References

[1] JLS 8 - 15.13.3. Run-Time Evaluation of Method References

https://docs.oracle.com/javase/specs/jls/se8/html/jls-15.html#jls-15.13.3

[2] IntelliJ - Specifying compilation settings

https://www.jetbrains.com/help/idea/specifying-compilation-settings.html

[3] Eclipse - Bug Report

https://bugs.eclipse.org/bugs/show_bug.cgi?id=521182

[4] StackOverflow - Creating a method reference on a null reference does not throw an exception

https://stackoverflow.com/questions/37681625/creating-a-method-reference-on-a-null-reference-does-not-throw-an-exception

[5] StackOverflow - java.lang.NullPointerException is thrown using a method reference but not a lambda

https://stackoverflow.com/questions/37413106/java-lang-nullpointerexception-is-thrown-using-a-method-reference-but-not-a-lamb/37413546

Hibernate issue

Recently encountered a small bug at work, related to Hibernate 3.3.2/3.4.0. It's already solved in the latest versions, but I found out after I spend some time debugging and since nobody at work generally has time/effort to upgrade libraries to the new version...

In other words, perhaps this will help someone out there.

The following java code contained the offending hql statement. It's an insert into select with a subselect in it. Not entirely trivial, in other words. Especially as JPA does not support this sort of statement.

public static final String BULKINSERT = 
                "INSERT INTO Settings " + 
                "(period, section, method) " + 
                "SELECT tp, sc, 'DUM' " + 
                "FROM SectionCode sc, TimePeriod tp " +
                "WHERE sc.sectionId = :sectionId " +
                "AND sc.deleted = false " +
                "AND tp.periodnr = :period " +
                "AND NOT EXISTS (" +
                "    SELECT '' " +
                "    FROM Settings ai " +
                "    WHERE ai.period.periodnr = :period " +
                "    AND ai.section = sc)";

Turning on Hibernate logging in jboss:

<category name="org.hibernate"> 
    <priority value="TRACE"/>  
</category>

I noticed it was translated by Hibernate into the following query.

14:02:21,934 TRACE [QueryPlanCache] located HQL query plan in cache (INSERT INTO Settings (period, section, method) SELECT tp, sc, 'DUM' FROM SectionCode sc, TimePeriod tp WHERE sc.sectionId = :sectionId AND sc.deleted = false AND tp.periodnr = :period AND NOT EXISTS (    SELECT ''     FROM Settings ai     WHERE ai.period.periodnr = :period     AND ai.section = sc))
14:02:21,934 TRACE [HQLQueryPlan] executeUpdate: INSERT INTO Settings (period, section, method) SELECT tp, sc, 'DUM' FROM SectionCode sc, TimePeriod tp WHERE sc.sectionId = :sectionId AND sc.deleted = false AND tp.periodnr = :period AND NOT EXISTS (    SELECT ''     FROM Settings ai     WHERE ai.period.periodnr = :period     AND ai.section = sc)
14:02:21,948 TRACE [QueryParameters] named parameters: {sectionId=W, period=6}
14:02:21,953 DEBUG [AbstractBatcher] about to open PreparedStatement (open PreparedStatements: 0, globally: 0)
14:02:21,953 DEBUG [ConnectionManager] opening JDBC connection
14:02:22,232 DEBUG [SQL] insert into SETTINGS ( SETTINGID, TIMEPERIODID, SECTIONCODEID, METHOD) select SETTING_SEQUENCE.nextval, timeperiod1_.TIMEPERIODID AS col_0_0_, sectioncode0_.SECTIONCODEID AS col_1_0_, 'DUM' AS col_2_0_ FROM SECTIONCODE sectioncode0_, TIMEPERIOD timeperiod1_ WHERE sectioncode0_.SECTIONCODEID = ? AND sectioncode0_.DELETED = 'N' AND timeperiod1_.TIMEPERIODID = ? AND NOT (EXISTS (SELECT '' FROM SETTINGS setting2_ WHERE setting2_.TIMEPERIODID = ? AND setting2_.SECTIONCODEID = SECTIONCODEID

insert into SETTINGS ( SETTINGID, TIMEPERIODID, SECTIONCODEID, METHOD) 
select SETTING_SEQUENCE.nextval, 
timeperiod1_.TIMEPERIODID AS col_0_0_,
       sectioncode0_.SECTIONCODEID AS col_1_0_,
       'DUM' AS col_2_0_
  FROM SECTIONCODE sectioncode0_, TIMEPERIOD timeperiod1_
 WHERE     sectioncode0_.SECTIONCODEID = 'W'
       AND sectioncode0_.DELETED = 'N'
       AND timeperiod1_.TIMEPERIODID = 6
       AND NOT (EXISTS
                   (SELECT ''
                      FROM SETTINGS setting2_
                     WHERE setting2_.TIMEPERIODID = 6
                           AND setting2_.SECTIONCODEID =
                                  SECTIONCODEID));

It's hard to tell at first, but in the output above, in the subselect, it says "AND setting2_.SECTIONCODEID = SECTIONCODEID". It should be "AND setting2_.SECTIONCODEID = sectioncode0_.SECTIONCODEID". The current implementation causes the second SECTIONCODEID to automatically refer back to settings2_ table, causing the expression to always evaluate to true. This explained why I never saw any records being added in certain cases.

In order to get the query to work properly, I was forced to reach out to native SQL instead of HQL. Upgrading our software to the latest and greatest Hibernate is a tad too involved for now.

References

HHH-5274 - HQL-Insert with Select and Sub-Select fails

https://hibernate.onjira.com/browse/HHH-5274