Thursday, 29 October 2015

Making Mistakes In JPQL

Well, crap. My boss mentioned that I introduced a Bug in my JPQL, whilst fixing a Bug.

I think it's a Bug that is easy to create, and is therefore worth a little blogpost.

In Java the || and && operators are short-circuiting operators. This means the expression on the right of the operator will not be evaluated if the expression on the left already makes the entire expression true1.

When using Hibernate (or any other ORM) I do tend to program JPQL in the same way as I program in Java. As this is translated into JPQL, the translation is sometimes not what you would expect.

The class diagram looked something like the following. Bear in mind that each Class represents a table in the database, as is usual in an ORM.

The problem

The following JPQL had an issue:
String queryStr = "SELECT person " +
                  "FROM Person person " +
                  "JOIN FETCH person.function " +
                  "WHERE person.lastname = :lastname " +
                  "AND person.fired = false " +
                  "AND (person.address is null OR person.address.deleted = false) ";
The idea here is to provide us with a list of all Persons with a specific last name, who are both not fired and may or may not have an existing address.

It will be translated by Hibernate (in our case) into the following SQL statement:
select * -- a lot of fields here
from Person person0_ inner join Function function1_ on person0_.FUNCTIONID=function1_.FUNCTIONID, Address address2_ 
where person0_.ADDRESSID=address2_.ADDRESSID 
and person0_.LASTNAME=? 
and person0_.FIRED='N' 
and (person0_.ADDRESSID is null or address2_.DELETED='N')
Can you spot the problem?

The Solution

The problem above, apparently, is that besides the explicit inner join (inner join Function) there is an implicit inner join upon the Address table (from Address address2_ where person0_.ADDRESSID=address2_.ADDRESSID). This means that only persons will be shown that have an existing address. The condition in the where at the bottom regarding "person0_.ADDRESSID is null" is ignored.

The following, slightly more complicated, JPQL solves this problem:
String queryStr = "SELECT person " +
                  "FROM Person person " +
                  "JOIN FETCH person.function " +
                  "LEFT JOIN person.address address " +
                  "WHERE person.lastname = :lastname " +
                  "AND person.fired = false " +
                  "AND (address is null OR address.deleted = false) ";
This will be translated appropriately by Hibernate into the following SQL Query:
SELECT * -- a lot of fields here
FROM Person person0_
      ON person0_.FUNCTIONID=function1_.FUNCTIONID
   LEFT OUTER JOIN Address address2_
      ON person0_.ADDRESSID     =address2_.ADDRESSID
WHERE person0_.LASTNAME   =?
AND person0_.FIRED        ='N'
AND (address2_.ADDRESSID IS NULL OR address2_.DELETED      ='N')


Of course, I would not have made such a crappy mistake, if I did my tests properly. Apparently, testing is hard, and it is easy to miss to test all eventualities.

In this case it is obvious I forgot to test for the Address==null case.

Well, we live and learn.


The JavaTM Tutorials - Equality, Relational, and Conditional Operators
StackOverflow - Explicit vs Implicit SQL Joins

Thursday, 22 October 2015

Running Glassfish on port 80

Well, in short. Linux does not allow non-root programs to listen/occupy ports below 1024. It is a security feature.

So what can I do to make a server that is non-root (because of security) able to receive/send messages through a privileged port? Luckily the question has already been asked multiple times on the Internet.

To summarize:
  • Run the program as root anyway
  • Have a root daemon run the program as root, then change the rootrights automatically to something less
  • Kernel param CAP_NET_BIND_SERVICE
  • Run a proxy server in front of it
  • Software that performs what you want
  • Port-forwarding using IPTables

For the matter of this blog, I am going to assume that I wish to make my Glassfish instance available via both port 80 (the default web port) and port 443 (the default ssl web port). For that is the use case I was faced with.

Run the program as root anyway

Well, that should work, but is a huge security risk in most cases.

We would just need to change the ports of the HTTP Listeners in Glassfish administration console. It can be found in the default-config option.

Have a root daemon run the program as root, then change the rootrights automatically to something less

This seems to be one of the functions of the inetd daemon.


sudo setcap cap_net_bind_service=ep some-binary
In Linux, the things root can do have been broken up into a set of capabilities. CAP_NET_BIND_SERVICE is the ability to bind to ports <= 1024.

Unfortunately, we would need to set the binary to "java". This automatically means that any java program will have the ability to access any ports. This is not what we want.

Run a proxy server in front of it

That's what I did initially. There are lots of web servers that are perfectly capable of redirecting traffic to another port. For example the Apache HTTP server and Nginx. I choose Nginx, as it seems to be quite easy to setup and doesn't come with a lot of extras, extras I do not need.

The excellent part of this solution is that we can use Nginx to serve static webpages/images/files and not bother the Glassfish with it.

Tricky is that I need the original originating IP address of the client, and that is lost without some magic configuration.

The bad part is that a proxy server is another link in the chain that needs to be supported and maintained etc.

Software that performs what you want

For example authbind seems mentioned a lot.

Port-forwarding using IPTables

This is the solution I went for eventually.
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080
iptables -t nat -A PREROUTING -p udp -m udp --dport 80 -j REDIRECT --to-ports 8080 

# Do the same thing for ssl:
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 8181
iptables -t nat -A PREROUTING -p udp -m udp --dport 443 -j REDIRECT --to-ports 8181
Removing all the rules can be done with:
echo "Stopping firewall and allowing everyone..."

iptables -F

iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
To save the configuration, you can use iptables-save and iptables-restore. Or try something like:
]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ]


StackOverflow - Is there a way for non-root processes to bind to “privileged” ports (<1024) on Linux?
Debian Administration - Running network services as a non-root user
Ubuntu Help - Iptables HowTo
Oracle Blogs - Byron's Pointless Ramblings - Run GlassFish V3 as a non-root Service on Linux Ubuntu/Debian
The JavaDude Weblog - Glassfish and https running secure applications

Thursday, 15 October 2015

SSL Certificates in Glassfish

There are two files in glassfish1, to wit:
  • ./glassfish/domains/domain1/config/keystore.jks
  • ./glassfish/domains/domain1/config/cacerts.jks
The way I read it, it means your private keys are stored in the keystore.jks, and the root certificates and intermediate certificates of Certificate Authorities (CA) are stored in cacerts.jks. When configured correctly, these two files should contain all the certificates needed to create a necessary chain of trust.

Checking out the keystore, can be done using the following commandline:
keytool -list -v -keystore keystore.jks
The default keystore password is "changeit".
You get the following:
Enter keystore password:
Keystore type: JKS
Keystore provider: SUN

Your keystore contains 2 entries

Alias name: glassfish-instance
Creation date: May 15, 2013
Entry type: PrivateKeyEntry
Certificate chain length: 1
Owner: CN=localhost-instance, OU=GlassFish, O=Oracle Corporation, L=Santa Clara, ST=California, C=US
Issuer: CN=localhost-instance, OU=GlassFish, O=Oracle Corporation, L=Santa Clara, ST=California, C=US
Serial number: 43ce5f77
Valid from: Wed May 15 07:33:41 CEST 2013 until: Sat May 13 07:33:41 CEST 2023
Certificate fingerprints:
MD5: C0:FA:88:64:36:7A:1B:62:1B:F1:BD:8F:5A:7A:9A:E7
SHA1: B1:FA:A8:2B:7C:83:18:A8:9B:C6:46:50:41:EC:FC:7C:DF:69:B3:33
SHA256: 52:AB:1F:37:75:68:92:8F:3D:02:49:D7:3C:8E:BC:53:76:9B:68:E2:B8:83:AF:ED:4C:39:99:FE:45:F1:F1:67
Signature algorithm name: SHA256withRSA
Version: 3


#1: ObjectId: Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 56 50 2C 8F D9 A2 55 80 18 8F 3D 90 AC 77 28 C3 VP,...U...=..w(.
0010: FE A0 55 F6 ..U.
Alias name: s1as
Creation date: May 15, 2013
Entry type: PrivateKeyEntry
Certificate chain length: 1
Owner: CN=localhost, OU=GlassFish, O=Oracle Corporation, L=Santa Clara, ST=California, C=US
Issuer: CN=localhost, OU=GlassFish, O=Oracle Corporation, L=Santa Clara, ST=California, C=US
Serial number: 4a9972f
Valid from: Wed May 15 07:33:38 CEST 2013 until: Sat May 13 07:33:38 CEST 2023
Certificate fingerprints:
MD5: 79:0D:FC:CF:99:32:2B:BE:77:36:40:4A:14:E1:2D:91
SHA1: 4A:57:58:F5:92:79:E8:2F:2A:91:3C:83:CA:65:8D:69:64:57:5A:72
SHA256: AB:48:B2:E6:C4:4C:50:86:7F:B3:70:30:83:F1:CE:E8:06:F4:B5:75:F0:E3:AD:5B:23:38:10:02:A8:85:F5:56
Signature algorithm name: SHA256withRSA
Version: 3


#1: ObjectId: Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 4C 05 82 BD 8C 02 B8 05 00 04 14 0A FB 29 AA F7 L............)..
0010: 48 6C CB 86 Hl..

There's also a keystore that comes bundled with your java installation, usually it can be found somewhere in the security directory.
You can view all the certificates in there using:
keytool -list -keystore /usr/lib/jvm/java-1.8.0-openjdk-

Backup your keystore.jks

Just a simple copy will do.
cp keystore.jks keystore.jks.backup

Delete the default self-signed certificate

keytool -delete -alias s1as -keystore keystore.jks -storepass

Generating a certificate request

This request we are about to generate, is submitted to the Certificate Signing Authority. See for more information chapter "To Sign a Certificate by Using keytool" in [1]. Using RSA, gives a default keysize of 2048.
[glassfish@vps386 config]$ keytool -genkeypair -keyalg RSA -keystore keystore.jks -validity 365 -alias s1as
Enter keystore password:
What is your first and last name?
What is the name of your organizational unit?
What is the name of your organization?
What is the name of your City or Locality?
What is the name of your State or Province?
What is the two-letter country code for this unit?
Is, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown correct?
[no]: yes

Enter key password for
(RETURN if same as keystore password):
[glassfish@vps386 config]$
The CSR (Certificate Signing Request) can then be generated into the file s1as.csr:
keytool -certreq -alias s1as -file s1as.csr -keystore keystore.jks -storepass changeit
Viewing the generated file should look something like the following:

Submit the CSR to a Certificate Authority

I did this using GoDaddy.Com. Most CAs have a web interface that provides this functionality.

Download the CA certificates and any intermediate CA certificates

This is where it gets a little problematic. I keep hearing that a lot of CAs do not provide all or the proper certificates in the download, and you are forced to check out their public repository for the right files.

Then there are all the possible formats in which the certificates can be stored.

Downloading the zip file from, gave me the following files:
the root and intermediate certificates of your CA
your private certificate that should be kept private

Check out the CA certificates

keytool -printcert -v -file gd_bundle-g2-g1.crt

Import the CA certificate and any intermediate CA certificates

keytool -import -v -trustcacerts -alias godaddy -file /home/glassfish/junk/gd_bundle-g2-g1.crt -keystore cacerts.jks -storepass changeit
Certificate was added to keystore
[Storing cacerts.jks]

Replace the original self-signed certificate with the certificate you obtained from the CA

keytool -import -v -trustcacerts -alias s1as -file /home/glassfish/junk/b9683876305fc322.crt -keystore keystore.jks -storepass changeit

Verify the certificate chain

Unfortunately, not providing all the required certificates, to build up the chain will cause an exception when adding your private key4.

openssl s_client -connect
Shows the following Certificate chain:
Certificate chain
0 s:/OU=Domain Control Validated/
i:/C=US/ST=Arizona/L=Scottsdale/, Inc./OU= Daddy Secure Certificate Authority - G2
1 s:/C=US/ST=Arizona/L=Scottsdale/, Inc./OU= Daddy Secure Certificate Authority - G2
i:/C=US/ST=Arizona/L=Scottsdale/, Inc./CN=Go Daddy Root Certificate Authority - G2
2 s:/C=US/ST=Arizona/L=Scottsdale/, Inc./CN=Go Daddy Root Certificate Authority - G2
i:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority
3 s:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority
i:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority
Another way to verify the chain, is by using Microsoft Windows:
In Linux, I find the Keystore-Explorer5 to fulfill my needs:


[1] GlassFish Server Open Source Edition Security Guide Release 4.0
[2] GlassFish Server Open Source Edition Administration Guide Release 4.0
[3] Java Dude Blog - Glassfish V3.1.2 and SSL
[4] StackOverflow - Keytool error java lang exception failed to establish chain from reply
[5] SourceForge - Keystore-Explorer

Wednesday, 7 October 2015

An Evening with Java Champion Martijn Verburg

I managed to attend a lecture of Martijn Verburg1 at the Supernova conference room in the Jaarbeurs in Utrecht yesterevening. Courtesy of Blue4IT.

Martijn Verburg, a.k.a. The Diabolical Developer, provided two interactive sessions. The first was "The Diabolical Developer's guide to Performance tuning" The second was "The Habits of Highly Effective Teams".

"The Diabolical Developer's guide to Performance tuning"

Martijn Verburg is the CEO of JClarity1, a company that does Performance analysis of Cloud applications. JClarity introduced the “Performance Diagnostic Methodology” (PDM) to quickly get to the root of the problem.

The points to aid in analysing performance issues were in the following order:
  1. What is your Cloud made of?
  2. Draw diagrams
  3. Measure each Architectural Layer
  4. What is the CPU doing? Overworked or Bored?
  5. Linux commandline tools are pure awesomeness
  6. Garbage collection logs
  7. Oracle micro benchmarks
  8. Visualvm, Netbeans
The talk had a focus sometimes on the Cloud. He seemed critical of it.

The Cloud, or virtualisation in general, seems to be 20 to 25% slower than the old situation of dedicated hardware. It seems a lot of the time virtualised resources are overcommitted on existing hardware. Apparently with virtualisation it is possible that your performance goes down because of "noisy neighbours", i.e. someone else gets a lot of traffic/cpu/stuff to deal with and the others on the same node suffer.

One of the things he noticed in the field, is that performance issues were seldom the cause of some badly programmed application. The issue was usually located a lot lower in the stack, somewhere between the hardware and the JVM.

When analysing performance, start at the bottom and work your way up. So, check the hardware and the Operating System. What is it doing?

Keep your methods small. Java is better able to optimise small methods during Runtime.


Some tools mentioned:
  • JMeter
  • Gatling
  • there seems to be a JDBC driver, that is nothing more than a wrapper around an existing JDBC driver to analyse the traffic
  • vmstat
  • jcviewer
  • visualvm
  • javap -c
  • Netbeans
  • Protobuf
  • Jitwatch

"The Habits of Highly Effective Teams"

I managed to find the slides at [3].

The points in order were:
  1. Social interaction
  2. Strong leadership
  3. Empowerment over Control
  4. Shared goals
  5. Respect and Trust
  6. Common culture
  7. Automation and tools
  8. Encourage debate
  9. Embrace diversity
He mentioned Dunbar's number, on average people can sustain about 150 relationships.

He mentioned the Peter principle, which was interesting as I had never heard of it.

He did mention that where a lot of managers say that you have to earn respect and trust, that simply does not and will never work. It's better to assume total respect and trust, until you are proven wrong by the person you respect and trust.

When debating a technological something, as we engineers often do, the only thing that has any value in a debate are hard facts and empirical data (and not opinions, assumptions, emotions and the like). If you have an inkling that some decision is wrong, prototype it and verify it. Martijn Verburg and his team use the Matt Raible method, where they draw up a Matrix of all the technologically important stuff and give points for a certain framework where it excels or falls short. Then they start prototyping with the three frameworks with the highest scores.

Make the shared goals SMART (Specific, Measurable, Attainable, Relevant, Time-bound).


Martijn Verburg seems highly quotable. Here are some that stuck in my mind.
“Virtualisation is ruining it for everyone.”
“Seagull managers fly in, make a lot of noise, dump on everyone, then fly out.”
“Having to use swap when running out of memory, is like getting your beer from Pluto instead of the other side of the pub.”
“Don't optimize. Java is way smarter than you are.”
“Lead, follow or get out of the way.”
P.S. the picture at the top of the article is the ceiling of the conference room in the Jaarbeurs in Utrecht. They have some very interesting architecture. Seemed like we were sitting right under a rocketengine.


[1] Blue4IT - Kennisavond met Java Champion Martijn Verburg
[2] JClarity
[3] Slideshare Slides - Habits of Highly Effective Teams

Thursday, 1 October 2015

Calling super constructors

So, a constructor in a subclass, if it needs to call the constructor in the super class, is required to do this in the first line of the body of the constructor1.

If it does not, Java will insert a call to the default super constructor. If it does not exist, you get a compile time error.

So, what if you need to do things, before the super call takes place?

As the object is in the process of being constructed (that's why it's called a constructor), the object is in a non-valid state. This likely explains why the language designers felt that a call to another constructor should be the first part of the body of a constructor.

Well, apparently the only solution is to do everything you need inside the expressions that take the place of the super constructor arguments.

I was forced to do just that in my MudWebException.

So, there's an easy workaround, but it feels clumsy.

Take care not to access any of the methods or properties of the object that is being constructed, as it is in a non-valid state. As a matter of fact, I think it's a really bad idea to call methods in the object from within your constructor.


[1] JLS 8.0 Section 8.8.7. Constructor Body
StackOverflow - Why does this and super have to be the first statement in a constructor?