Showing posts with label jdbc. Show all posts
Showing posts with label jdbc. Show all posts

Thursday, 27 March 2025

Using different JDBC drivers

So I had the problem that I needed to connect to both Oracle databases as well as Postgres database.

And just including the dependencies in my Maven was not enough.

One of the drivers gets overwritten by another driver in the service provider mechanism.

At first I tried to register them by hand (which works):

DriverManager.registerDriver(new org.postgresql.Driver());
DriverManager.registerDriver(new OracleDriver()); return DriverManager.getConnection(url, user, password);

Or, you could do the extreme magic thing1 2:

Class.forName("org.postgresql.Driver");
Class.forName("oracle.jdbc.driver.OracleDriver"); return DriverManager.getConnection(url, user, password);

Which causes both drivers to run a static block registering themselves on the DriverManager in the same fashion as my code on top.

All this, because the service provider mechanism that automatically loads the JDBC driver, only allows one Driver to be registered3.

References

[1] StackOverflow - What is the difference between "Class.forName()" and "Class.forName().newInstance()"?
https://stackoverflow.com/questions/2092659/what-is-the-difference-between-class-forname-and-class-forname-newinstanc/2093236#2093236
[2] Baeldung - Loading JDBC Drivers
https://www.baeldung.com/java-jdbc-loading-drivers
[3] Baeldung - Java Service Provider Interface
https://www.baeldung.com/java-spi

Friday, 29 May 2020

Adding MariaDB JDBC Connector to Paraya 5 - Followup

Just decided to do a followup on Adding MariaDB JDBC Connector to Payara 51.

I wanted to see if it was possibly very easy to do this using the command line.

Installing a MariaDB driver

$ ./asadmin add-library /home/mrbear/software/mariadb-java-client-2.3.0.jar
Command add-library executed successfully.

Creating a new JDBC Connection pool

The syntax:

Usage: asadmin [asadmin-utility-options] create-jdbc-connection-pool
        [--datasourceclassname ] [--restype ]
        [--steadypoolsize ]
        [--maxpoolsize ]
        [--maxwait ]
        [--poolresize ]
        [--idletimeout ] [--initsql ]
        [--isolationlevel ]
        [--isisolationguaranteed[=]]
        [--isconnectvalidatereq[=]]
        [--validationmethod ]
        [--validationtable ]
        [--failconnection[=]]
        [--allownoncomponentcallers[=]]
        [--nontransactionalconnections[=]]
        [--validateatmostonceperiod ]
        [--leaktimeout ]
        [--leakreclaim[=]]
        [--creationretryattempts ]
        [--creationretryinterval ]
        [--sqltracelisteners ]
        [--statementtimeout ]
        [--statementleaktimeout ]
        [--statementleakreclaim[=]]
        [--lazyconnectionenlistment[=]]
        [--lazyconnectionassociation[=]]
        [--associatewiththread[=]]
        [--driverclassname ]
        [--matchconnections[=]]
        [--maxconnectionusagecount ]
        [--ping[=]] [--pooling[=]]
        [--statementcachesize ]
        [--validationclassname ]
        [--wrapjdbcobjects[=]]
        [--description ] [--property ]
        [-?|--help[=]] jdbc_connection_pool_id

Note the following:

  • The escape character backslash (\) is used in the --property option to distinguish the semicolon (;).
  • Two backslashes (\\) are used to distinguish the equal sign (=).
  • Two backslashes (\\) are used to distinguish the colon (:).
./asadmin create-jdbc-connection-pool --driverclassname org.mariadb.jdbc.Driver --restype java.sql.Driver --property databaseName=mmud:password=topsecretpassword:user=root:URL=jdbc\\:mariadb\\://localhost\\:3306/mmud:serverName=localhost mmudpool

Let's test it!

./asadmin ping-connection-pool mmudpool

In case you wish to delete it, try:

./asadmin delete-jdbc-connection-pool mmudpool

Creating the JNDI

./asadmin create-jdbc-resource --connectionpoolid mmudpool jdbc/mmud

References

[1] Adding MariaDB JDBC Connector to Payara 5

https://randomthoughtsonjavaprogramming.blogspot.com/2018/12/adding-mariadb-jdbc-connector-to-paraya.html
[2] Payara Blog - An intro to connection pools in payar server 5
https://blog.payara.fish/an-intro-to-connection-pools-in-payara-server-5

Thursday, 27 December 2018

Adding MariaDB JDBC Connector to Paraya 5

It's as easy as pie.

  1. start Payara
    $ cd ~/payara5/bin
    $ ./asadmin start-domain
    Waiting for domain1 to start ...............
    Successfully started the domain : domain1
    domain Location: /home/mrbear/payara5/glassfish/domains/domain1
    Log File: /home/mrbear/payara5/glassfish/domains/domain1/logs/server.log
    Admin Port: 4848
    Command start-domain executed successfully.
  2. install the database driver
    $ ./asadmin add-library /home/mrbear/software/mariadb-java-client-2.3.0.jar
    Command add-library executed successfully.
    or try:
    $ cp ~/Downloads/mariadb-java-client-2.3.0.jar ~/payara5/glassfish/domains/domain1/lib

    The last part will probably require a payara application server reboot, though.

  3. connect to http://localhost:4848
  4. surf to Resources -> JDBC -> JDBC Connection Pools
  5. Create a new one
  6. Settings:
    Pool name[obvious]
    Resource Typejavax.sql.DataSource
    Datasource classnameorg.mariadb.jdbc.MariaDbDataSource

    Leave the rest as default.

  7. Define the following additional properties:
    NameValue
    serverNamelocalhost
    URLjdbc:mariadb://localhost:3306/mmud
    databaseNamemmud
    userusername
    passworditsasecret
  8. Try the "Ping" button to verify that the connection is established.
  9. surf to Resources -> JDBC -> JDBC Resources
  10. Create a new one
  11. Settings:
    NameValue
    JNDI Namethis is the name as the resource is defined in your application
    Pool Namethe pool defined previously, it's a dropdown which is convenient.
  12. Press finish or OK or something
  13. Done!

References

Using MySQL with Payara Server
https://blog.payara.fish/using-mysql-with-payara
Payara - Using Hibernate 5 on Payara Server
https://blog.payara.fish/using-hibernate-5-on-payara-server

Thursday, 6 December 2018

Adding MariaDB JDBC Connector to Wildfly 14

  1. install the database driver as a module
    1. create directory WILDFLY_HOME/modules/com/mariadb/main
      $ mkdir -p modules/com/mariadb/main
    2. copy database driver into that one
      $ cp ~/Downloads/mariadb-java-client-2.3.0.jar .
    3. Create the file module.xml in the same folder with the following content:
      <module xmlns="urn:jboss:module:1.3" name="com.mariadb">
          <resources>
              <resource-root path="mariadb-java-client-2.3.0.jar"/>
          </resources>
          <dependencies>
              <module name="javax.api"/>
              <module name="javax.transaction.api"/>
          </dependencies>
      </module>
  2. start Wildfly
    $ cd $WILDFLY_HOME/bin
    $ ./standalone.sh
  3. connect to http://localhost:9990/error/index.html
  4. run ./add-user.sh
    [mrbear@localhost bin]$ ./add-user.sh 

    What type of user do you wish to add? 
     a) Management User (mgmt-users.properties) 
     b) Application User (application-users.properties)
    (a): a

    Enter the details of the new user to add.
    Using realm 'ManagementRealm' as discovered from the existing property files.
    Username : mrbear
    Password recommendations are listed below. To modify these restrictions edit the add-user.properties configuration file.
     - The password should be different from the username
     - The password should not be one of the following restricted values {root, admin, administrator}
     - The password should contain at least 8 characters, 1 alphabetic character(s), 1 digit(s), 1 non-alphanumeric symbol(s)
    Password : 
    Re-enter Password :  
    What groups do you want this user to belong to? (Please enter a comma separated list, or leave blank for none)[  ]: 
    About to add user 'mrbear' for realm 'ManagementRealm'
    Is this correct yes/no? yes
    Added user 'mrbear' to file '/home/mrbear/wildfly-14.0.1.Final/standalone/configuration/mgmt-users.properties'
    Added user 'mrbear' to file '/home/mrbear/wildfly-14.0.1.Final/domain/configuration/mgmt-users.properties'
    Added user 'mrbear' with groups  to file '/home/mrbear/wildfly-14.0.1.Final/standalone/configuration/mgmt-groups.properties'
    Added user 'mrbear' with groups  to file '/home/mrbear/wildfly-14.0.1.Final/domain/configuration/mgmt-groups.properties'
    Is this new user going to be used for one AS process to connect to another AS process? 
    e.g. for a slave host controller connecting to the master or for a Remoting connection for server to server EJB calls.
    yes/no? no
  5. click on "Try again" on the Website
  6. log on with the new added user
  7. Configuration - Subsystems - Datasources and drivers - JDBC drivers - [+]
  8. Fill out the following information:
    Driver namemariadb
    Driver Module Namecom.mariadb
    Module Slot
    Driver Class Nameorg.mariadb.jdbc.Driver
    Drive Datasource Class Name
    Driver XA Datasource Class Name
  9. Configuration - Subsystems - Datasources and drivers - Datasources - [+]
  10. Select "Add Datasource" to add a non-XA Datasource
  11. Select "Custom" template
  12. Attributes:
    Name[name]
    JNDI Namejava:/jdbc/[name]
  13. JDBC Driver
    Driver Namemariadb
    Driver Module Name[empty]
    Driver Class Name[empty]
  14. Connection:
    Connection URLjdbc:mariadb://localhost:3306/[databasename]
    User nameroot
    Passworditsasecret
    Security Domain[empty]
  15. Test connection
  16. Reload of server required - so reload the server
  17. Test connection: Successfully tested connection for datasource [name].

References

JDBC Driver / DataSource Setup
https://hal.gitbooks.io/dev/content/recipes/jdbc-driver-setup.html
About MariaDB Connector/J
https://mariadb.com/kb/en/library/about-mariadb-connector-j/
GitHub - Use MariaDB Connector/J Driver
https://github.com/MariaDB/mariadb-connector-j/blob/master/documentation/use-mariadb-connector-j-driver.creole
IronJacamar 1.1 - Userguide - Deploying DS Descriptor
http://www.ironjacamar.org/doc/userguide/1.1/en-US/html_single/index.html#deployingds_descriptor

Thursday, 1 June 2017

flexibleJDBCRealm

I have recently changed my security realm settings, and I thought I'd document them here.

I'm still using the flexibleJDBCRealm1 as I've documented in previous blogs2,3.

In the Glassfish administration console, under Configurations -> server-config -> Security -> Realms -> myRealm, the settings are now as follows.
NameValueDescription
datasource.jndijdbc/mydbthe data source to my database
jaas.contextflexibleJdbcRealm
password.digestSHA-512I have upgraded from SHA1 to SHA2, which seems more secure
password.encodingHEX:128See note below
sql.groupsselect groupid from mmv_groups where name in (?)using a database view, makes it easier to change table layout without effecting the securityrealm
sql.passwordselect password from mmv_users where name in (?)same as above

Note

The SHA-512 encoding always creates 128 characters as the hash.

However, in the source code of the flexibleJDBCRealm, this hash is converted from a byte[] into a hexadecimal string by means of a call "new BigInteger(1, aData).toString(16);".

This effectively means that if the byte[] starts with one or more "0"s, these are removed in the BigInteger call leaving you with a hash that is less than 128 characters.

This is why I need to use "HEX:128", instead of just "HEX".

MariaDB/MySQL

The values are easily verifiable in the database.

I can just do a
SELECT SHA2(usertable.password, 512) from usertable where user='mrbear';

It should yield the exact same result as the encryption function of the flexibleJDBCRealm.

References

[1] FlexibleJDBCRealm
http://flexiblejdbcrealm.wamblee.org/site/
[2] Security Realms in Glassfish
http://randomthoughtsonjavaprogramming.blogspot.nl/2016/04/security-realms-in-glassfish.html
[3] Glassfish Security Realms
http://randomthoughtsonjavaprogramming.blogspot.nl/2014/10/glassfish-security-realms.html
[4] Installation instructions
http://flexiblejdbcrealm.wamblee.org/site/documentation/snapshot/installation.html

Thursday, 14 January 2016

Glassfish Logging to Database

I wanted to redirect the logging messages from the Glassfish Application Server to the database.

Glassfish uses the standard Logging API available in the JDK. There are some who find the already existing logging APIs (log4j, etc) available better than the default provided in the SDK. But I find it sufficient for my purposes.

The Logging API uses Handlers to indicate what needs to be done with all those log messages. The default handlers available in the API are shown in the diagram below.

I have opted for having the logrecords sent via the network (local interface) to a small daemon that posts the log records into a database table. So that would mean using the SocketHandler.

In general, this is what you want. Several application servers all sending their log messages to a central repository that can deal with it. Of course, there are way more sophisticated solutions readily available compared to this home-brewed thing. But it helps to get the general idea of how it works.

The software I made is available at https://github.com/maartenl/sql-logging. The sequence diagram below is basically how it works.

Since the data send over the network is in the XML format, I'm using StAX for interpreting the XML stream properly.

The funny thing is that the DatabaseHandler in the diagram above, is basically a child of the Handler class (depicted in the class diagram up top). So, what we have here is a kind of Logging proxy, really.

The database table follows the fields in the LogRecord class closely:
create table mm_systemlog (
  id bigint(20) NOT NULL AUTO_INCREMENT primary key,
  millis bigint(20) not null,
  sequence bigint(20) not null,
  logger varchar(255),
  level varchar(25),
  class varchar(255),
  method varchar(255),
  thread int(10),   
  message text not null,
  INDEX(millis)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Glassfish

Glassfish has a logging.properties file (glassfish4.1/glassfish/domains/domain1/config/logging.properties). It is possible to add or change the Log Handler used by Glassfish by changing it in the logging.properties file1 3.

I was unable to provide Glassfish with a Logging Handler that directly posted the LogRecords into a database (see [2]). Hence the current solution described above.

The (fairly easy) steps were as follows:
  • the developer has put the custom handler JAR file into the domain-dir/lib/ext directory.
  • the class that extends java.util.logging.Handler must be in the server classpath.
  • add the new handler to the handlers attribute in the glassfish/domains/domain1/config/logging.properties file
    handlers=java.util.logging.ConsoleHandler,com.mrbear.logging.handlers.SimpleSocketHandler
  • reboot glassfish

XML Formatter

Below is an example of what the XML format looks like.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE log SYSTEM "logger.dtd">
<log>
<record>
<date>2016-01-11T06:50:33</date>
<millis>1452491433234</millis>
<sequence>0</sequence>
<logger>com.mrbear.logging.tests.SimpleSocketHandlerTest</logger>
<level>SEVERE</level>
<class>com.mrbear.logging.tests.SimpleSocketHandlerTest</class>
<method>testSocketHandler</method>
<thread>1</thread>
<message>Hello, World</message>
</record>
<record>
<date>2016-01-11T06:50:33</date>
<millis>1452491433261</millis>
<sequence>1</sequence>
<logger>com.mrbear.logging.tests.SimpleSocketHandlerTest</logger>
<level>INFO</level>
<class>com.mrbear.logging.tests.SimpleSocketHandlerTest</class>
<method>testSocketHandler</method>
<thread>1</thread>
<message>Welcome Home</message>
</record>
<record>
<date>2016-01-11T06:50:33</date>
<millis>1452491433263</millis>
<sequence>2</sequence>
<logger>com.mrbear.logging.tests.SimpleSocketHandlerTest</logger>
<level>CONFIG</level>
<class>com.mrbear.logging.tests.SimpleSocketHandlerTest</class>
<method>testSocketHandler</method>
<thread>1</thread>
<message>Config ....</message>
</record>
<record>
<date>2016-01-11T06:50:33</date>
<millis>1452491433265</millis>
<sequence>3</sequence>
<logger>com.mrbear.logging.tests.SimpleSocketHandlerTest</logger>
<level>FINE</level>
<class>com.mrbear.logging.tests.SimpleSocketHandlerTest</class>
<method>testSocketHandler</method>
<thread>1</thread>
<message>Fine ....</message>
</record>
<record>
<date>2016-01-11T06:50:33</date>
<millis>1452491433266</millis>
<sequence>4</sequence>
<logger>com.mrbear.logging.tests.SimpleSocketHandlerTest</logger>
<level>FINEST</level>
<class>com.mrbear.logging.tests.SimpleSocketHandlerTest</class>
<method>testSocketHandler</method>
<thread>1</thread>
<message>Finest ....</message>
</record>
<record>
<date>2016-01-11T06:50:33</date>
<millis>1452491433266</millis>
<sequence>5</sequence>
<logger>com.mrbear.logging.tests.SimpleSocketHandlerTest</logger>
<level>WARNING</level>
<class>com.mrbear.logging.tests.SimpleSocketHandlerTest</class>
<method>testSocketHandler</method>
<thread>1</thread>
<message>Warning ....</message>
</record>
I would like to point out that, strictly speaking, the above XML document is not valid, as there is no closing </log> tag.

This makes sense, as the xml is streamed and never quite ended.

I would also like to point out that the XML document contains for each record both the number of milliseconds sinds 1970 and the date. This at first seems superfluous to me, but after some thought it makes sense. If you add both, you get valuable information in what timezone the server logging messages is working. This information is most likely lost in the milliseconds approach.

References

[1] Oracle Blog - Configure my Custom Log Handler in GlassFish 3.1
https://blogs.oracle.com/naman/entry/configure_my_custom_log_handler
[2] StackOverflow - Capture GlassFish log file into SQL/JPA data base
http://stackoverflow.com/questions/12397861/capture-glassfish-log-file-into-sql-jpa-data-base
[3] 7 Administering the Logging Service - Adding a Custom Logging Handler
Oracle GlassFish Server Administration Guide
HK2 - Dependency Injection Kernel
https://hk2.java.net/2.3.0/
Oracle JavaTM Tutorials - Lesson: JDBC Introduction
https://docs.oracle.com/javase/tutorial/jdbc/overview/index.html
JavaBendeR - Simple Log server with java SocketHandler and centralization of log records
http://javabender.blogspot.nl/2010/09/simple-log-server-with-java.html
Java Logging API Tutorial – Examples of Logger Levels, Handlers, Formatters and Filters
http://www.journaldev.com/977/java-logging-api-tutorial-examples-logger-levels-handlers-formatters-filters
Oracle JavaTM Tutorials - Reading XML Data into a DOM
https://docs.oracle.com/javase/tutorial/jaxp/dom/readingXML.html
Oracle JavaTM Tutorials - Using StAX
https://docs.oracle.com/javase/tutorial/jaxp/stax/using.html

Sunday, 19 October 2014

Glassfish Security Realms

The JDBCRealm

In the Glassfish Administration Console, go to Configurations -> server-config -> Security -> Realms -> and select "New".

We're going to choose com.sun.enterprise.security.ee.auth.realm.jdbc.JDBCRealm.

The jaas.context name of the new realm is "jdbcRealm".

A jdbcRealm needs both a user table and a group table. The group table should store both userids as well as groupids. In other words,
  • the group table doesn't only contain groups, and
  • has a composite primary key, consisting of userid and groupid.

Database Schema

The user table is called "mm_admin" in my case, and the groups table is called "mm_admin_mm_groups".

I have created the "mm_groups" table to store extra information for groups, but that table is ignored by the jdbcRealm.

Some serious disadvantages of the jdbcRealm are:
  • groups are cached, which means if a user changes groups, the Glassfish Server needs to be rebooted.
  • flexibility is about zero, for instance if I have an expiration date attached to my usertable, I'm out of luck.
All this causes me to look elsewhere, namely towards the flexibleJDBCRealm5

The flexibleJDBCRealm

The great advantage of the flexibleJDBCRealm is that you can add SQL queries that are run to retrieve passwords and groups.

Instructions

Well, according to instructions found here6, you have to:
  1. download the flexiblejdbcrealm-deploy-1.2-all.jar
  2. put it into glassfish/domains/domain1/lib directory
  3. change glassfish/domains/domain1/config/login.conf to add
    flexibleJdbcRealm {
    org.wamblee.glassfish.auth.FlexibleJdbcLoginModule required;
    };
  4. reboot the glassfish
  5. In the Glassfish Administration Console, go to Configurations -> server-config -> Security -> Realms -> and select "New"
  6. enter the class name org.wamblee.glassfish.auth.FlexibleJdbcRealm
  7. set the properties, for example like in the image above.

Properties

The properties that can be entered could use a little more explaining compared to what is available at [6].
password.digest
uses MessageDigest(String algorithm). Possible values are "PLAIN", "MD2", "MD5", "SHA-1", "SHA-256", "SHA-384" and SHA-512.
password.encoding
the encoding, I find HEX to be the most useful. Possibile values are "TEXT", "HEX" and "BASE64". A variation of "HEX" is "HEX:<digits>", for example "HEX:40". This padds the beginning of the encoding with zeros to reach the length required. For example MySQL itself always creates passwords using the SHA() function to 40 hex digits also padding the beginning with zeros.
sql.groups
select groupid from mm_admin_mm_groups where name in (?)
sql.password
select passwd from mm_admin where name in (?) and validuntil > now()
jaas.context
flexibleJdbcRealm
datasource.jndi
of course, the connection string to the database
NOTE: he's complaining about setting properties in the realm with a = in them. So I had to go back to an "in (?)" construction for the SQL queries.

Logging

In glassfish/domains/domain1/config/logging.properties, add: org.wamblee.level=FINEST.

Application Configuration

My web.xml would look like thusly:
<?xml version="1.0" encoding="UTF-8"?>
<web-app version="3.1" 
    xmlns="http://xmlns.jcp.org/xml/ns/javaee" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee 
             http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd">

    <session-config>
        <session-timeout>
            30
        </session-timeout>
    </session-config>
    <security-constraint>
        <display-name>Administration</display-name>
        <web-resource-collection>
            <web-resource-name>Administration REST Resources</web-resource-name>
            <description/>
            <url-pattern>/resources/administration/*</url-pattern>
            <url-pattern>/administration/*</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <description/>
            <role-name>deputy</role-name>
        </auth-constraint>
    </security-constraint>
    <login-config>
        <auth-method>BASIC</auth-method>
        <realm-name>deputyRealm</realm-name>
    </login-config>
    <security-role>
        <description/>
        <role-name>deputy</role-name>
    </security-role>
</web-app>
My glassfish-web.xml looks like:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE glassfish-web-app 
    PUBLIC 
    "-//GlassFish.org//DTD GlassFish Application Server 3.1 Servlet 3.0//EN" 
    "http://glassfish.org/dtds/glassfish-web-app_3_0-1.dtd">

<glassfish-web-app error-url="">
  <security-role-mapping>
    <role-name>deputy</role-name>
    <group-name>deputy</group-name>
  </security-role-mapping>
  <class-loader delegate="true"/>
  <jsp-config>
    <property name="keepgenerated" value="true">
      <description>Keep a copy of the generated servlet class' java code.</description>
    </property>
  </jsp-config>
</glassfish-web-app>
In this file you can create a mapping between security roles and groups.

Declarative Security

An example of declarative security using Annotations:
@DeclareRoles("deputy")
@RolesAllowed("deputy")
@Stateless
@Path("/administration/worldattributes")
public class WorldattributesBean extends AbstractFacade<Worldattribute>
{
This means the role definition in the web.xml can be removed.

In the example above, the security role is applied to the entire Bean, so for each method in the bean, instead of on each method that requires the role.

Programmatic Security

Unfortunately, some fine-grained security can only be done with programmatic means, for example determining if a person who has authenticated themselves owns a certain object that needs mutated upon. In my example, this info in a Rest service can be obtained through a SecurityContext:
import javax.ws.rs.core.SecurityContext;
import javax.ws.rs.core.Context;
...
@POST
@Override
@Consumes(
{
     "application/xml", "application/json"
})
public void create(Worldattribute entity, @Context SecurityContext sc)
{
  itsLog.info("create");
  final String name = sc.getUserPrincipal().getName();

FAQ

I get WEB9102: Web Login Failed: com.sun.enterprise.security.auth.login.common.LoginException: Login failed: No LoginModules configured for flexibleJdbcRealm?
You forgot to add the entry to the login.conf as specified above.

References

[1] Oracle - Security in the Java EE Platform
http://docs.oracle.com/javaee/7/tutorial/doc/security-intro.htm
[2] The Java EE 7 Tutorial
Release 7 for Java EE Platform
[3] Oracle - Using the JDBC Realm for User Authentication
http://docs.oracle.com/javaee/6/tutorial/doc/glxgo.html
[4] Understanding Web Security
http://java.dzone.com/articles/understanding-web-security
[5] FlexibleJDBCRealm
http://flexiblejdbcrealm.wamblee.org/site/
[6] Installation instructions
http://flexiblejdbcrealm.wamblee.org/site/documentation/snapshot/installation.html

Friday, 8 August 2014

Mysql JDBC Issue

Hammered myself into a wall here. Turns out when I create my Java Entities from a Database Table (using NetBeans Wizards), I get the following:

@Entity
@Table(name = "mm_log", catalog = "mmud", schema = "")
@NamedQueries(
{
    @NamedQuery(name = "Log.findAll", query = "SELECT l FROM Log l"),
    @NamedQuery(name = "Log.findByName", query = "SELECT l FROM Log l WHERE l.name = :name"),
    @NamedQuery(name = "Log.findByMessage", query = "SELECT l FROM Log l WHERE l.message = :message")
})
public class Log implements Serializable

Notice the catalog and the schema in the annotation for Table?

Now, it turns out1 that "schema" is unsupported in MySQL, and "catalog", apparently, is just a fancy way in MySQL of saying "database".

So, if I create a brand new database (call it "newmmud"), create a nice JDBC Connection Pool to it in my Glassfish server, yet this blasted Entity (Log, in the example) will still refer to the database "mmud"!

That's what I call confusing!

Here's the message, for posterity's sake:
“You're correct. For legacy reasons (including compatibility with ODBC, and SQL Server), JDBC's concept of "catalog" maps to MySQL's concept of "databases".

Starting in MySQL-4.1, "schema" is an alias for "database", but it doesn't act like a SQL-standard schema, so we don't support the concept in the JDBC driver.”
- Mark Matthews

References

[1] Forum - Re: catalog versus schema
http://forums.mysql.com/read.php?39,137564,137629#msg-137629

Sunday, 9 October 2011

Different Ways to use JPQL

Introduction

There are several ways in which to execute JPQL (Java Persistence Query Language). I'll introduce the "old way" first, with JDBC and direct connections to the database, before branching off into the ways used by ORMs (Object Relational Mapping).

Each way has its advantages and disadvantages.

The old way:
  • direct database access
  • database connection pool
Using an ORM
  • hql in java code
  • named queries in annotations
  • named queries in xml
  • Criterion API
  • retrieving the object by its primary key

The Old Way

Direct Database Access

This is an example of direct database connection with jdbc. Setting it up, sending the query, reading the result, close the resultset and shutting down the connection.


/**
 * Connects to the database using an url. The url looks something like
 * "jdbc:mysql://localhost.localdomain/mud?user=root&password=". Uses the
 * classname in Constants.dbjdbcclass to get the right class for interfacing
 * with the database server. If the database used is changed (or to be more
 * specific the jdbc driver is changed) change the constant.
 * 
 * @throws InstantiationException
 *             happens when it is impossible to instantiate the proper
 *             database class, from the class name as provided by
 *             Constants.dbjdbcclass.
 * @throws ClassNotFoundException
 *             happens when it is impossible to find the proper database
 *             class, from the class name as provided by
 *             Constants.dbjdbcclass.
 * @throws IllegalAccessException
 *             happens when it is not possible to create the database class
 *             because access restrictions apply.
 * @throws SQLException
 *             happens when a connection to the database Server could not be
 *             established.
 */

public Macro runQuery() throws SQLException, InstantiationException,
    ClassNotFoundException, IllegalAccessException
{
  Macro result = null;
  try
  {
    Class.forName(Constants.dbjdbcclass).newInstance();

    // jdbc:mysql://[host][,failoverhost...][:port]/[database]
    // [?propertyName1][=propertyValue1][&propertyName2][=propertyValue2]...
    String theUrl = Constants.dburl + "://" + Constants.dbhost
      + Constants.dbdomain + "/" + Constants.dbname + "?user="
      + Constants.dbuser + "&password=" + Constants.dbpasswd;
    theConnection = DriverManager.getConnection(theUrl);

    PreparedStatement statGetMacro = null;
    statGetMacro = theConnection.prepareStatement(sqlGetMacro);
    statGetMacro.setString(2, macroname);
    res = statGetMacro.executeQuery();
    if (res != null && res.next())
    {
      result = new Macro(res.getString("macroname"), res.getString("contents"));
      res.close();
    }
    statGetMacro.close();
  } catch (Exception e)
  {
    throw new MudDatabaseException(Constants.DATABASECONNECTIONERROR, e2);
  }
  finally
  {
    if (theConnection != null) {theConnection.close();}
  }
  return result;
}
Advantages:
  • complete control
  • very little libraries required
Disadvantages:
  • sql syntax spread throughout the java code
  • lots of error-prone boiler plate code required
  • expensive in setting up new database connections for every query
  • sytax of query not checked, until method called
  • changing the query requires source code change, recompilation, repackaging, etc

Database Connection Pool

This is example of database connection with jndi resource pool, where the only thing required is the jndi name. In this case it is irrelevant which connection we receive from the connection pool. The actual theConnection.close() does nothing, but returns the connection back to the pool.


public void runQuery() throws Exception
{
  Connection con=null;
  ResultSet rst=null;
  PreparedStatement stmt=null;

  try
  {
    Context ctx = new InitialContext();
    DataSource ds = (DataSource) ctx.lookup("jdbc/mmud");
    con = ds.getConnection();

    stmt=con.prepareStatement("select * from links where type = 1 order by creation");
    rst=stmt.executeQuery();
    while(rst.next())
    {
        out.println("<li><A HREF=\"" + rst.getString("url") + "\">" +
          rst.getString("linkname") + "</A> (<i>" +
          rst.getString("name") + "</i>)<P>");
    }
    rst.close();
    stmt.close();
  }        
  finally
  {
    if (rst != null) {try {rst.close();} catch (Exception e){}}
    if (stmt != null) {try {stmt.close();} catch (Exception e)}}
    if (con != null) {try {con.close();} catch (Exception e){}}
  }
}

Advantages:
  • database connection resource pool
  • no need to open and close connection, connections are reused
Disadvantages:
  • sql syntax spread throughout the java code
  • still a lot of error-prone boiler plate code required
  • sytax of query not checked, until method called
  • if connections/resources not properly released, the pool will run out of connections/resources
  • changing the query requires source code change, recompilation, repackaging, etc

Using an ORM

hql in java code

@Override
public void deleteAddress() 
{
  StringBuilder queryBuilder = new StringBuilder();
  queryBuilder.append("DELETE Address address ");
  queryBuilder.append("WHERE  address.country = 'Italy' ");
  queryBuilder.append("AND NOT EXISTS (SELECT '' ");
  queryBuilder.append("            FROM Customer cust ");
  queryBuilder.append("            WHERE cust.address = address ");
  queryBuilder.append("           ) ");

  Query query = em.createQuery(queryBuilder.toString());
  query.executeUpdate();
}

Unfortunately, the code above is what I encounter frequently in the source of my employers current software product.

Advantages:
  • none
Disadvantages:
  • a StringBuffer is created every time the method is called, at the very least this should be a constant
  • jpql is created and parsed and compiled every time the method is called
  • sql/hql/jpql syntax spread throughout the java code
  • sytax of query not checked, until method called
  • changing the query requires source code change, recompilation, repackaging, etc


Named queries in annotations

Queries can be defined in the annotations on top of Entity classes. The annotation would look a little like the one below.

@NamedQueries({
    @NamedQuery(name = "Character.findAll", query = "SELECT c FROM Character c"),
    @NamedQuery(name = "Character.findByName", query = "SELECT c FROM Character c WHERE c.name = :name")})

One of the problems often heard is that the annotation has to be on an Entity class, when you perhaps would prefer to have it someplace else. In that case, it might be an idea to define the constants as strings in a different part, like a Service or a DAO or what have you, and refer to them in the NamedQuery annotations. Like in the two-part example below.

As the name of a named query is global within the persistent provider, it is essential to create unique names within the application. That is why in most cases, the name of a named query is prefixed with the class name.

/**
 * Game bean.
 * @author Mr. Bear
 */

@Stateless
@Remote(GameBeanRemote.class)
public class GameBean implements GameBeanRemote
{
    public static final String character = "Character";

    /**
     * label for the find all characters query.
     */

    public static final String FINDALLCHARS_QUERYNAME = character + ".findAll";

    /**
     * query for the find of all characters query.
     */

    public static final String FINDALLCHARS_QUERY = "SELECT c FROM Character c";

    /**
     * label for the find character based on name query.
     */

    public static final String FINDCHAR_QUERYNAME = character + ".findByName";

    /**
     * query for the find of character based on name query.
     */

    public static final String FINDCHAR_QUERY = "SELECT c FROM Character c WHERE c.name = :name";

    @PersistenceContext(unitName = "datasource")
    private EntityManager em;
/**
 * The Character entity.
 * @author Mr. Bear
 */

@Entity
@Table(name = "mm_usertable")
@NamedQueries({
   @NamedQuery(name = GameBean.FINDALLCHARS_QUERYNAME, query = GameBean.FINDALLCHARS_QUERY),
   @NamedQuery(name = GameBean.FINDCHAR_QUERYNAME, query = GameBean.FINDCHAR_QUERY)})
public class Character implements Serializable {
    private static final long serialVersionUID = 1L;
    @Id
    @Basic(optional = false)
    @NotNull
    @Size(min = 1, max = 20)
    @Column(name = "name")
    private String name;

Using the named queries in code is as simple as getting the entity manager and telling him the name of the NamedQuery you wish to use, add the parameter values, and request the result. You can refer to the NamedQueries by name, hence the name, get it?

@Override
public CommandOutput retrieveCharacter(String name)
{
    itsLog.entering(this.getClass().getName(), "retrieveCharacter");
    Query q = em.createNamedQuery("Character.findByName", Character.class);
    q.setParameter("name", name);
    Character c = (Character) q.getSingleResult();
    if (c == null)
    {
        itsLog.exiting(this.getClass().getName(), "retrieveCharacter");
        return new CommandOutput("", "", "");
    }
    CommandOutput co = new CommandOutput(c.getName(), c.getArm(), c.getBeard());
    itsLog.exiting(this.getClass().getName(), "retrieveCharacter ", co);
    return co;
}

The great part of NamedQueries is that they are checked and pre-compiled by Hibernate when the server starts, so you get instant messages if you screwed up the syntax.

Advantages:
  • the Persistence provider might be able to precompile them, and this can help catch bugs before deployment
  • sql syntax no longer spread throughout the java code
  • sql queries externalised to the java code
  • there's a central place for queries, making them easy to find and easy to re-use.
Disadvantages:
  • named queries in annotations can only be added to the Entity classes or the SuperMapClass. This is especially awkward when you have a query spanning multiple entities.
  • named queries are referenced using a key, which is global to the application.
  • named queries are a little difficult to debug, as a change requires an application server reboot.
  • changing the query in the annotation requires source code change, recompilation, repackaging, etc

In the examples below, JBoss is used in combination with Hibernate. The first example is the jboss log when things go wrong, and the second example is when things go right.

09:19:23,635 ERROR [SessionFactoryImpl] Error in named query: FullAddress.findAddressesOfCustomers
org.hibernate.QueryException: could not resolve property: entitynr of: mrbear.entity.Address [DELETE Address address WHERE address.country = 'Italy' AND address.entitynr > 1000 AND NOT EXISTS (SELECT '' FROM Customer cust WHERE cust.address = address) ]
at org.hibernate.persister.entity.AbstractPropertyMapping.propertyException(AbstractPropertyMapping.java:67)
09:23:10,946 INFO [AnnotationBinder] Binding entity from annotated class: mrbear.entity.Address
09:23:10,948 INFO [QueryBinder] Binding Named query: FullAddress.findAddressesOfCustomers => DELETE Address address WHERE address.country = 'Italy' AND address.entityNr > 1000 AND NOT EXISTS (SELECT '' FROM Customer cust WHERE cust.address = address)

Let me point you especially to the fact that HQL (or JPQL for that matter), contrary to SQL, is case-sensitive (for the most part). It would have to be, as it has a direct link to java classes and these are also case-sensitive.

In the example above, the getter and setter for entity number was specified as getEntityNr and setEntityNr, hence the problem with the casing in the first example.

According to Gavin King [1]:
We leave it up to you if you want to utilize the named query feature. However, we consider query strings in the application code (except if they're in annotations) to be the second choice; you should always externalize query strings if possible.

Named queries in xml

If you do not like to have your JPQL in your code at all, there is always the possibility of putting the queries into the xml files that are a part of the configuration of the jar file.

There is no reason why you cannot have both, both named queries in your annotations and queries in your xml persistence configuration files.

One of the great things is that the named queries in the persistence xml files take precedence over the named queries in the annotations. This means, that, when deployed in a specific production environment, it is possible to change the named queries, without having to touch/recompile/build/package the java source code. So, you can do convention over configuration.

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
  <persistence-unit name="mmud" transaction-type="JTA">
    <provider>org.hibernate.ejb.HibernatePersistence</provider>
    <jta-data-source>mmud</jta-data-source>
    <!-- Named JPQL queries per entity, but any other organization is possible  -->
    <mapping-file>META-INF/jpql/Character.xml</mapping-file>
    <exclude-unlisted-classes>false</exclude-unlisted-classes>
    <properties/>
  </persistence-unit>
</persistence>
<?xml version="1.0" encoding="UTF-8"?>
<entity-mappings version="1.0" xmlns="http://java.sun.com/xml/ns/persistence/orm"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence/orm http://java.sun.com/xml/ns/persistence/orm_1_0.xsd "
>

    <named-query name="Character.findByName">
        <query>
            SELECT
                c
            FROM
                Character c
            WHERE
                c.password = :name
            OR
                c.name = 'Midevia'
        </query>
    </named-query>
</entity-mappings>

So in the code example that uses annotations, when using these two persistence xml files displayed above, suddenly "Midevia" is retrieved from the database, instead of the name you requested.

Advantages:
  • the Persistence provider might be able to precompile them, and this can help catch bugs before deployment
  • sql syntax no longer anywhere in the java code
  • there's a central place for queries, making them easy to find and easy to re-use.
  • named queries can be spread through many different xml files, making them more easy to categorise.
Disadvantages:
  • named queries are referenced using a key, which is global to the application.
  • named queries in xml, can be changed, if you don't mind hacking your way through xml.
  • named queries are a little difficult to debug, as a change requires an application server reboot.

Criterion API

@Override
public CommandOutput retrieveCharacter(String name)
{
    itsLog.entering(this.getClass().getName(), "retrieveCharacter");

    // setup criteria builders/queries
    CriteriaBuilder criteriaBuilder = em.getCriteriaBuilder();
    CriteriaQuery criteriaQuery = criteriaBuilder.createQuery();

    // add from
    Root<Character> from = criteriaQuery.from(Character.class);
    CriteriaQuery<Character> select = criteriaQuery.select(from);

    // add predicate
    Predicate predicate = criteriaBuilder.equal(from.get("name"), name);
    criteriaQuery.where(predicate);

    // create the select query
    TypedQuery<Character> typedQuery = em.createQuery(select);

    // get the results
    List<Character> resultList = typedQuery.getResultList();

    if (resultList == null || resultList.size() != 1)
    {
       itsLog.exiting(this.getClass().getName(), "retrieveCharacter");
       return new CommandOutput("", "", "");
    }
    CommandOutput co = new CommandOutput(
            resultList.get(0).getName(),
            resultList.get(0).getArm(),
            resultList.get(0).getBeard());
    itsLog.exiting(this.getClass().getName(), "retrieveCharacter ", co);
    return co;
}
Advantages:
  • the Persistence provider might be able to precompile them, and this can help catch bugs before deployment
  • sql syntax no longer anywhere in the java code
  • the query is dynamic, making it easy to change depending on input from the user
  • it is java, so every tool and compiler can aid in the development, like syntax checking and type checking and the like
Disadvantages:
  • fairly unreadable

Retrieving the object by its primary key

Ideally, we do not wish to use SQL or HQL or the Criteria API at all. If possible, we just want to tell the Entity Manager to get the objects we need. In most cases, this is not easy. But for some concepts, like getting an object based on its primary key (so we are always sure there's just one (or zero) available) there's a much more straightforward way.

@Override
public CommandOutput retrieveCharacter(String name)
{
    itsLog.entering(this.getClass().getName(), "retrieveCharacter");
    Character c = em.find(Character.class, name);
    if (c == null)
    {
        itsLog.exiting(this.getClass().getName(), "retrieveCharacter");
        return new CommandOutput("", "", "");
    }
    CommandOutput co = new CommandOutput(c.getName(), c.getArm(), c.getBeard());
    itsLog.exiting(this.getClass().getName(), "retrieveCharacter ", co);
    return co;
}
Advantages:
  • no SQL/HQL/JPSQL of any kind required
  • the JPA implementation can cache the object and retrieve is quickly.
Disadvantages:
  • only for specific uses, like retrieving objects by primary key.

Conclusion

I hope to have demonstrated the different uses that exist in retrieving data from a database using any kind of ORM, compatible with the JPA. As you are no doubt aware from this piece, I strongly consider the last four examples to be good practices of creating JPQL queries in your application.

As always, each has its strong and weak points, and therefore are applicable to different situations. Evaluate the situation and pick the one that is best suited for the job.

References


[1] Java Persistence with Hibernate
Christian Bauer, Gavin King
[2] chapter 2.3 - Hibernate Community Documentation
http://docs.jboss.org/hibernate/annotations/3.5/reference/en/html_single/
[3] Organize Your Named JPQL Queries
http://eubauer.de/kingsware/2011/03/25/organize-your-named-jpql-queries/
[4] Hibernate: Criteria vs. HQL
http://stackoverflow.com/questions/197474/hibernate-criteria-vs-hql
[5] Are Hibernate named HQL queries (in annotations) optimised?
http://stackoverflow.com/questions/2641997/are-hibernate-named-hql-queries-in-annotations-optimised
[6] GlassFish Project - Java Persistence Example
http://glassfish.java.net/javaee5/persistence/persistence-example.html
[7] Use string constants in annotations
http://javahowto.blogspot.com/2009/04/use-string-constants-in-annotations.html
[8] Java Persistence 2.0 Public Draft: Criteria API
http://blogs.oracle.com/ldemichiel/entry/java_persistence_2_0_public1
[9] Dynamic, typesafe queries in JPA 2.0
http://www.ibm.com/developerworks/java/library/j-typesafejpa/
[10] Where to put named queries in JPA?
http://jdevelopment.nl/put-named-queries-jpa/
[11] Java Glossary
http://randomthoughtsonjavaprogramming.blogspot.com/p/glossary.html