At work, there is a bit of Legacy around.
The following example of several Enum classes we have, is what is currently frustrating me (and I try actively to change them).
Let's enumerate (oh, look, I made a joke) the issues:
Replacing magic numbers with constants and enums is, of course, one of the Good Thingstm.
But in this example, they seem to have totally missed the point.
Not only have they replaced magic numbers with "magic enums", it's an eyesore to use the underscore in the Enum instances.
When you have to take refuge in the use of underscores to make things work, it is a sure sign that you are doing something seriously wrong.
Monday, 29 December 2014
Monday, 22 December 2014
The Terrible Dangers of Autoboxing (Part 3) - Solution
The solution to my previous post regarding the Terrible Dangers of Autoboxing, is written below.
In our case, the type of the second expression is 0L (long) and the third is null but returns from a Long method.
Java assumes that the second and third operands have types that are convertible to numeric types.
In that case, the conditional operator causes binary numeric promotion.2
During binary numeric promotion, it is possible an unboxing is performed3 (on null in our case), which consequently fails with a NullPointerException.
The solution is to replace
P.S. Rewriting the conditional expression into an if statement, will also solve it.
In our case, the type of the second expression is 0L (long) and the third is null but returns from a Long method.
Java assumes that the second and third operands have types that are convertible to numeric types.
In that case, the conditional operator causes binary numeric promotion.2
During binary numeric promotion, it is possible an unboxing is performed3 (on null in our case), which consequently fails with a NullPointerException.
The solution is to replace
return hasSurfaceArea() && getSurfaceInM2() == null ? 0L : getSurfaceInM2();
With:return hasSurfaceArea() && getSurfaceInM2() == null ? Long.valueOf(0) : getSurfaceInM2();
P.S. Rewriting the conditional expression into an if statement, will also solve it.
Quote
Here is a brilliant quote that every Java programmer needs to remember and understand (especially that last bit):“Autoboxing and unboxing blur the distinction between primitive types and reference types, but they do not eliminate it.1”
References
- [1] Java SE Documentation - Autoboxing
- http://docs.oracle.com/javase/7/docs/technotes/guides/language/autoboxing.html
- [2] Chapter 15.25 Conditional Operator ? :
- Java Language Specification
- [3] Chapter 5.6.2 Binary Numeric Promotion
- Java Language Specification
- Java Tutorials - Autoboxing and Unboxing
- http://docs.oracle.com/javase/tutorial/java/data/autoboxing.html
Monday, 15 December 2014
The Terrible Dangers of Autoboxing (Part 3)
Please explain why the code throws a NullPointerException.
Output:/opt/tools/java/jdk1.7.0_65/bin/java com.mrbear.autoboxing.Plot
0
3
3
Exception in thread "main" java.lang.NullPointerException
at com.mrbear.autoboxing.Plot.getSurfaceArea(Plot.java:28)
at com.mrbear.autoboxing.Plot.main(Plot.java:39)
Process finished with exit code 1
Also, please mention how to fix it.0
3
3
Exception in thread "main" java.lang.NullPointerException
at com.mrbear.autoboxing.Plot.getSurfaceArea(Plot.java:28)
at com.mrbear.autoboxing.Plot.main(Plot.java:39)
Process finished with exit code 1
((Solution))
Wednesday, 3 December 2014
REST + Angular = Restangular
JQuery and REST
I have already mentioned calling REST services using JQuery1 in my entry on REST and Error messages at my blog.At the risk of being redundant, here it is:
Unfortunately, it requires a lot of settings and a lot of DOM manipulation to create the view and make it work as it should.
I was looking for an alternative, something that's easy to use without a lot of boilerplate.
I would prefer something like JSF, but solely on the client side, and which fully supports REST. I found something to my liking in Angular.
Angular
AngularJS is, according to their website2, a Superheroic JavaScript MVW Framework.“AngularJS is what HTML would have been, had it been designed for building web-apps. Declarative templates with data-binding, MVW, MVVM, MVC, dependency injection and great testability story all implemented with pure client-side JavaScript!”As far as I can tell, you add Angular specific xml attributes to your html tags, and Angular takes care of the updating of those tags, depending on what's changed. Of course, that's just the tip of the proverbial iceberg.
Angular and REST
The Angular version of the JQuery $.ajax call is called $http3, and is about what you would expect.Both in JQuery and in Angular, it is a wrapper around the XMLHttpRequest.
This is not really a REST-aware API, to say the least. Therefore Angular also provides a $resource service. This service creates resource objects that makes it easy to communicate with RESTful backends.
It is an optional module in Angular, called ngResource, and needs to be installed.
Restangular and REST
However, I have opted to make use of Restangular4. It seems to be more self-evident how it works, follows the guideline of Angular more closely and seems to abstract away a lot of the boilerplate, for example URLs.In the example above there are numerous interesting tidbits:
- In the config call, I set the base url. All consequent calls to REST resources have this url as root.
- In the config call, I needed to set the "id" of the resource accessed, otherwise Restangular assumes that the "id" is a field called "id". In this case it was "name"5 6.
- var restBase = Restangular.all("worldattributes") sets a resource (in this case for world attributes).
- restBase.getList()This creates a GET XMLHttpRequest for url "/karchangame/resources/administration/worldattributes" and retrieves all objects. It assumes JSON. As you see, I promptly put all info into the Angular scope, so it can be reached in the HTML file (see below).
.then(function(worldattributes)
{
// returns a list of worldattributes
$scope.worldattributes = worldattributes;
$scope.setFilter();
$scope.isNew = false;
});
- The objects returned by Restangular, are themselves also REST resources, therefore contain methods such as put() which updates it (causing an automatic HTTP PUT request).
- The call remove is translated to a DELETE XMLHttpRequest.
- And my update method decides on a POST or PUT depending on wether it is a new or a changed world attribute.
- The Restangular.copy call is convenient, as it makes a deep copy of the object.
- Sometimes, the backend REST urls are not standard, in which case you wish to turn from the default, and start making calls with a bit more control. An example of such is visible with the customDELETE in the disown function.
Notice that the HTML file looks almost exactly as a simple HTML file would, besides the added Angular attributes and the references to the $scope object (by means of {{ and }}).
References
- [1] JQuery.ajax() | JQuery API Documentation
- http://api.jquery.com/jquery.ajax/
- [2] AngularJS - Main Website
- https://angularjs.org/
- [3] ngbook - The Complete Book on AngularJS
- Ari Lerner
- [4] Github - Restangular
- https://github.com/mgonto/restangular
- [5] Stackoverflow - Restangular put loses part of the url
- http://stackoverflow.com/questions/24262949/restangular-put-loses-part-of-the-url
- [6] Restangular - GitHub : What if each of my models has a different ID name like CustomerID for Customer?
- https://github.com/mgonto/restangular#what-if-each-of-my-models-has-a-different-id-name-like-customerid-for-customer
- Foundation 5
- http://foundation.zurb.com/
Labels:
angularjs,
frameworks,
javascript,
jquery,
ngResource,
rest,
restangular,
XHR,
XMLHttpRequest
Friday, 14 November 2014
SQL Operator Precedence
Just a small note for me to remember how this works.
Clearly, AND takes precedence over OR, as usual.
select *
from ((select 'FINANCE' as dept, 42 as employees from dual) union
(select 'SALES' as dept, 32 as employees from dual)) departments
where
departments.dept = 'FINANCE'
or
departments.dept = 'SALES'
and
departments.employees < 30;
returns: FINANCE 42
from ((select 'FINANCE' as dept, 42 as employees from dual) union
(select 'SALES' as dept, 32 as employees from dual)) departments
where
departments.dept = 'FINANCE'
or
departments.dept = 'SALES'
and
departments.employees < 30;
returns: FINANCE 42
Clearly, AND takes precedence over OR, as usual.
A AND B OR C => (A AND B) OR C
A OR B AND C => A OR (B AND C)
A OR B AND C => A OR (B AND C)
References
- Stackoverflow - sql logic operator precedence
- http://stackoverflow.com/questions/1241142/sql-logic-operator-precedence-and-and-or
- SQL Logical Operators
- http://www.praetoriate.com/t_garmany_easysql_sql_logical_operators.htm
Sunday, 9 November 2014
J-Fall 2014
Well, I have once again been privileged enough to attend the J-Fall 2014 on Wednesday, the 5th of November, in Nijkerk, the Netherlands.
I'll provide a short summary of the Breakout Sessions I followed, and the parts that really struck me as interesting. There's more information on the sessions available at [1].
My chosen schedule:
The Keynotes were not actually that interesting. The Keynote of Oracle was more related to awards for proven Java-ness, than what Oracle has planned for the future. The keynote of Oracle was provided by James Weaver, who was also giving a Session last year regarding JavaFX and guitar playing.
The Keynote of ING was surprisingly similar to last year, regarding the banks new way of working with software releases (Agile, Scrum, three releases every week, automated testing, DevOps, etc.)
The Keynote of Salves was interesting, in that it had a high geek-techy-factor, with drones flying, and brainwaves to control them, with pattern recognition in cameras. But the complexity of the setup, and the conference location and The Dreaded Demo Effect got to 'em, which is a shame.
I found the best Keynote to be the one provided by Quintor, a consultancy business started in 2005, regarding a simulated Cluster with Hadoop they set up during one of their "Summer Camps" to brute-force a Shortest Path Algorithm. They needed some penguins to find an igloo between ice blocks and rocks. They created "Willem", a server that was running 20 or so virtual environments, chained in a Hadoop cluster with Apache HBase as work memory and Akka Reactive Streams.
It seems to me that event-sourcing is basically based on the database transactionlog that has been in Databases for the past hundreds of years.
The idea is that we store the events that happened, instead of storing only the changes of data. In the latter case we lose a lot of history. In order to keep the side effects out of the way when replaying the event log, these are not re-done.
A simple example is, instead of storing Price and Orders, we store events like PriceChanged or OrderCancelled.
Of course, this presents problems as you have to rerun the event log in order to find out the current situation of the system. This is solved by creating snapshots.
Another problem is the fact that Searching for stuff becomes harder. So it makes sense to have basically two different modules, one module is responsible for changes (and stores events and the likes) and one module is responsible for querying and searching (and searches, for example, over Snapshots).
The abbreviation concerning this is called CQRS, Command Query Responsibility Segregation2. Here the command model is the eventstore (the "journal") and the query model can have several data stores, each containing for example snapshots.
It is possible to have the module that stores events present hooks to the searchmodule to keep the segregated models synchronised.
Some software that is used for the event model and storage of events are:
Akka can be used for all this, it provides a new experimental persistence module called appropriately Akka Persistence, that uses event sourcing.
Code can be found on github at https://github.com/sandermak/akka-eventsourcing. The url provided by Sander was http://bit.ly/akka-es, convenient for if the source moves.
One of the advantages, are that events show the business processes a lot better than normal persisted objects do. With events it is much more clear what is happening.
Conclusion is that event sourcing is very applicable to Domain Driven Design.
A word on versioning, a change in event-structure is going to cause a problem for the history (which uses the old event-structure). There are several solutions for this:
I rated this as the best Session of the day!
In Java 7 "Futures" were introduced to take care of doing things in parallel using ThreadPools.
A Future is a placeholder for a value that hasn't been set yet, but will (hopefully) be set in the future.
The Java API for Futures is very extensive with a lot of different classes containing a lot of different methods. In the future (my, was that a pun?) there might well come to the front a framework that abstracts away a lot of the nitty-gritty of this highly detailed API. Where the other sessions were light on the amount of code, this one more than made up for them.
Some codepoints:
Modelling the flow (probably in a flow model) helps a lot, as you lose a lot of visibility on what the code is actually doing.
You tend to define/plan everything up front, with Futures, and then you start the ball rolling, so to speak, with the last call in the chain.
JAXRS 2.0 has the possibility to perform REST services Asynchronously. This can be done, using:
HTTP connections are released during requests, this will lead to much better scalability. (HTTP threads can be re-pooled)
His source is on github at martijnblankestijn/reactive-rest-demo.
One diagram that basically says it all, where green is retrieval of customer information, and red and purple are contract data and communication data respectively of said customer:
The reason why, is that the Cost of A Change should be predictable, and should not result in slowly escalating costs, regarding lost/unattainable deadlines or high maintenance.
The way to do this is with Functional Modularisation, because a functional change should only impact that functional module.
There should be few hard connections between domains, for example no Foreign Keys or the like.
There should be no query between modules, which is a problem when you wish to do some serious Search Queries. A query between modules and across several modules, causes these modules to be tightly coupled.
So, in other words, cross module searches should be postponed as long as possible.
When this isn't possible, you could use CQRS, Command Query Responsibility Segregation.
Elasticsearch can be used to do the searching.
All the modules can be queried individually by the SearchModule, and the SearchModule uses elasticsearch.
I've managed to nick their Lab and plan on practicing more of the assignments at home (there wasn't enough time for me at the conference, there were quite many of them).
Here are some book titles mentioned:
Characteristics of Micro services:
Conway's Law3: organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.
Basically what this means is that, for example, if you have three teams, one for the UI, one for the Middleware, and one for the database, you end up with a software product that uses a UI, a Middleware and a Database. If the entire team works on ONE microservice, you get the absolute best product. It also means you cannot have a standard team. It is SOA (Service Oriented Architecture) done right.
Unfortunately, at the end of all that it was 17:00 hours, and I was Conference-tired, so I decided to skip Using Docker to Develop, Test and Run Maven projects.
The main conclusion that I can take away from this J-Fall is that somehow a number of the Sessions was regarding a Separation of Concerns in one way or another (either by an event log, or modules, or micro services, or you-name-it). I heard the term CQRS more than once. Perhaps its importance is felt more and more now that the entire world is getting wired and complexity and unlikely combinations of projects is growing at a rapid rate. (Internet-of-Things anyone?)
I'll provide a short summary of the Breakout Sessions I followed, and the parts that really struck me as interesting. There's more information on the sessions available at [1].
My chosen schedule:
- Event-sourced architectures with Akka (Sander Mak)
- Reactive programming with Java 8 and Java EE 7 (Martijn Blankestijn)
- Modularity and Domain Driven Design : a killer combination? (Tom de Wolf & Stijn van Den)
- Hand-on-Labs: Hands-on with Java 8 (Urs Peter & Nanne Baars)
- Microservices: the how and why? (Mylène Reiners)
Keynotes
Bert Ertman, after 10 years at the NLJUG, has decided to step back and become a regular attendee of J-Fall. He is the most visible of the people behind J-Fall, as he has opened J-Fall, to the best of my knowledge, every year. Let us hope we shall see more of him and not less.The Keynotes were not actually that interesting. The Keynote of Oracle was more related to awards for proven Java-ness, than what Oracle has planned for the future. The keynote of Oracle was provided by James Weaver, who was also giving a Session last year regarding JavaFX and guitar playing.
The Keynote of ING was surprisingly similar to last year, regarding the banks new way of working with software releases (Agile, Scrum, three releases every week, automated testing, DevOps, etc.)
The Keynote of Salves was interesting, in that it had a high geek-techy-factor, with drones flying, and brainwaves to control them, with pattern recognition in cameras. But the complexity of the setup, and the conference location and The Dreaded Demo Effect got to 'em, which is a shame.
I found the best Keynote to be the one provided by Quintor, a consultancy business started in 2005, regarding a simulated Cluster with Hadoop they set up during one of their "Summer Camps" to brute-force a Shortest Path Algorithm. They needed some penguins to find an igloo between ice blocks and rocks. They created "Willem", a server that was running 20 or so virtual environments, chained in a Hadoop cluster with Apache HBase as work memory and Akka Reactive Streams.
The Sessions
Event-sourced architectures with Akka
A great number of projects are always based around the idea of Shared Mutable State. This shared mutable state makes things very hard to scale, so it has traditionally been pushed to the database as far as possible. Stateless code always scales better, because it can be easily run parallel.It seems to me that event-sourcing is basically based on the database transactionlog that has been in Databases for the past hundreds of years.
The idea is that we store the events that happened, instead of storing only the changes of data. In the latter case we lose a lot of history. In order to keep the side effects out of the way when replaying the event log, these are not re-done.
A simple example is, instead of storing Price and Orders, we store events like PriceChanged or OrderCancelled.
Of course, this presents problems as you have to rerun the event log in order to find out the current situation of the system. This is solved by creating snapshots.
Another problem is the fact that Searching for stuff becomes harder. So it makes sense to have basically two different modules, one module is responsible for changes (and stores events and the likes) and one module is responsible for querying and searching (and searches, for example, over Snapshots).
The abbreviation concerning this is called CQRS, Command Query Responsibility Segregation2. Here the command model is the eventstore (the "journal") and the query model can have several data stores, each containing for example snapshots.
It is possible to have the module that stores events present hooks to the searchmodule to keep the segregated models synchronised.
Some software that is used for the event model and storage of events are:
- Cassandra
- Kafka
- DynamoDB
- MongoDB
- HBase
- HBase
- MapDB
- JDBC
Akka can be used for all this, it provides a new experimental persistence module called appropriately Akka Persistence, that uses event sourcing.
Code can be found on github at https://github.com/sandermak/akka-eventsourcing. The url provided by Sander was http://bit.ly/akka-es, convenient for if the source moves.
One of the advantages, are that events show the business processes a lot better than normal persisted objects do. With events it is much more clear what is happening.
Conclusion is that event sourcing is very applicable to Domain Driven Design.
A word on versioning, a change in event-structure is going to cause a problem for the history (which uses the old event-structure). There are several solutions for this:
- don't change your event structure (which, likely isn't possible)
- use both old and new event-structure at the same time (which, introduces legacy code that needs to be maintained)
- change ALL your events, also the historical ones (which, from a purity standpoint is debatable)
- create a snapshot of the historical (old) events, and continue using new events henceforth (losing a bit of history)
I rated this as the best Session of the day!
Reactive programming with Java 8 and Java EE 7
Most of our programming is run sequentially, which, with the current advent of multiple cores, becomes inefficient (they say).In Java 7 "Futures" were introduced to take care of doing things in parallel using ThreadPools.
A Future is a placeholder for a value that hasn't been set yet, but will (hopefully) be set in the future.
The Java API for Futures is very extensive with a lot of different classes containing a lot of different methods. In the future (my, was that a pun?) there might well come to the front a framework that abstracts away a lot of the nitty-gritty of this highly detailed API. Where the other sessions were light on the amount of code, this one more than made up for them.
Some codepoints:
- ForkJoinPool.commonPool() => suited to the number of cores the JVM is running on.
- Callable -> interface, provides a task to perform.
- ManagedExecutorService -> for externally configurated threadpools for example in an application server like Glassfish. It is a part of JEE 7.
- CompletableFuture -> takes care of Promise Pipelining. These may trigger actions.
Modelling the flow (probably in a flow model) helps a lot, as you lose a lot of visibility on what the code is actually doing.
You tend to define/plan everything up front, with Futures, and then you start the ball rolling, so to speak, with the last call in the chain.
JAXRS 2.0 has the possibility to perform REST services Asynchronously. This can be done, using:
public void retrieve(@Suspended AsyncResponse response, ...
{
response.resume...
}
{
response.resume...
}
HTTP connections are released during requests, this will lead to much better scalability. (HTTP threads can be re-pooled)
His source is on github at martijnblankestijn/reactive-rest-demo.
One diagram that basically says it all, where green is retrieval of customer information, and red and purple are contract data and communication data respectively of said customer:
Modularity and Domain Driven Design : a killer combination?
The idea of this session was to be serious about the standard software design pattern Separation of Concerns, where you have Loose Coupling and High Cohesion.The reason why, is that the Cost of A Change should be predictable, and should not result in slowly escalating costs, regarding lost/unattainable deadlines or high maintenance.
The way to do this is with Functional Modularisation, because a functional change should only impact that functional module.
There should be few hard connections between domains, for example no Foreign Keys or the like.
There should be no query between modules, which is a problem when you wish to do some serious Search Queries. A query between modules and across several modules, causes these modules to be tightly coupled.
So, in other words, cross module searches should be postponed as long as possible.
When this isn't possible, you could use CQRS, Command Query Responsibility Segregation.
Elasticsearch can be used to do the searching.
All the modules can be queried individually by the SearchModule, and the SearchModule uses elasticsearch.
Hand-on-Labs: Hands-on with Java 8
This was fairly interesting. They decided on providing a lab on several features in Java 8: the Lambdas (of course), and the new DateTime API.I've managed to nick their Lab and plan on practicing more of the assignments at home (there wasn't enough time for me at the conference, there were quite many of them).
Microservices: the how and why?
Unfortunately, this was a bit of a miss. I am stuck with no clear idea of what Microservices are exactly, but it seems to follow quite closely in the footsteps of the Session on Modularity.Here are some book titles mentioned:
- Domain Driven Design4 - Erik Evens
- Release It!5 - Michael T. Nygard
- Antifragile6 - Nassim Nicholas Taleb
Characteristics of Micro services:
- small (1000 lines of code)
- do one thing, and do it well (Unix philosophy)
- independent
- language agnostic communication
- highly decoupled
Conway's Law3: organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.
Basically what this means is that, for example, if you have three teams, one for the UI, one for the Middleware, and one for the database, you end up with a software product that uses a UI, a Middleware and a Database. If the entire team works on ONE microservice, you get the absolute best product. It also means you cannot have a standard team. It is SOA (Service Oriented Architecture) done right.
Missed Sessions
I was planning on visiting Hot migration of JSF to cool SPA with smart hacks, but somehow that fell through.Unfortunately, at the end of all that it was 17:00 hours, and I was Conference-tired, so I decided to skip Using Docker to Develop, Test and Run Maven projects.
Conclusion
J-Fall always provides an excellent glimpse in what the Software Community (at that current time) feels is most important in sharing with their colleagues.The main conclusion that I can take away from this J-Fall is that somehow a number of the Sessions was regarding a Separation of Concerns in one way or another (either by an event log, or modules, or micro services, or you-name-it). I heard the term CQRS more than once. Perhaps its importance is felt more and more now that the entire world is getting wired and complexity and unlikely combinations of projects is growing at a rapid rate. (Internet-of-Things anyone?)
References
- [1] NLJUG - J-Fall 2014
- http://www.nljug.org/jfall/2014/
- [2] Martin Fowler - CQRS
- http://martinfowler.com/bliki/CQRS.html
- [3] Wikipedia - Conway's Law
- http://en.wikipedia.org/wiki/Conway%27s_law
- [4] Domain Driven Design
- Erik Evens
- [5] Release It!
- Michael T. Nygard
- [6] Antifragile
- Nassim Nicholas Taleb
- ElasticSearch
- http://www.elasticsearch.org/
Friday, 31 October 2014
getAsModifableSet()
At work we have taken great care to limit access to a Set of values, as elements of the set have constraints between each other, that need to be sustained.
So any call to retrieve a set, is always done using Collections.unmodifiableSet.
I would have been much more at ease, if we had also decided to make the elements of the set Immutable Objects, but alas, this is the status quo.
As it is right now, it's possible to change the attributes of the individual elements, and so break the constraints between them.
The typo in the name is actually in the code, I swear! My grasp of the intricacies of spelling is beyond reproach.
And it returns a very implementation-specific Set as well.
And it's public too.
I'm sure there are always good reasons for these things, but sometimes I'd like to run my head into a wall.
So any call to retrieve a set, is always done using Collections.unmodifiableSet.
public Set<T> getCycles()
{
return Collections.unmodifiableSet(cycles);
}
{
return Collections.unmodifiableSet(cycles);
}
I would have been much more at ease, if we had also decided to make the elements of the set Immutable Objects, but alas, this is the status quo.
As it is right now, it's possible to change the attributes of the individual elements, and so break the constraints between them.
Surprise
Imagine the surprise of my colleague, when, out of nowhere, he found the following method:public abstract SortedSet<T> getAsModifableSet();
The typo in the name is actually in the code, I swear! My grasp of the intricacies of spelling is beyond reproach.
And it returns a very implementation-specific Set as well.
And it's public too.
I'm sure there are always good reasons for these things, but sometimes I'd like to run my head into a wall.
Sunday, 26 October 2014
Covariant return type
“In object-oriented programming, a covariant return type of a method is one that can be replaced by a "narrower" type when the method is overridden in a subclass.”
Since JDK 5.0 it is possible to provide covariant return types in methods of subclasses.[2] Before this release, the Java programming language was invariant with regard to method return types.
Unfortunately, covariance is not possible with method parameters. If you wish to use that, reference [1] has a good explanation of how to do this using Generics.
Example
Scala
In Scala all three are possible, contravariant, covariant and invariant for both method parameters as well as method return types. It is used fairly frequently.For more information, the blog in [3] has some excellent explanation.
References
- [1] Covariant Parameter Types
- https://www.java-tips.org/java-se-tips-100019/24-java-lang/482-covariant-parameter-types.html
- [2] Wikipedia - Covariant return type
- http://en.wikipedia.org/wiki/Covariant_return_type
- [3] Atlassian Blogs - Covariance and Contravariance in Scala
- http://blogs.atlassian.com/2013/01/covariance-and-contravariance-in-scala/
Sunday, 19 October 2014
Glassfish Security Realms
The JDBCRealm
In the Glassfish Administration Console, go to Configurations -> server-config -> Security -> Realms -> and select "New".We're going to choose com.sun.enterprise.security.ee.auth.realm.jdbc.JDBCRealm.
The jaas.context name of the new realm is "jdbcRealm".
A jdbcRealm needs both a user table and a group table. The group table should store both userids as well as groupids. In other words,
- the group table doesn't only contain groups, and
- has a composite primary key, consisting of userid and groupid.
Database Schema
The user table is called "mm_admin" in my case, and the groups table is called "mm_admin_mm_groups".I have created the "mm_groups" table to store extra information for groups, but that table is ignored by the jdbcRealm.
Some serious disadvantages of the jdbcRealm are:
- groups are cached, which means if a user changes groups, the Glassfish Server needs to be rebooted.
- flexibility is about zero, for instance if I have an expiration date attached to my usertable, I'm out of luck.
The flexibleJDBCRealm
The great advantage of the flexibleJDBCRealm is that you can add SQL queries that are run to retrieve passwords and groups.Instructions
Well, according to instructions found here6, you have to:- download the flexiblejdbcrealm-deploy-1.2-all.jar
- put it into glassfish/domains/domain1/lib directory
- change glassfish/domains/domain1/config/login.conf to add flexibleJdbcRealm {
org.wamblee.glassfish.auth.FlexibleJdbcLoginModule required;
}; - reboot the glassfish
- In the Glassfish Administration Console, go to Configurations -> server-config -> Security -> Realms -> and select "New"
- enter the class name org.wamblee.glassfish.auth.FlexibleJdbcRealm
- set the properties, for example like in the image above.
Properties
The properties that can be entered could use a little more explaining compared to what is available at [6].- password.digest
- uses MessageDigest(String algorithm). Possible values are "PLAIN", "MD2", "MD5", "SHA-1", "SHA-256", "SHA-384" and SHA-512.
- password.encoding
- the encoding, I find HEX to be the most useful. Possibile values are "TEXT", "HEX" and "BASE64". A variation of "HEX" is "HEX:<digits>", for example "HEX:40". This padds the beginning of the encoding with zeros to reach the length required. For example MySQL itself always creates passwords using the SHA() function to 40 hex digits also padding the beginning with zeros.
- sql.groups
- select groupid from mm_admin_mm_groups where name in (?)
- sql.password
- select passwd from mm_admin where name in (?) and validuntil > now()
- jaas.context
- flexibleJdbcRealm
- datasource.jndi
- of course, the connection string to the database
Logging
In glassfish/domains/domain1/config/logging.properties, add: org.wamblee.level=FINEST.Application Configuration
My web.xml would look like thusly:<?xml version="1.0" encoding="UTF-8"?>
<web-app version="3.1"
xmlns="http://xmlns.jcp.org/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd">
<session-config>
<session-timeout>
30
</session-timeout>
</session-config>
<security-constraint>
<display-name>Administration</display-name>
<web-resource-collection>
<web-resource-name>Administration REST Resources</web-resource-name>
<description/>
<url-pattern>/resources/administration/*</url-pattern>
<url-pattern>/administration/*</url-pattern>
</web-resource-collection>
<auth-constraint>
<description/>
<role-name>deputy</role-name>
</auth-constraint>
</security-constraint>
<login-config>
<auth-method>BASIC</auth-method>
<realm-name>deputyRealm</realm-name>
</login-config>
<security-role>
<description/>
<role-name>deputy</role-name>
</security-role>
</web-app>
My glassfish-web.xml looks like:<web-app version="3.1"
xmlns="http://xmlns.jcp.org/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd">
<session-config>
<session-timeout>
30
</session-timeout>
</session-config>
<security-constraint>
<display-name>Administration</display-name>
<web-resource-collection>
<web-resource-name>Administration REST Resources</web-resource-name>
<description/>
<url-pattern>/resources/administration/*</url-pattern>
<url-pattern>/administration/*</url-pattern>
</web-resource-collection>
<auth-constraint>
<description/>
<role-name>deputy</role-name>
</auth-constraint>
</security-constraint>
<login-config>
<auth-method>BASIC</auth-method>
<realm-name>deputyRealm</realm-name>
</login-config>
<security-role>
<description/>
<role-name>deputy</role-name>
</security-role>
</web-app>
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE glassfish-web-app
PUBLIC
"-//GlassFish.org//DTD GlassFish Application Server 3.1 Servlet 3.0//EN"
"http://glassfish.org/dtds/glassfish-web-app_3_0-1.dtd">
<glassfish-web-app error-url="">
<security-role-mapping>
<role-name>deputy</role-name>
<group-name>deputy</group-name>
</security-role-mapping>
<class-loader delegate="true"/>
<jsp-config>
<property name="keepgenerated" value="true">
<description>Keep a copy of the generated servlet class' java code.</description>
</property>
</jsp-config>
</glassfish-web-app>
In this file you can create a mapping between security roles and groups.<!DOCTYPE glassfish-web-app
PUBLIC
"-//GlassFish.org//DTD GlassFish Application Server 3.1 Servlet 3.0//EN"
"http://glassfish.org/dtds/glassfish-web-app_3_0-1.dtd">
<glassfish-web-app error-url="">
<security-role-mapping>
<role-name>deputy</role-name>
<group-name>deputy</group-name>
</security-role-mapping>
<class-loader delegate="true"/>
<jsp-config>
<property name="keepgenerated" value="true">
<description>Keep a copy of the generated servlet class' java code.</description>
</property>
</jsp-config>
</glassfish-web-app>
Declarative Security
An example of declarative security using Annotations:@DeclareRoles("deputy")
@RolesAllowed("deputy")
@Stateless
@Path("/administration/worldattributes")
public class WorldattributesBean extends AbstractFacade<Worldattribute>
{
This means the role definition in the web.xml can be removed.@RolesAllowed("deputy")
@Stateless
@Path("/administration/worldattributes")
public class WorldattributesBean extends AbstractFacade<Worldattribute>
{
In the example above, the security role is applied to the entire Bean, so for each method in the bean, instead of on each method that requires the role.
Programmatic Security
Unfortunately, some fine-grained security can only be done with programmatic means, for example determining if a person who has authenticated themselves owns a certain object that needs mutated upon. In my example, this info in a Rest service can be obtained through a SecurityContext:import javax.ws.rs.core.SecurityContext;
import javax.ws.rs.core.Context;
...
@POST
@Override
@Consumes(
{
"application/xml", "application/json"
})
public void create(Worldattribute entity, @Context SecurityContext sc)
{
itsLog.info("create");
final String name = sc.getUserPrincipal().getName();
import javax.ws.rs.core.Context;
...
@POST
@Override
@Consumes(
{
"application/xml", "application/json"
})
public void create(Worldattribute entity, @Context SecurityContext sc)
{
itsLog.info("create");
final String name = sc.getUserPrincipal().getName();
FAQ
- I get WEB9102: Web Login Failed: com.sun.enterprise.security.auth.login.common.LoginException: Login failed: No LoginModules configured for flexibleJdbcRealm?
- You forgot to add the entry to the login.conf as specified above.
References
- [1] Oracle - Security in the Java EE Platform
- http://docs.oracle.com/javaee/7/tutorial/doc/security-intro.htm
- [2] The Java EE 7 Tutorial
- Release 7 for Java EE Platform
- [3] Oracle - Using the JDBC Realm for User Authentication
- http://docs.oracle.com/javaee/6/tutorial/doc/glxgo.html
- [4] Understanding Web Security
- http://java.dzone.com/articles/understanding-web-security
- [5] FlexibleJDBCRealm
- http://flexiblejdbcrealm.wamblee.org/site/
- [6] Installation instructions
- http://flexiblejdbcrealm.wamblee.org/site/documentation/snapshot/installation.html
Labels:
flexibleJDBCRealm,
java,
jdbc,
JDBCRealm,
jee 7,
security,
web applications
Sunday, 12 October 2014
REST and Error Messages
WebApplicationException
In Java REST you can throw a WebApplicationException, which indicates that something went wrong. You can add the HTTP Status to the exception, to indicate what went wrong.throw new WebApplicationException(Response.Status.BAD_REQUEST);
But that amount of information is in most cases too little. Sure, a HTTP Status of 404 (Not Found) is quite clear, but you'd like some more information.Luckily, I found out that we can add an entity in the Response and add the Response to the WebApplicationException.
In fact, it is the most convenient to just create your own subclass of WebApplicationException to handle this automatically.
Your own WebApplicationException
An Error Entity
The entity that gets translated into JSON and passed over the line, in my case called ErrorDetails, can provide all the information you need.The getResponse method is the one that creates the Response, which is used to provide the HTTP Status code and the error payload for the WebApplicationException.
The json generated looks like:
{
timestamp="2014-09-21T21:34:01.831",
errormessage="sdsgsdgsgd was not found.",
stacktrace=””,
user="sdsgsdgsgd"
}
timestamp="2014-09-21T21:34:01.831",
errormessage="sdsgsdgsgd was not found.",
stacktrace=””,
user="sdsgsdgsgd"
}
JQuery
It just took a little effort to retrieve the ErrorDetail object from the Xqhr object in the JQuery Ajax call. This should be done during error processing, as shown in the JQuery Ajax call below.The webError in the code above is a function to parse the JSON containing ErrorDetails object, provided below.
It parses the jqXHR.responseText, to be precise. And if a stacktrace is provided, the details are put into a HTML tag with id "warning". An 'alert' is always provided.
References
- Whatever can go wrong … Error Handling and RESTful Services
- http://djna.wordpress.com/2009/12/07/whatever-can-go-wrong-error-handling-and-restful-services/
- RESTful API Design: what about errors?
- https://blog.apigee.com/detail/restful_api_design_what_about_errors
- How to get the jQuery $.ajax error response text?
- http://stackoverflow.com/questions/1637019/how-to-get-the-jquery-ajax-error-response-text
- Use jQuery to catch and display ASP.NET AJAX service errors
- http://encosia.com/use-jquery-to-catch-and-display-aspnet-ajax-service-errors/
Sunday, 5 October 2014
Getting the Stacktrace
Found this snippet to get the stacktrace as a printable String on StackOverflow[1].
However, this only provides the stacktrace of the exception, not the possible underlying exception as well.
For that, you should look at:
Of course, if you do not wish to reinvent the wheel, there is always org.apache.commons.lang.exception.ExceptionUtils.getStackTrace(Throwable).
Just thought I'd write this down here, I always forget how to do it.
However, this only provides the stacktrace of the exception, not the possible underlying exception as well.
For that, you should look at:
Of course, if you do not wish to reinvent the wheel, there is always org.apache.commons.lang.exception.ExceptionUtils.getStackTrace(Throwable).
Just thought I'd write this down here, I always forget how to do it.
References
- [1] How can I convert a stack trace to a string?
- http://stackoverflow.com/questions/1149703/how-can-i-convert-a-stack-trace-to-a-string
Tuesday, 30 September 2014
Formatting XML in GEdit with Xmllint
This is just to write down my notes.
Extending Gedit functionality with external tools/scripts is apparently possible and quite easy.
Just go to Edit -> Preferences, select tab Plugins and add the plugin "External Tools".
Once that is done, you can add your own tools, by going to Extra -> Management External Tools -> add your own.
It doesn't really get much easier than that.
I tried it by incorporating xmllint, in the following fashion, and it worked like a charm:
Extending Gedit functionality with external tools/scripts is apparently possible and quite easy.
Just go to Edit -> Preferences, select tab Plugins and add the plugin "External Tools".
Once that is done, you can add your own tools, by going to Extra -> Management External Tools -> add your own.
It doesn't really get much easier than that.
I tried it by incorporating xmllint, in the following fashion, and it worked like a charm:
#!/bin/sh
xmllint --format $GEDIT_CURRENT_DOCUMENT_PATH
I did, of course, change some settings, like:xmllint --format $GEDIT_CURRENT_DOCUMENT_PATH
- input
- document
- Output
- replace-document
References
- GEdit - ExternalTools
- https://wiki.gnome.org/Apps/Gedit/Plugins/ExternalTools
Friday, 26 September 2014
JPA: ManyToOne with Composite Primary Key
Had an issue with how to model some classes that were entities and therefore tables in the database.
I thought I'd write a little blog about it, to get my thoughts in order. The subject matter is not that difficult, once you see how it works.
A guild has ranks, and a person who is a member of a guild can be assigned to a certain rank. In the database that would look like follows. (Courtesy of Mysql-Workbench2)
As you can see there are a number of primary keys, and foreign keys.
These collections are not required, but are convenient.
Once again, seeing as a guildrank is used in exactly one guild, it is the reverse of the relation in the Guild, so ManyToOne.
Many users are able to have the same rank in the guild, and therefore this is a OneToMany relation.
One user can have at most one guild, so it is the reverse of the relation in the guild, therefore ManyToOne.
One user can have at most one guildrank, so it is the reverse of the relation in the guildrank, therefore ManyToOne.
It seems the hard part is taken care of by the fact that there is a annotation @JoinColumns that is able to take care of table relations to more than one field at the same time.
Note: You might end up with a warning/error like the following:
I thought I'd write a little blog about it, to get my thoughts in order. The subject matter is not that difficult, once you see how it works.
A guild has ranks, and a person who is a member of a guild can be assigned to a certain rank. In the database that would look like follows. (Courtesy of Mysql-Workbench2)
As you can see there are a number of primary keys, and foreign keys.
- The guild is identified by its name.
- The guildrank is identified by its ranknumber as well as the guild to which it belongs. Therefore the primary key is a composite of both ranknumber and guild.
- The guildrank therefore also has a foreign key constraint to the Guild.
- The user is identified by its name.
- The user can be a member of a guild, though not mandatory. So there's a foreign key reference to the guild.
- The user can have a rank within the guild, though not mandatory. So there's a foreign key reference to the guildrank, by means of the two fields guild and rank.
Aggregation and Composition
We see here an example of both Aggregation and Composition. User and Guild is an example of an aggregation, they are things of themselves but have a relation. Guild and Guildrank is an example of a composition. The guildrank cannot exist without the guild, and they form a parent-child relationship.The guild
First the easy one, the Guild. This is the only entity that is standalone, i.e. not dependent on any other entity. This makes it quite easy. It just contains collections of ranks (One guild has potentially many ranks, so OneToMany) and members (One guild has potentially many members, so OneToMany).These collections are not required, but are convenient.
A Guildrank
As a guildrank has a composite primary key, we require a separate object to store the primary key composite. This is the case for most (all?) ORMs as the 'findByIdentifier' method takes an object.Once again, seeing as a guildrank is used in exactly one guild, it is the reverse of the relation in the Guild, so ManyToOne.
Many users are able to have the same rank in the guild, and therefore this is a OneToMany relation.
GuildrankPK
User
The User can be in a guild and can have a guildrank.One user can have at most one guild, so it is the reverse of the relation in the guild, therefore ManyToOne.
One user can have at most one guildrank, so it is the reverse of the relation in the guildrank, therefore ManyToOne.
It seems the hard part is taken care of by the fact that there is a annotation @JoinColumns that is able to take care of table relations to more than one field at the same time.
Note: You might end up with a warning/error like the following:
Exception [EclipseLink-48] (Eclipse Persistence Services - 2.5.0.v20130507-3faac2b): org.eclipse.persistence.exceptions.DescriptorException
Exception Description: Multiple writable mappings exist for the field [User.guild]. Only one may be defined as writable, all others must be specified read-only.
Mapping: org.eclipse.persistence.mappings.ManyToOneMapping[guildrank]
Descriptor: RelationalDescriptor(User --> [DatabaseTable(User)])
In order to prevent this, either the guild or the guild-part of the guildrank will have to be read-only. This is done for the guildrank part, by specifying insertable = false, updatable = false in the JoinColumn.Exception Description: Multiple writable mappings exist for the field [User.guild]. Only one may be defined as writable, all others must be specified read-only.
Mapping: org.eclipse.persistence.mappings.ManyToOneMapping[guildrank]
Descriptor: RelationalDescriptor(User --> [DatabaseTable(User)])
Note
In general I dislike the use of composite primary keys, and I favour the use of identification by meaningless numbers. That way the name of the guild, for example, is not spread out all over the database in essence duplicating information and making it almost impossible to change. This is also called Database Normalization3.References
- [1] Wikibooks - ManyToOne
- http://en.wikibooks.org/wiki/Java_Persistence/ManyToOne
- [2] MySQL Workbench
- http://www.mysql.com/products/workbench/
- [3] Wikipedia - Database normalization
- http://en.wikipedia.org/wiki/Database_normalization
Friday, 19 September 2014
Maven and PlantUML
This blog explains about integrating PlantUML with Netbeans and Maven. For integrating PlantUML with Netbeans and Ant, see my previous blogpost here.
The blog at [2] explained to me how to add PlantUML to my Maven project, using the special plugin developed by jeluard1.
Just adding the following to the plugins did the trick:
These can be changed by going to your Netbeans Project Properties (right-click your project, select properties) - select "Actions" - select "Generate Javadoc".
Then add the plantuml Maven goal, com.github.jeluard:plantuml-maven-plugin:generate. You're likely to end up with the following:
The blog at [2] explained to me how to add PlantUML to my Maven project, using the special plugin developed by jeluard1.
Just adding the following to the plugins did the trick:
<?xml version="1.0" encoding="UTF-8"?>
<plugin>
<groupId>com.github.jeluard</groupId>
<artifactId>plantuml-maven-plugin</artifactId>
<version>1.1</version>
<configuration>
<outputInSourceDirectory>false</outputInSourceDirectory>
<outputDirectory>${basedir}/target/site/apidocs</outputDirectory>
<sourceFiles>
<directory>${basedir}/src/main/java/</directory>
<includes>
<include>**/*.java</include>
</includes>
</sourceFiles>
</configuration>
<dependencies>
<dependency>
<groupId>net.sourceforge.plantuml</groupId>
<artifactId>plantuml</artifactId>
<version>8004</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>com.github.jeluard</groupId>
<artifactId>plantuml-maven-plugin</artifactId>
<version>1.1</version>
<configuration>
<outputInSourceDirectory>false</outputInSourceDirectory>
<outputDirectory>${basedir}/target/site/apidocs</outputDirectory>
<sourceFiles>
<directory>${basedir}/src/main/java/</directory>
<includes>
<include>**/*.java</include>
</includes>
</sourceFiles>
</configuration>
<dependencies>
<dependency>
<groupId>net.sourceforge.plantuml</groupId>
<artifactId>plantuml</artifactId>
<version>8004</version>
</dependency>
</dependencies>
</plugin>
Netbeans and Maven
In Netbeans you can select Actions on your project to perform. There is a coupling between the action and the goals in Maven that are executed3.These can be changed by going to your Netbeans Project Properties (right-click your project, select properties) - select "Actions" - select "Generate Javadoc".
Then add the plantuml Maven goal, com.github.jeluard:plantuml-maven-plugin:generate. You're likely to end up with the following:
generate-sources javadoc:javadoc com.github.jeluard:plantuml-maven-plugin:generate
References
- [1] GitHub - jeluard/maven-plantuml-plugin
- https://github.com/jeluard/maven-plantuml-plugin
- [2] Smartics - Using PlantUML
- http://www.smartics.de/archives/1313
- [3] Netbeans - MavenBestPractices
- http://wiki.netbeans.org/MavenBestPractices
Labels:
documentation,
integration,
javadoc,
Maven,
netbeans,
plantuml
Friday, 12 September 2014
Moving From Ant to Maven
My project "karchangame" is Ant-based, basically because when you create a new project in Netbeans, the Ant configuration is the default.
This has worked well for a long time, until I decided recently to upgrade some of the libraries that I use. Now, in Ant, you just download the libraries you need and put the jar-files in your classpath.
That works fine if your libraries are not complicated. But I noticed that some of my libraries are now dependant on yet other libraries.
In short, I just spent an hour in getting the libraries I need, then getting the required libraries of those libraries, ad infinitum.
Maven takes care of this whole slog, by putting the responsibility for defining the required libraries for a framework/library squarely on the shoulders of that framework/library.
What I was stuck with was finding the best way of changing my Ant-based project into a Maven-based project.
The easiest way that I could come up with is to create a brand new Maven-based project. The original was a Web Application, so the new Maven project should also be a Web Application. As far as I could tell every possibility for a new ant-based project is also available as a new maven-based project.
And then start moving files over to the appropriate place in the new Maven structure.
I really like the fact that Git actually detects these moves instead of like in the old days, when a move was an explicit delete and create of two non-related files, making you lose your entire history of that file.
The difference in the directory structure is as follows:
You do notice that Maven actually has a more layered structure, whereas Netbeans Ant basically dumps everything in the root.
So, the move basically entailed the following:
The last part was adding plantuml back into the mix. But I'll talk about that in the next Blog.
This has worked well for a long time, until I decided recently to upgrade some of the libraries that I use. Now, in Ant, you just download the libraries you need and put the jar-files in your classpath.
That works fine if your libraries are not complicated. But I noticed that some of my libraries are now dependant on yet other libraries.
In short, I just spent an hour in getting the libraries I need, then getting the required libraries of those libraries, ad infinitum.
Maven takes care of this whole slog, by putting the responsibility for defining the required libraries for a framework/library squarely on the shoulders of that framework/library.
What I was stuck with was finding the best way of changing my Ant-based project into a Maven-based project.
Moving from Ant to Maven
The easiest way that I could come up with is to create a brand new Maven-based project. The original was a Web Application, so the new Maven project should also be a Web Application. As far as I could tell every possibility for a new ant-based project is also available as a new maven-based project.
And then start moving files over to the appropriate place in the new Maven structure.
I really like the fact that Git actually detects these moves instead of like in the old days, when a move was an explicit delete and create of two non-related files, making you lose your entire history of that file.
The difference in the directory structure is as follows:
You do notice that Maven actually has a more layered structure, whereas Netbeans Ant basically dumps everything in the root.
So, the move basically entailed the following:
From ant | To Maven |
---|---|
build.xml | pom.xml |
- | nb-configuration.xml |
nbproject | - |
lib | - (actually stored in your m2 repo) |
src/conf | src/main/resources |
src/java | src/main/java |
web | src/main/webapp |
test | src/test/java |
build | target |
dist/karchangame.war | target/karchangame-1.0-SNAPSHOT.war |
Pom.xml
I only needed to make a few changes to my pom.xml file, in order to get all the dependencies sorted out.JMockit
Needed to add JMockit, or my testcode didn't compile.<dependency>
<groupId>org.jmockit</groupId>
<artifactId>jmockit</artifactId>
<version>1.10</version>
<scope>test</scope>
</dependency>
<groupId>org.jmockit</groupId>
<artifactId>jmockit</artifactId>
<version>1.10</version>
<scope>test</scope>
</dependency>
AntiSamy
AntiSamy to prevent evil hackers from gaining access.<dependency>
<groupId>org.owasp.antisamy</groupId>
<artifactId>antisamy</artifactId>
<version>1.5.3</version>
</dependency>
<groupId>org.owasp.antisamy</groupId>
<artifactId>antisamy</artifactId>
<version>1.5.3</version>
</dependency>
URL Validation
<dependency>
<groupId>commons-validator</groupId>
<artifactId>commons-validator</artifactId>
<version>1.4.0</version>
</dependency>
It's amazing to see Maven automatically download all the required libraries.<groupId>commons-validator</groupId>
<artifactId>commons-validator</artifactId>
<version>1.4.0</version>
</dependency>
The last part was adding plantuml back into the mix. But I'll talk about that in the next Blog.
References
- Apache Maven
- http://maven.apache.org/
- Netbeans - MavenBestPractices
- http://wiki.netbeans.org/MavenBestPractices
Friday, 5 September 2014
Welcome Banners in Linux
Seriously, there are more ways to create a Welcome Banner in Linux than there are to skin a cat.
As a matter of fact, only one of several ways will work on your system. It depends on what kind of system you have.
In my case I wished to automate the process, by referring to a file instead of having to type a new logon banner message every time.
To effect this globally, we'll need to change it in a gconf database. The database used are dependent on who is logged on, but we don't want that.
The file /etc/gconf/2/path will show in which paths the database is consulted. The paths are in order of precedence. This means if an entry is found in one of the first databases, the entry is ignored in one of the latter databases.
By default there is a Mandatory Source, a User Source and a Defaults Source5. They are:
If you wish you can set the logon message manually, using the gconf-editor tool. If you start it up as root, you will be able to select under "File" different profiles, namely the "Defaults" one and the "Mandatory" one.
Create file /etc/dconf/db/gdm.d/01-mysettings:
As a matter of fact, only one of several ways will work on your system. It depends on what kind of system you have.
In my case I wished to automate the process, by referring to a file instead of having to type a new logon banner message every time.
Redhat Enterprise Linux 5
Change your custom.conf1, presumably it can be found somewhere in /etc/gdm/.Redhat Enterprise Linux 6/Ubuntu 12
The new way2 4, using gconftool-2 to set appropriate key-value pairs.gconftool-2 --config-source=xml:readwrite:/etc/gconf/gconf.xml.defaults --type bool -s /apps/gdm/simple-greeter/banner_message_enable true
gconftool-2 --config-source=xml:readwrite:/etc/gconf/gconf.xml.defaults --type string -s /apps/gdm/simple-greeter/banner_message_text "Your-Login-Banner"
gconftool-2 --config-source=xml:readwrite:/etc/gconf/gconf.xml.defaults --type string -s /apps/gdm/simple-greeter/banner_message_text "$(cat /opt/tools/info/message_of_the_day)"
gconftool-2 --config-source=xml:readonly:/etc/gconf/gconf.xml.defaults --get /apps/gdm/simple-greeter/banner_message_text
gconftool-2 --config-source=xml:readwrite:/etc/gconf/gconf.xml.defaults --type string -s /apps/gdm/simple-greeter/banner_message_text "Your-Login-Banner"
gconftool-2 --config-source=xml:readwrite:/etc/gconf/gconf.xml.defaults --type string -s /apps/gdm/simple-greeter/banner_message_text "$(cat /opt/tools/info/message_of_the_day)"
gconftool-2 --config-source=xml:readonly:/etc/gconf/gconf.xml.defaults --get /apps/gdm/simple-greeter/banner_message_text
To effect this globally, we'll need to change it in a gconf database. The database used are dependent on who is logged on, but we don't want that.
The file /etc/gconf/2/path will show in which paths the database is consulted. The paths are in order of precedence. This means if an entry is found in one of the first databases, the entry is ignored in one of the latter databases.
By default there is a Mandatory Source, a User Source and a Defaults Source5. They are:
- xml:readonly:/etc/gconf/gconf.xml.mandatory
- xml:readwrite:$(HOME)/.gconf
- xml:readonly:/etc/gconf/gconf.xml.defaults
If you wish you can set the logon message manually, using the gconf-editor tool. If you start it up as root, you will be able to select under "File" different profiles, namely the "Defaults" one and the "Mandatory" one.
Redhat Enterprise Linux 7
The new new way! People are migrating from GConf (gconftool-2, gconf-editor) over to GSettings3 and dconf (dconf-tool, dconf-editor).Fedora 20
The workaround for Fedora, because the new new way doesn't work6.Create file /etc/dconf/db/gdm.d/01-mysettings:
[org/gnome/login-screen]
banner-message-enable=true
banner-message-text='hostname: wiggins\n“How often have I said to you that when you have eliminated the impossible,\n whatever remains, however improbable, must be the truth?”\n\n- Sherlock Holmes, The Sign of the Four (1890)'
Don't forget to run, to recreate the database with the new settings:banner-message-enable=true
banner-message-text='hostname: wiggins\n“How often have I said to you that when you have eliminated the impossible,\n whatever remains, however improbable, must be the truth?”\n\n- Sherlock Holmes, The Sign of the Four (1890)'
rm /etc/dconf/db/gdm
dconf update
dconf update
References
- [1] Linux: Display a login banner gfor Gnome (GDM) manager
- http://www.cyberciti.biz/tips/howto-unix-linux-change-gnome-login-banner.html
- [2] RHEL6: Login Banners | SecureOS
- http://secureos.wordpress.com/2011/07/21/rhel6-login-banner/
- Wikipedia - GConf
- http://en.wikipedia.org/wiki/GConf
- [3] Chapter 3. GSettings and dconf
- https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Desktop_Migration_and_Administration_Guide/gsettings-dconf.html
- [4] GConf configuration system
- https://projects.gnome.org/gconf/
- [5] Redhat - Introduction GConfiguration Overview
- https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Desktop_Deployment_Guide/ch-intro-gconf-overview.html
- [6] Ask Fedora - How do I disable user list in gdm?
- https://ask.fedoraproject.org/en/question/9875/how-do-i-disable-user-list-in-gdm/
Friday, 29 August 2014
Building Real Software: Don’t waste time on Code Reviews
Article on Code Reviews that explains how to get the most out of code reviews, without waisting time.
Building Real Software: Don’t waste time on Code Reviews: Less than half of development teams do code reviews and the other half are probably not getting as much out of code reviews as they should...
Friday, 15 August 2014
Java Enum values() method
I remember there being a values() method that provides all possible values of an Enumerated Type in Java. I checked the javadoc1 and couldn't find it...
But it exists.
Which is weird.
Luckily, someone else had the same problem and asked it in [2].
I was intriged so I looked up the Language Spec3. Yes, the method is available in each and every Enum class, but it is 'especially' inserted by the compiler and not inherited from the superclass4.
Luckily in the javadoc1, values() and valueOf(String) methods are both mentioned in the description of the valueOf(Class<T> , String) method (nowadays).
But it exists.
Which is weird.
Luckily, someone else had the same problem and asked it in [2].
I was intriged so I looked up the Language Spec3. Yes, the method is available in each and every Enum class, but it is 'especially' inserted by the compiler and not inherited from the superclass4.
Luckily in the javadoc1, values() and valueOf(String) methods are both mentioned in the description of the valueOf(Class<T> , String) method (nowadays).
References
- [1] Javadoc - Enum
- http://docs.oracle.com/javase/8/docs/api/java/lang/Enum.html
- [2] CodeRanch - Values method documented Enum
- http://www.coderanch.com/t/559674/java/java/values-method-documented-Enum
- [3] Oracle - The Java Language Specification (Java 8)
- http://docs.oracle.com/javase/specs/jls/se8/jls8.pdf
- [4] Oracle Java Tutorials - Enum Types
- http://docs.oracle.com/javase/tutorial/java/javaOO/enum.html
Friday, 8 August 2014
Mysql JDBC Issue
Hammered myself into a wall here. Turns out when I create my Java Entities from a Database Table (using NetBeans Wizards), I get the following:
Notice the catalog and the schema in the annotation for Table?
Now, it turns out1 that "schema" is unsupported in MySQL, and "catalog", apparently, is just a fancy way in MySQL of saying "database".
So, if I create a brand new database (call it "newmmud"), create a nice JDBC Connection Pool to it in my Glassfish server, yet this blasted Entity (Log, in the example) will still refer to the database "mmud"!
That's what I call confusing!
Here's the message, for posterity's sake:
@Entity
@Table(name = "mm_log", catalog = "mmud", schema = "")
@NamedQueries(
{
@NamedQuery(name = "Log.findAll", query = "SELECT l FROM Log l"),
@NamedQuery(name = "Log.findByName", query = "SELECT l FROM Log l WHERE l.name = :name"),
@NamedQuery(name = "Log.findByMessage", query = "SELECT l FROM Log l WHERE l.message = :message")
})
public class Log implements Serializable
@Table(name = "mm_log", catalog = "mmud", schema = "")
@NamedQueries(
{
@NamedQuery(name = "Log.findAll", query = "SELECT l FROM Log l"),
@NamedQuery(name = "Log.findByName", query = "SELECT l FROM Log l WHERE l.name = :name"),
@NamedQuery(name = "Log.findByMessage", query = "SELECT l FROM Log l WHERE l.message = :message")
})
public class Log implements Serializable
Notice the catalog and the schema in the annotation for Table?
Now, it turns out1 that "schema" is unsupported in MySQL, and "catalog", apparently, is just a fancy way in MySQL of saying "database".
So, if I create a brand new database (call it "newmmud"), create a nice JDBC Connection Pool to it in my Glassfish server, yet this blasted Entity (Log, in the example) will still refer to the database "mmud"!
That's what I call confusing!
Here's the message, for posterity's sake:
“You're correct. For legacy reasons (including compatibility with ODBC, and SQL Server), JDBC's concept of "catalog" maps to MySQL's concept of "databases".- Mark Matthews
Starting in MySQL-4.1, "schema" is an alias for "database", but it doesn't act like a SQL-standard schema, so we don't support the concept in the JDBC driver.”
References
- [1] Forum - Re: catalog versus schema
- http://forums.mysql.com/read.php?39,137564,137629#msg-137629
Saturday, 2 August 2014
Scrum
I managed to take a Scrum Foundation course (a one-day course to introduce me to Scrum) on Thursday, July 31st 2014 at Xebia1.
The trainer in question was Martien van Steenbergen2.
I'm going to write some things that I picked up on and I found to be important:
If you look at a graph of the adoption of different technological innovations in the US over the last 100 years (television, radio, internet, etc), there are a couple of things worth noting:
The portmanteau chaordic refers to a system of organization that blends characteristics of chaos and order.
The trainer in question was Martien van Steenbergen2.
Introduction
So basically Scrum is one of many implementations of the Agile Software Development Process.I'm going to write some things that I picked up on and I found to be important:
If you look at a graph of the adoption of different technological innovations in the US over the last 100 years (television, radio, internet, etc), there are a couple of things worth noting:
- every innovation follows the well-known S-curve, a few early adopters, followed by the herd of followers, followed by saturation of the market. This is remarkably the same for both old innovations of long ago, and new innovations of today
- the S-curves are becoming more steep, network effect increases quickly, the market becomes saturated much more quickly, i.e. new things get adopted much more quickly and easily
- there are more S-curves in a given period now then there were in a similar period in the past, i.e. the rate of new innovations is increasing
The portmanteau chaordic refers to a system of organization that blends characteristics of chaos and order.
Some characteristics of Scrum
- 10% is bureaucracy, so in a two week sprint, your meetings/standups/etc add up to about a workday.
- we complete what we start
- a stable team is very important
- manage work and processes, but lead people.
- a scrum team consists of 6 people on average, give or take one.
- trying to estimate things correctly right at the beginning, is impossible
- keep the process (scrum) lightweight
- keep it flexible, which means, don't plan too much ahead, things will change
- keep your focus, which means, if you do plan, plan for short periods of time, for example a sprint, and keep your focus during that period. Become flexible once more when the period comes to a close. In effect you are trying to be both flexible and efficient at the same time.
- take responsibility for tasks, instead of getting tasks assigned to you
- feedback is very important, without feedback you do not know if things are working correctly or if things need to be adjusted
- we should talk about work during standups, and think less in terms of "what did I do, what am I going to do".
- history is important, without history we cannot make predictions on the future
- based on history you can make predictions if only one of the following are fixed: deadline or scope. See the Iron Triangle4
Glossary
- sprint3
- a time-box in which we commit to doing a list of things
- velocity
- what tasks can we complete during a sprint
- delta
- a change to a current situation, basically everything is a delta
- product backlog
- a list of all things that need doing
- product owner
- owner, responsible, employer, takes on risk, competence, available
- scrum master
- monitors the performance of the sprint(s), velocity
- development team
- performs work
Quotes
“If you chase two rabbits, you will catch neither.”- Russion proverb, on context/task switching
“If the rate of change on the outside exceeds the rate of change on the inside, the end is near.”- Jack Welch, former CEO of General Electric (on adaptability)
“Big Projects usually fail, regardless of process. The secret to project success is to strongly recommend and enforce limits on size and complexity. These two factors trump all other factors.”- Henrik Kniberg (on limits)
“It doesn't matter who does it, as long as it gets done.”- ? (on team effort)
“Where there is no standard, there can be no Kaizen.”- ? (on continuous improvement)
References
- [1] Xebia - Summer Special Scrum Foundation Training
- http://training.xebia.com/summer-specials/summer-special-scrum-foundation
- [2] Pearl Language
- http://pearllanguage.org/Home
- [3] Scrum.org - Scrum Guide
- https://www.scrum.org/Scrum-Guide
- [4] Wikipedia - The Iron Triangle
- http://en.wikipedia.org/wiki/Project_management_triangle
- Scrumalliance.org
- http://www.scrumalliance.org/
- Scrum.org
- https://www.scrum.org
- Wikipedia - Agile software development
- http://en.wikipedia.org/wiki/Agile_software_development
- Alan Klement - Replacing User Story with Job Story
- http://alanklement.blogspot.nl/2013/09/replacing-user-story-with-job-story.html
- eduScrum - Dutch website on using Scrum in the classroom
- http://eduscrum.nl/
- The getKanban Board Game
- http://www.getkanban.com/
- The New New Product Development Game
- by Hirotaka Takeuchi and Ikujiro Nonaka
Labels:
agile,
chaordic,
DSDM,
iron triangle,
kanban,
lean,
methodology,
scrum,
software development,
sprint
Sunday, 27 July 2014
From Hibernate to EclipseLink
I've decided to try porting my little application from Hibernate to EclipseLink.
I am not currently using a lot of Hibernate specific functionality. The ones that occur to me at the moment is:
Below are the two different (very different!) implementations for Hibernate and EclipseLink. I find the Hibernate one to be more powerful.
Unfortunately, I cannot not set it, as that will trigger the following:
And there is not a convenient way of turning a filter on or off. Right now, the turning on/off is part of the subquery.4
However, I do like the fact that my AdditionalCriteria suddenly works everywhere once I turn it on. I do not have to set specifics on the fields of an Entity. Of course, this does limit the flexibility, but in my case it is not an issue.
Of course, it does tie me to the MySQL implementation, but you cannot have everything.
This was only a warning and the thing does work, but I should heed it.
It is described in Chapter 10.2.5.4. JPQL Input Parameters of the JPQL Language Spec7
I am not currently using a lot of Hibernate specific functionality. The ones that occur to me at the moment is:
- Hibernate Filters
- Hibernate DELETE_ORPHAN
Filters
Filters are, as far as I know, currently not a part of the JPA Specification. Every ORM has its own implementation of dealing with it.Below are the two different (very different!) implementations for Hibernate and EclipseLink. I find the Hibernate one to be more powerful.
Hibernate
Definition of the filter:@FilterDef(name = "activePersons", defaultCondition="active = 1")
package mmud.database.entities;
Using the Filter on Entities:package mmud.database.entities;
import org.hibernate.annotations.Filter;
import org.hibernate.annotations.Filters;
@Entity
@Filters({ @Filter(name = "activePersons") })
public class Person implements Serializable { ...
import org.hibernate.annotations.Filters;
@Entity
@Filters({ @Filter(name = "activePersons") })
public class Person implements Serializable { ...
@Entity
public class Room implements Serializable {
@OneToMany(cascade = CascadeType.ALL, mappedBy = "room")
@Filter(name = "activePersons")
private Set<Person> persons = new HashSet<>();
Enabling the Filter, upon starting a session:public class Room implements Serializable {
@OneToMany(cascade = CascadeType.ALL, mappedBy = "room")
@Filter(name = "activePersons")
private Set<Person> persons = new HashSet<>();
// Hibernate specific
Session session =
((org.hibernate.ejb.EntityManagerImpl)em.getDelegate()).getSession();// JPA1.0
// Session session = getEntityManager().unwrap(Session.class); // JPA2.0
session.enableFilter("activePersons");
Session session =
((org.hibernate.ejb.EntityManagerImpl)em.getDelegate()).getSession();// JPA1.0
// Session session = getEntityManager().unwrap(Session.class); // JPA2.0
session.enableFilter("activePersons");
EclipseLink
EclipseLink uses the AdditionalCriteria2 3 annotation on Entity level.@Entity
@AdditionalCriteria("(:activePersonFilter <> 1 or this.active = 1)")
public class Person implements Serializable { ...
...
...
getEntityManager().setProperty("activePersonFilter", 1); // turns filter on
@AdditionalCriteria("(:activePersonFilter <> 1 or this.active = 1)")
public class Person implements Serializable { ...
...
...
getEntityManager().setProperty("activePersonFilter", 1); // turns filter on
Unfortunately, I cannot not set it, as that will trigger the following:
org.eclipse.persistence.exceptions.QueryException.missingContextPropertyForPropertyParameterExpression(QueryException.java:260)
So I am obligated to turn the filter on or off at the boundary where the entityManager (or EntityManagerFactory) is first called.And there is not a convenient way of turning a filter on or off. Right now, the turning on/off is part of the subquery.4
However, I do like the fact that my AdditionalCriteria suddenly works everywhere once I turn it on. I do not have to set specifics on the fields of an Entity. Of course, this does limit the flexibility, but in my case it is not an issue.
Deleting Orphans
Well, Hibernate was one of the first to implement the Delete-orphan functionality and it took a while for it to become meanstream in the JPA specification. But it's there now, and should be supported by all ORMs.Hibernate
@OneToMany(cascade = CascadeType.ALL, mappedBy = "belongsto")
@Cascade({ org.hibernate.annotations.CascadeType.DELETE_ORPHAN })
private Set<item> items;
@Cascade({ org.hibernate.annotations.CascadeType.DELETE_ORPHAN })
private Set<item> items;
JPA 2.0
@OneToMany(cascade = CascadeType.ALL, mappedBy = "belongsto", orphanRemoval = true)
private Set<item> items;
private Set<item> items;
Sha1
Hibernate
Apparently JPA doesn't have an sha1 function, yet mysql does. Hibernate had no problems with it, so I looked for a solution from EclipseLink5.@NamedQuery(name = "User.authorise", query = "select p from Person p WHERE p.name = :name and p.password = sha1(:password)")
JPA 2.1
It turns out JPA5 has a specific reserved word called "FUNCTION" to support this, that I quite like. It prevents me from having to write a specific MySQL Dialect.Of course, it does tie me to the MySQL implementation, but you cannot have everything.
@NamedQuery(name = "User.authorise", query = "select p from Person p WHERE p.name = :name and p.password = FUNCTION('sha1', :password)")
Notes
I found EclipseLink to be very strict with the interpretation of JPQL.Issue 1 - References
For example, it complains about:SELECT max(id) FROM ItemDefinition i
Wanting something more along the lines of:SELECT max(i.id) FROM ItemDefinition i
Issue 2 - Like statements
Then there's an issue with the following query statement:SELECT s FROM SillyName s WHERE :name like s.name
That gave me the error message "You have attempted to set a value of type class java.lang.String for parameter name with expected type of class java.lang.Boolean". I had to do some voodoo magic to make it work:SELECT s FROM SillyName s WHERE concat('',:name) like s.name
Yikes!Issue 3 - Update statements
Furthermore, I got a warning when creating the following update statement:UPDATE Item i SET i.container = :container WHERE i = :item and i.belongsto = :person
The warning was:“Input parameters can only be used in the WHERE clause or HAVING clause of a query.”Apparently JPQL doesn't allow named parameters (or parameters of any kind) outside a WHERE or HAVING clause.
This was only a warning and the thing does work, but I should heed it.
It is described in Chapter 10.2.5.4. JPQL Input Parameters of the JPQL Language Spec7
Netbeans
I especially liked the fact that in Netbeans, I can mouse over the NamedQueries in the Entities and it tells me in a popup what is wrong.References
- [1] The Java EE 7 Tutorial - Introduction to the Java Persistence API
- http://docs.oracle.com/javaee/7/tutorial/doc/persistence-intro001.htm
- [2] AdditionalCriteria
- http://wiki.eclipse.org/EclipseLink/Development/AdditionalCriteria
- [3] AdditionCriteria
- https://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Basic_JPA_Development/Mapping/Additional_Criteria
- [4] StackOverflow - Is there a way to disable additionalcriteria in EclipseLink?
- http://stackoverflow.com/questions/15847792/is-there-a-way-to-disable-additionalcriteria-in-eclipselink
- [5] StackOverflow - JPA How to persist column with sha1 encryption
- http://stackoverflow.com/questions/7868939/jpa-how-to-persist-column-with-sha1-encryption
- [6] EclipseLink UserGuide - Support for Native Database Functions
- http://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Basic_JPA_Development/Querying/Support_for_Native_Database_Functions#FUNC
- [7] Oracle - JPQL Language References
- http://docs.oracle.com/cd/E17904_01/apirefs.1111/e13946/ejb3_langref.html
Labels:
delete_orphan,
EclipseLink,
filter,
hibernate,
JPA,
jpql,
mysql,
orm,
orphanRemoval,
sha1,
sql
Tuesday, 22 July 2014
Venn Diagrams
At work I seem to be the authority on the presence (or absence) of a colleague of mine who is much requested within the company for his domain model knowledge. Let's call him Hans, for that is his name.
The reason I am an authority is because I always walk with him from the train station to work.
A fact is that we both have long hair.
Hence a colleague of mine assumed that Long hair is a requirement for people who work here and go by train.
Now, we run into a little problem with Venn diagrams. As always, Wikipedia[1] to the rescue.
It's not possible to show all possibilities using regular circles, when the sets increase to more than three. In our case, for four possibilities, we can use ellipses.
The relationship between Venn and Euler diagrams can be displayed as an Euler Diagram.
I'm going to stop now, before I break my brain on the recursion.
The reason I am an authority is because I always walk with him from the train station to work.
A fact is that we both have long hair.
Hence a colleague of mine assumed that Long hair is a requirement for people who work here and go by train.
Venn Diagrams
So we have the following sets:- people who go by train
- people who have long hair
- people who work at my company
Problems
As I pointed out, the people my colleague, Bart, knows who work at my company and travel by train, also happen to have long hair. But the people my colleague knows is just a subset of people at my company. So let's add the set "people who Bart knows". So we have the following sets:- people who go by train
- people who have long hair
- people who work at my company
- people who Bart knows
Now, we run into a little problem with Venn diagrams. As always, Wikipedia[1] to the rescue.
It's not possible to show all possibilities using regular circles, when the sets increase to more than three. In our case, for four possibilities, we can use ellipses.
Euler Diagrams[2]
Venn Diagrams are a Subset of Euler Diagrams, because Euler Diagrams are a bit more flexible when it comes to showing only some instead of all possible intersections.The relationship between Venn and Euler diagrams can be displayed as an Euler Diagram.
I'm going to stop now, before I break my brain on the recursion.
References
- [1] Wikipedia - Venn Diagram
- http://en.wikipedia.org/wiki/Venn_diagram
- [2] Wikipedia - Euler Diagram
- http://en.wikipedia.org/wiki/Euler_diagram
Subscribe to:
Posts (Atom)