Thursday 21 December 2017

CTF training

On the 14th of December, our company decided to hold a CTF training provided by Dennis van Halteren and Joep van Gassel of Nováccent Group1 for us poor software developers.

I had never heard of it before, so I was kind of interested.

Capture the Flag is a competition where different teams compete to each find the most "Flags": issues in especially for the competition buildt pieces of software. Different flags have different scores. The Application you are trying to hack will let you know in different fashions about the flags you have found. Some flags are hard, some are easy, their respective points reflect this.

The CTF training consisted of an introduction into security and how to test it, and the main CTF competition.

One of the good points of the introduction, was regarding some of the tools that are used in security and penetration testing, which I will mention here:

BeEF3
a tool for injecting bad stuff into browsers of users by means of a vulnerable website
Kali Linux4
a Linux OS containing a lot of penetration test tools and security tools by default

The introduction was very general and lacked depth and lasted about an hour. I already learned most of it during a session working with WebGoat from OWASP2 some years previously.

The CTF competition was a lot of fun. We sorted ourselved into groups of two or three software designers. Each group was provided with a key with which to enter the portal from which all the assignments could be started. A dashboard showing the different scores was available on the big screen.

A number of web portals were provided to us, which all contained different challenges for us to overcome:

Some of the security issues to detect were:

  • caching
  • escalating priviledges
  • using left-over files that should have been deleted
  • SQL injection
  • using error messages to gain insight into the backend
  • wrong permissions that allow too much
  • examining web code for vulnerabilities (HTML, JavaScript, CSS)

The intensity was very high, all of us competing in finding the most flags the fastest. The combination of thinking very hard, trying stuff, frustration if it didnt't work, the cooperation and communication between team members, made most of us feel quite exhausted at the end of the day.

There was no followup after the competition, which I felt was a pity. The trainers were available for questions regarding some of the assignments during and after the competition.

References

[1] Nováccent Group
https://www.novaccent.nl/
[2] OWASP WebGoat
https://www.owasp.org/index.php/Category:OWASP_WebGoat_Project
[3] Kali Linux - a free and open-source penetration testing platform
https://www.kali.org/
[4] BeEF - The Browser Exploitation Framework
http://beefproject.com/
Practice CTF List / Permanant CTF List
http://captf.com/practice-ctf/
Wikipedia - Wargame (hacking)
https://en.wikipedia.org/wiki/Wargame_(hacking)

Thursday 14 December 2017

The Dangers of Optional.orElse

Our architect at work explained how to properly use the Optional class, and sometimes it is not easy. I shall explain one of the intricaties in this blog with the aid of Cake, because who doesn't love cake?

Now some people tell me that the cake is a lie1 2. Now, this may or may not be the case. So there may or there may not be cake.

This is basically the definition of the Optional2 class in Java 8.

Optional<Cake> cake;

One of my colleagues is a fan of Eddie Izzard4 5.

Our architect at work presented us the code he encountered of the Optional.orElse. I've changed it a bit by adding more cake.

If you run this program, you'll notice that after you have received a nice cake, you immediately die!

This is due to the fact that the expression in the .orElse is immediately evaluated after the new Cake(). This is very basic Java and what is to be expected.

Unfortunately, we software designers seem to have a blind spot, when it comes to the orElse() method. We automatically compare it to the if-else construction we know and love, and then we assume the behaviour is the same.

It is as if your brain automatically shunts over to the wrong abstraction.

The .orElse() is actually only suitable for constants.

Conclusion

In order to fix the problem, you need to use a lambda. To use a lambda, you need to use a different method of the Optional class, namely .orElseGet().

The code would look as follows:

    cake.orElseGet(this::death);

I had really hoped, that they would have changed the method name to something better. Some notable good examples would have been:

  • "orElseConstant"
  • "orDefault"

References

[1] Know Your Memes - The Cake is a Lie!
http://knowyourmeme.com/memes/the-cake-is-a-lie
[2] Wikipedia - Portal (video game)
https://en.wikipedia.org/wiki/Portal_(video_game)
[3] Oracle Javadoc - Optional
https://docs.oracle.com/javase/8/docs/api/java/util/Optional.html
[4] Wikipedia - Eddie Izzard
https://en.wikipedia.org/wiki/Eddie_Izzard
[5] Youtube - Eddie izzard-cake or death
https://www.youtube.com/watch?v=BNjcuZ-LiSY

Friday 8 December 2017

My wife's new labtop

Well, World of Warcraft has once again got its hooks into me. I plan on reactivating my account. Ever since the proclamation from Blizzard that World of Warcraft Classic is going to be actively developed, I've got the itch.

My wife would like to play as well, but our labtop, though working fine, is having trouble if the settings are higher than 0.

So, the search for an entry level gaming labtop on a tight budget was on!

As I have no experience with buying labtops or computers on a regular basis, time to do some research.

I decided, for an entry level gaming labtop videocard, to shoot for the NVidia GTX 960M or the NVidia GTX 1050.

I hear the processor is not the most interesting/important part of gaming, so a 7th generation i5 would be fine.

Memory is also not that important, as long as it is at least 8 Gigabytes. I do hope to be able to upgrade this, when I decide to do some software development on said labtop.

An SSD is a very nice to have, otherwise you end up on loading screens that take a while in any serious multi-gigabyte game.

brand type proccessor video (mem) hdd/ssd mem/max
Asus RoG RoG GC533VW-FY079T i7 6700HQ 960M (2gb) hdd 8/16
MSI 6L62M 7RDX-1267NL i5 7300HQ 1050 (2gb) hdd/ssd 8/32
Acer Nitro AN515-51-5048 i5 7300HQ 1050 (4gb) hdd/ssd 8/32
Lenovo Legion Y520-15LBKN-80WK00Q4mh i7 7700HQ 1050 (2gb) hdd/ssd 8/16
Asus RoG 6L502VM-FY479T i7 7700HQ 1060 hdd/ssd 8/16
HP Pavilion Power 15-cb001nd i5 7300HQ 1050 (2gb) hdd/ssd 8/16

We decided on the HP, also because my wife's company sells it and we can get a (small) discount.

I would have liked to go for a Republic of Gamers (RoG) labtop, pure for the novelty value and the design, but alas it was not to be.

Originally my wife had something in mind around the 700 euro price tag, but we failed miserably in that category. These gaming labtops start from 899 euros and up.

Saturday 2 December 2017

Automated executing of MySQL/MariaDB scripts

I am running MariaDB and I wish to execute sql scripts without all this hassle of entering my password. Of course this carries severe security risks with it, that we need to be aware of and, if possible, mitigate.

Via the commandline

It is possible to execute sql scripts via the commandline1, but the problem here is that the password you use is visible in the process list. So this is a security risk.

Let's not do this.

mysql_config_editor

I firstly checked out mysql_config_editor2, which enables you to put the password and other options into an encrypted configuration file. But it turns out that MariaDB does not come with that specific tool. The encryption used seems quite weak, and there's an article about the security issues at [3]. There is also the blogpost at [4] giving some details.

So now what?

Well, there is always the plan to use the configuration file .my.cnf5, and you can store your mysql or mariadb password in there and everything would be hunky-dory.

The .my.cnf looks like this:

[client]
password=topsecretpassword

You are no doubt aware that the password is stored in cleartext.

The following security measures should be in place:

  • always make sure the permissions on the file are set to -rw-------
  • create a user in your database with only those permissions that are required by your scripts. In most cases, this is select/update/delete/insert statements.
  • when you are finished with your scripts, it might be a good idea to remove the password from the conf file. I understand that with cron jobs and batch scripts this might not be possible.

There is an example of a my.cnf containing every possible configuration option at /usr/share/mysql/my-large.cnf when you install MariaDB.

References

[1] StackOverflow - How to execute a MySQL command from a shell script?
https://stackoverflow.com/questions/8055694/how-to-execute-a-mysql-command-from-a-shell-script
[2] MysqlManual 5.7 - mysql_config_editor
https://dev.mysql.com/doc/refman/5.7/en/mysql-config-editor.html
[3] MariaDb Blog -
https://mariadb.com/resources/blog/mysql-56-security-through-complacency
[4] Todd's MySQL Blog - Understanding mysql_config_editor’s security aspects
http://mysqlblog.fivefarmers.com/2012/08/16/understanding-mysql_config_editors-security-aspects/
[5] MariaDB - Configuring MariaDB with my.cnf
https://mariadb.com/kb/en/library/configuring-mariadb-with-mycnf/

Thursday 23 November 2017

@SuppressWarnings

I accidentally stumbled upon a blogpost showing all available SuppressWarnings values1. Convenient.

References

[1] all that jazz - List of SuppressWarnings arguments
https://jazzy.id.au/2008/10/30/list_of_suppresswarnings_arguments.html

Thursday 16 November 2017

On Broken Window Theory and Cake

Occasionally, among my varied interests, I enjoy cooking. I find that it is one of mankind's most interesting practical applications of Alchemy, but that is a Blogpost for another day. I recently baked two cakes for my colleagues at work. I had no special reason for doing it besides the joy of cooking.

In this case it provided a perfect example of the Broken Window Theory1.

I put the two cakes on the counter of the cafeteria, as soon as I came in that morning. Two hours later, I noticed that both cakes were still in one piece.

My colleague and I surmised, that this was because the cake was still undamaged. To test this, we both took a piece of cake. With two hours all but a small bit of one cake remained and at the end of the day, the second cake suffered a similar fate.

Note: interestingly enough even a perfect cake will suffer, if it sits next to a cake that is already in the process of being eaten.

On code

Of course, a similar conclusion can be drawn from your code. If your code is a work of beauty, naturally people who need to make changes, will do so with respect for the beauty of the code.

With code that is already showing wear and tear, or code that has originally not been well designed, this process of entropy is increased dramatically.

It is in many ways similar to the design principles of the Boyscout Rule2 of Uncle Bob.

References

[1] Wikipedia - Broken Window Theory
https://en.wikipedia.org/wiki/Broken_windows_theory
[2] O'Reilly Commons - The Boy Scout Rule
http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule

Wednesday 8 November 2017

J-Fall 2017 - Writeup

2 november 2017 has come and gone, and with it J-Fall 2017 has drawn to a close. Time for the write up.

Getting to know the Blockchain in plain Java
Johan Kragt & CĂ©dric van Beijsterveldt
Welcome J-Fall 2017
Bert Jan Schrijver
Democratization of AI: from researchers to any user
Xavier Dupré
From Monolithic to SoC: Architecture & Teams
Guy Rombaut
CQRS Processing events
Martijn Blankestijn
Fostering an evolving architecture in the agile world
Roy van Rijn
From Code Commit to Production within a Day In One of the Most Critical Industries in the World
Gebrian uit de Bulten, Vincent van Kooten
Live Coding Time & State with Sonic Pi
Sam Aaron
Designing for Modularity with Java 9
Sander Mak
Safely Shoot Yourself in the Foot with Java 9
Heinz Kabutz

Afer all that, I felt the need to take a break and therefore did not attend the last presentation.

Getting to know the Blockchain in plain Java

This talk provided a very good basic level of how the blockchain works. It explained it by means of building a very simple blockchain in Java. The implementation mirrored the design choices made by Bitcoin. The presentation contained lots of demos, where you could see the blockchain being built. Based on the demos it became clear what the weaknesses of blockchains are, and what happens for example when an implementation forks.

It was most interesting and easy to follow.

Github repo was available at https://github.com/jcoreNL/Chain.

Democratization of AI: from researchers to any user

The Microsoft keynote was about machine learning, deep learning and Artificial Intelligence and how cloud services can facilitate this with custom made services and setup.

From Monolithic to SoC: Architecture & Teams

The talk focused on the evolution of both the backend and the frontend of applications. Where in the past these two were intertwined, nowadays we see increasingly that the two are separated.

You can find Guy Rombauts website here.

CQRS Processing events

It was a complicated talk, for anyone not familiar with some of the tools, software stack and principles behind CQRS. I hope to jot down some comments that struck me as important during the talk.

Event Sourcing (ES) basically means that all changes to the application state are stored as events. The current state is computed base on (re)playing the events.

Changing the state or the history is basically destruction of information, and is therefore frowned upon.

A good part of the talk was regarding what to do when the need for information changes. This requires some thinking on how to re-create the event stream or add information to the event stream. Some ways to do this:

  • multiple versions - has readside impact
  • upcasting - you can create a version 2 from version 1, this means the reads only need to know the latest version.
  • in place transformation - changes the event store
  • copy and transformation - creates a new event store to hold the new format, facilitates splitting and merging

See with regards to conversion also The Dark Side of Event Sourcing: Managing Data Conversion.

Then there's the difference between push and pull, where pull means periodically updating your read store, and push is registering yourself for new events. Push is therefore quite feasible in case events do not happen often.

“No one wants eventual consistency. It's a necessary evil. It's not cool. It's useful.”
- Jonas Bonér

Also check out what Martin Fowler has to say.

Fostering an evolving architecture in the agile world

It was a very interesting talk of Roy van Rijn, regarding the software application used by the Port of Rotterdam and how it evolved from a monolithic monster to more and more microservices.

“Don't do agile, be agile.”
- Roy van Rijn

“The purpose of bureaucracy is to compensate for incompetence and lack of discipline.”
- James C. Collins

“Schedule irreversible decisions for the last responsible moment, that is, the last chance to make the decision before it is too late.”
- Mary Poppendieck, Implementing Lean Software Development: From Concept to Cash

He also seems to have doubts about a DOD - Definition of Done document. He poses that your responsibility for your code will be replaced by a list. Which means, if the list is followed, it's no longer your problem.

positive/negativevisibleinvisible
positiveFeatureArchitecture
negativeBugTechnical Debt

He also showed us the greatest example of Survivorship Bias I have ever heard!

A general in the army put some grunts to work registering the places on returning airplanes were these were hit by enemy fire. He took this diagram to the airplane designers, and told them they needed more armour on marks on the airplane where the damage was. The airplane designers told him, they would put the armour at the places where no damage was.

Their reasoning was: the diagram had no damage points in those areas, because if your plane gets damaged there, it is no longer in a position to land safely at an airport. It crashes. Which means we need more armour there.

From Code Commit to Production within a Day In One of the Most Critical Industries in the World

A massively interesting talk about the company that is processing billions of financial transactions per day on the internet, Ingenico. About all the different payment systems that are in use around the world, and how to interact with them and what kind of challenges that brings.

Live Coding Time & State with Sonic Pi

Sam Aaron showed us the Sonic Pi, which is basically a Raspberry Pi with software to generate programmatically music live. It was an amazing piece of work, where he showed different styles of music, performed by a simple 10 lines of code. He uses it to show kids in school how programming can be fun.

He also told of the challenges to create the software, make it perform music in parallel in a multi-threaded way. How to keep the music in sync on different architectures with different threading principles and the challenges that this brings.

Check it out at Sonic Pi - The Live Coding Music Synth for Everyone.. Incidentally, he's looking for money to continue the work.

Designing for Modularity with Java 9

An interesting talk, where we are introduced shortly into modules in Java 9, before diving deep into it and seeing how dependencies work, etc.

Safely Shoot Yourself in the Foot with Java 9

The talk was about how to do weird and unsafe things in Java 9, which were already possible in Java 8, but now require some changes.

Java Modules:

  • The Size of the JVM has decreased significantly.
  • Security has been a big plus.
  • Faster performance

Some things to look up:

  • get/setPlain
  • get/setOpaque
  • getAquire/setRelease
  • @Contented
  • Stream.takeWhile(pred)
  • Stream.dropWhile(pred);

He showed how to overwrite the underlying bytearray in a static string, how to write UNIX Shell scripts using the new java jshell. He mentioned that the visualvm memory poker is gone from the JDK, but can still be gotten from github. He mentioned both Java Flight Recorder and Java Mission Control as good replacements. And lots of other things.

References

NLJUG - JFall 2017
http://jfall.nl/

Thursday 2 November 2017

J-Fall 2017 Starts Now!

It is 2 november 2017 and it is time again for J-Fall 2017. I am already on route to the Cinemec in Ede and looking forward to an entire day of finding out what's new in the Java Community.

I plan to focus on a bit of Blockchain, a bit of Java 9 and a bit of Microservices.

References

NLJUG - JFall 2017
http://jfall.nl/

Sunday 8 October 2017

Automatic updates with dnf-automatic

I have installed the package dnf-automatic, to allow for automatic updates to be installed.

dnf -y install dnf-automatic

There's only one configuration file available in /etc/dnf/automatic.conf.

I have set it to:

apply_updates = yes
download_updates = yes
upgrade_type = security

It shows that the timer is active:

[root@ ~]#systemctl status dnf-automatic.timer
● dnf-automatic.timer - dnf-automatic timer
Loaded: loaded (/usr/lib/systemd/system/dnf-automatic.timer; enabled; vendor preset: disabled)
Active: active (waiting) since Tue 2017-09-19 07:27:51 CEST; 2 weeks 4 days ago

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
[root@ ~]# systemctl list-timers *dnf-automatic* --all
NEXT                          LEFT         LAST                          PASSED  UNIT                ACTIVATES
Sun 2017-10-08 08:28:05 CEST  5h 8min left Sat 2017-10-07 08:28:05 CEST  18h ago dnf-automatic.timer dnf-automatic.service

1 timers listed.

References

major.io - Automatic package updates with dnf
https://major.io/2015/05/11/automatic-package-updates-with-dnf/
Linux Audit - Automatic Security Updates with DNF
https://linux-audit.com/automatic-security-updates-with-dnf/

Monday 2 October 2017

Running Java Programs Online

I have been looking around for an online Java IDE (Integrated Development Environment), so I can do my programming whenever whereever.

Now there are a lot of free websites available where you can just paste a bunch of java code and have it run. Usually it means having all your java code (classes and everything) inside what is basically a TextArea in the browser. An example of these are listed here with my general view of them.

At the bottom are two options that seem to go more towards a full IDE.

Java Online Compiler
https://www.javaonlinecompiler.com/
Nice with a Java tutorial to work alongside.
Codingground - Compile and Execute Java Online (JDK 1.8.0)
https://www.tutorialspoint.com/compile_java_online.php
Seems to have a very nice clean layout.
CompileJava
https://www.compilejava.net/
It seems like a very bare-bones version.
ideone.com
https://ideone.com/
Very few features and the advertising gets annoying quick.
Browxy
http://www.browxy.com/
Seems good. The ability to run your program as an applet is a nice touch. Looks a little more impressive than the other options here.
Guru99
http://www.guru99.com/try-java-editor.html
The absolute minimum to run a Java program and the advertising is annoying.
CodeChef
https://www.codechef.com/ide
A very clean interface. Every programming language known to man available at your fingertips. Ability to upload programs. Used for practice and for competition.
remoteinterview.io Online IDED for Java
https://www.remoteinterview.io/online-java-compiler
Seems okay, but it is great for mob programming. You can invite your friends and you can hack together in the same program. Can handle many programming languages. Very nice interface, autocomplete and code colours. Seems to be used for interviewing potential software designers.
JDoodle
https://www.jdoodle.com/online-java-compiler
Seems to have only few features. However, you can add Maven external libraries which is nice.

Codiva

Codiva at https://www.codiva.io is very nice. It is one of the only ones that I could find that had a very low threshold (you can just try it, you can create an account if you like, but you are not required to) and it is one of the few ones where you can actually create different files, instead of putting all your code into a textarea. It also comes with code highlighting and code completion.

Codenvy

Codenvy seems to be an even more professional IDE, where not only do you have a programming environment, but you actually have a Docker image and stuff. It's available at https://codenvy.io.

I shall examine other highend options where a really full fledged IDE is available in your browser in a later blogpost.

Thursday 21 September 2017

Google OnBoard - Google Cloud

“Almost all the successful microservice stories have started with a monolith that got too big and was broken up.”
- Martin Fowler1

On the 20th of September 2017, I enjoyed a full day at Eindhoven HighTech Campus listening to Google explain everything about their Google Cloud Platform (GCP)2. It's the 63rd or so time this workshop was given. (20.000 people 36 cities)

Of course, I needed to write some of the stuff down, so here's the stuff.

I liked the April fools joke shorts that were interspersed in the lecture.

The problem everyone seems to be facing is that everything needs to be done faster, and better and there needs to be more focus on building business value, instead of having to spent time administrating the hardware.

This can be seen in the fact that the most expensive item in your organisation are your engineers, and they need to be able to work as efficient as possible. Google noticed in the past that time spent doing things that ultimately were of little value can not be regained and is forever lost.

When it comes to the evolution of clouds, you see the following waves happen:

1st wave - colocation
your kit, someone else's building, yours to manage.
2nd wave - virtualized data centers
standard virtual kit, for rent, still yours to manage.
3rd wave - intelligent services automated everything
invest your energy in great apps

Funnily enough, I am noticing this same trend in my home hobbyprojects as well. It can be summarised as follows:

1st wave
I had my own computer, but to install it in a data centre, it needed a rack-mount. My hardware in somebody elses data center.
2nd wave
My computer now is a VPS that I rent.
3rd wave
I started moving small apps over to the Google App Engine.

For a lot of software companies (mine included) we are still only on the second wave. The products of lots of software companies do not require some of the advantages of the cloud and it therefore makes no sense to make the effort.

You can just assemble the google "hexagons" that your application is going to need. It takes a little work to find out what Google has and how it can be best applied to your situation. Because, everybody's situation is different.

Why choose google? If you do use google, you use the netwerk Google created (which is fast), to access other Google users (which use the same network), which are therefore only a hub away.

Googles network is divided in regions and zones. A region might for example be Europe, and a zone might be a data center in a specific region, for example West-1a.

The Google hexagons can be roughly categorized into four main groups, as displayed in the following most important sheet:

StackDriver

StackDriver is a company which was bought up by Google and it specializes in "supporting glue" for different components in an application:
  • Logging
  • Monitoring
  • Debug
  • Error Reporting
  • Trace

Philips Hue

As an example of what it can do, Philips was there to explain how they have been using the cloud along with their new Smart-bulbs in their home-automation project called Hue.

It is a long day to listen, luckily with the free trial version3 of the Google Cloud Platform, it was possible to follow the two gentlemen along in their path through the different components of the platform.

There were some lighter notes as well. For example the spot-the-security-guard in pictures of the Google Data Center. The Machine Learning demos were quite fun too.

I can highly recommend the workshop, for anyone who wishes to have an idea about what Google can do to help your application become great. The sheets alone give a good impression.

References

[1] Martin Fowler - MonolithFirst
https://martinfowler.com/bliki/MonolithFirst.html
[2] Cloud OnBoard - Learn how to Build What's Next with Google
https://cloudplatformonline.com/2017-onboard-benelux.html
[3] Google - Free Trial
https://cloud.google.com/free/
Google Developer Group Netherlands
gdgnetherlands.org
Google QwikLabs
google.qwiklabs.com
Measure your latency to GCP regions
http://www.gcping.com/
Coursera - Google Cloud Platform
https://www.coursera.org/googlecloud
Google Certified
https://cloud.google.com/certification/

Wednesday 20 September 2017

Statics in Java - Answer

The output of the program in Statics in Java:
/opt/tools/java/jdk1.8.0_60/bin/java com.mrbear.Test1
Mount Monadnock

So why don't we get a NullPointerException, which most people would expect?

As indicated in the JLS 15.12.4.11, static methods are directly called on the Class, not on the Object, so the presence or absence of the object is entirely irrelevant.

This is one of the most compelling reasons for always using the Class to call static methods on instead of the Object. It prevents ambiguity for us poor software developers.

P.S. When you think about it, "null" in Java is actually a bit weird, as it can be forcibly cast to any java Object.

References

[1] The Java Language Specification, Java SE 8 Edition
https://docs.oracle.com/javase/specs/jls/se8/jls8.pdf

Thursday 14 September 2017

Statics in Java

This one comes straight out of the Java Language Specification1, paragraph 15.12.4.1:
class Test1 { 
    static void mountain() 
    { 
        System.out.println("Monadnock"); 
    } 

    static Test1 favorite()
    { 
        System.out.print("Mount "); 
        return null
    } 

    public static void main(String[] args) 
    { 
        favorite().mountain(); 
    } 
}
What is the output of this program?

References

[1] The Java Language Specification, Java SE 8 Edition
https://docs.oracle.com/javase/specs/

Wednesday 6 September 2017

Method Overloading in Java - Answer

The output of the program in Method Overloading in Java:
/opt/tools/java/jdk1.8.0_60/bin/java com.mrbear.App
Hello World!
Brian Goetz
Object
Object
So, from the tutorials1:
Note: Overloaded methods should be used sparingly, as they can make code much less readable.
Quoting2 from the JLS3:
When a method is invoked (§15.12), the name of the class, the name of the method, the number of actual arguments and the compile-time types of the arguments are used, at compile time, to determine the signature of the method that will be invoked (§15.12.2).

References

[1] Oracle The Java™ Tutorials - Defining Methods
https://docs.oracle.com/javase/tutorial/java/javaOO/methods.html
[2] StackOverflow - Overloaded method selection based on the parameters real type
https://stackoverflow.com/questions/1572322/overloaded-method-selection-based-on-the-parameters-real-type
[3] The Java Language Specification, Java SE 8 Edition
https://docs.oracle.com/javase/specs/jls/se8/jls8.pdf

Thursday 31 August 2017

Method Overloading in Java

package com.mrbear;

/**
 * Hello world!
 *
 */

public class App 
{
    public String getHim(String name) 
    {
        return name;
    }

    public String getHim(Object object) 
    {
        return "Object";
    }

    public static void main( String[] args )
    {
        System.out.println( "Hello World!" );
        App app = new App();
        System.out.println( app.getHim("Brian Goetz"));
        System.out.println( app.getHim((Object) "James Gosling"));
        Object object = "Joshua Bloch";
        System.out.println( app.getHim(object));
    }
}
What does the program above print to its screen?

References

Oracle The Java™ Tutorials - Defining Methods
https://docs.oracle.com/javase/tutorial/java/javaOO/methods.html

Google Takeout

The wife asked how to download all the photos in Google Photos at once, to store on her computer for easy viewing and backups and all that stuff.

It seems there's a Google producttm for that.

Google Takeout1 2 made by an engineering team at Google with a funny name3.

References

[1] Googel Product Forums
https://productforums.google.com/forum/#!topic/photos/sLseLCfw6Pw
[2] Wikipedia - Google Takeout
https://en.wikipedia.org/wiki/Google_Takeout
[3] Wikipedia - Google Data Liberation Front
https://en.wikipedia.org/wiki/Google_Data_Liberation_Front

Tuesday 22 August 2017

Opening Up Java EE

DZone mentioned1 that David Delabassee said on The Aquarium2 that Oracle wishes to move JEE technologies including reference implementations and test compatibility kit to an Open Source Foundation.

For more information on how and why, you'll have to check out the original post of David Delabassee.

References

[1] DZone - Oracle Opening Up Java EE
https://dzone.com/articles/oracle-opening-up-java-ee
[2] The Aquarium - Opening Up Java EE
https://blogs.oracle.com/theaquarium/opening-up-java-ee
AgileJava by Ivar Grimstad
http://www.agilejava.eu/2017/08/18/oracle-opening-up-java-ee/

Wednesday 16 August 2017

NLJUG JCP Event - Java 9

There was a session on Java 9 on Friday, the 11th of August 2017 in Utrecht1. I went. JUG-members across the world would dive into as much detail as possible in a short timeframe during an online session. It was coordinated between the NLJUG2 (Dutch Java Users Group), CJUG3 (Chicago Java Users Group) and VJUG4 (Virtual Java Users Group). The JPoint5 company provided the facilities to follow the session together with other Java people, which was very kind of them.

The session was split up into different parts:
  1. "The Pragmatic Developer’s Guide to Java 9” by Simon Maple (vJUG)
  2. "Java 9 and the impact on Maven projects” by Robert Scholte (Apache Maven)
  3. "Java 9 and Performance increases" by Jonathan Ross (CJUG)
  4. "Java EE 8 update" by Josh Juneau (CJUG)

It was quite convenient that Jonathan Ross happened to be in the neighbourhood, as he's usually found in the neighbourhood of Chicago. Apparently he is also fluent in Dutch. The coincidences were staggering.

I shall recap in short order all the subjects that passed the agenda.
Jigsaw
What the module system in Java brings to the table is the enforcement of boundaries between code. To my mind this is what is necessary to keep us programmers from unheedinly increasing the entropy in the system beyond manageable limits. Classpath will be replaced with Modulepath. We will have to see how that will work out. There is a new module-info.java file for expressing the different modules and the current module uses them.
JShell
The REPL for Java.
ProcessHandler
The "top" or "ps" command for Java.
Optional
It was already introduced in Java 8, but now it has a new stream() function, which isn't eager like map but lazy like streams are supposed to be.
Ahead-Of-Time compilation (AOT)
Causing faster startup times of your java programs, instead of Just-In-Time compilation (JIT). A big requirement for Internet Of Things stuff. (You cannot wait 10 seconds for your doorbell to start up)
CompileControl
Programmer written hints for the compiler
Compact strings
All string objects use UTF-16, which basically fits into two bytes. The vast majority of the strings in applications can be expressed by just one byte using ISO-8859-1/Latin-1. So for most strings, a byte-array makes more sense than a char-array. A special indicator if a string is UTF-8, will be used to decide upon a byte-array or a char-array for storage. The indicator does not increase memory size due to memory alignment.
Indified strings
Concatenating strings using invokeDynamic instead of StringBuilder. This is a major performance boost.
VarHandlers
A way of using primitives without having to wrap them in Objects (for example AtomicInteger). It looks a bit ugly, but it's better than using com.sun.Unsafe.
_ as keyword
Probably a first step for JDK 10 and unused parameters in Lambdas
Private methods in interfaces
Nice.
Try-with-resources small change
You can now use effectively final variables from outside the try block in the try-with-resources statement
Java 5 will no longer be supported by Java 9 compiler
Time to upgrade your ancient programs!
Jonathan Ross knows a lot about performance. I guess he needs it in his job in the Financial Markets.

Robert Scholte talked about Maven being ready for Java 9, even though our applications might not be ready. Java 9 brings some different requirements to the table, because of modules.

One of the big issues that might arise is that those different requirements might make the uptake of Java 9 a great deal slower than was the case for Java 8. Especially if you do not need modules right now.

Josh Juneau talked about JEE 8, the integration Java 8, and the new and upcoming release of the reference implementation Glassfish (with Payara being very close behind).

Other events scheduled.
August 19th, 2017
Virtual Hackday on Java 9 - https://www.meetup.com/virtualJUG/events/240545774/
Monday, October 23, 2017
Brian Goetz at the CJUG - https://www.meetup.com/ChicagoJUG/events/242432217/

References

[1] NLJUG JCP Event together with CJUG and VirtualJUG
https://www.eventbrite.nl/e/tickets-nljug-jcp-event-ism-cjug-en-virtualjug-36387496983
[2] NLJUG
http://www.nljug.org/
[3] CJUG
https://www.meetup.com/ChicagoJUG/
[4] VJUG
https://virtualjug.com/
[5] JPoint
https://www.jpoint.nl/
Java EE Guardians
https://javaee-guardians.io/
Java Public House (Podcasts)
http://www.javapubhouse.com/

Thursday 10 August 2017

Excluding packages in DNF

In the file /etc/dnf/dnf.conf, you can enter which packages need to be excluded.

I like to exclude java packages, until I am ready for them (as they require a reboot of my application server):
[main]
gpgcheck=1
installonly_limit=3
clean_requirements_on_remove=true
exclude=java*
Or for instance:
[main]
gpgcheck=1
installonly_limit=3
clean_requirements_on_remove=true
exclude=kernel* java*
When I am ready to install all packages, regardless of excludes, I can use the commandline below.
$ dnf -y update --disableexcludes=all

References

Fedora 24: Exclude package from update
https://www.hiroom2.com/2016/07/07/fedora-24-exclude-package-from-update/
SysTutorials - Making dnf/yum Not Update Certain Packages
https://www.systutorials.com/1661/making-dnf-yum-not-update-certain-packages/

Thursday 3 August 2017

EclipseLink Logging

In the persistence.xml I have added the following to enable logging for EclipseLink.

Just for my information (so I don't lose the info).

References

EclipseLink 2.5.x. Understanding EclipseLink - Specify Logging
http://www.eclipse.org/eclipselink/documentation/2.5/solutions/tlandgs002.htm#CIHHJIGF
Eclipse - EclipseLink/Examples/JPA/Logging
https://wiki.eclipse.org/EclipseLink/Examples/JPA/Logging

Friday 28 July 2017

Thursday 27 July 2017

Do interfaces inherit from Object class in java?

I was thinking this question suddenly at work...

Quick, Batman! To the StackOverflow1!!!

The answer comes straight from the JSL2.

References

[1] StackOverflow - Do interfaces inherit from Object class in java
https://stackoverflow.com/questions/6056124/do-interfaces-inherit-from-object-class-in-java
[2] JLS Java 8 - 9.2. Interface Members
http://docs.oracle.com/javase/specs/jls/se8/html/jls-9.html#jls-9.2

Thursday 20 July 2017

mount: unknown filesystem type 'exfat'

When attempting to mount a USB drive, I encountered the following error regarding the filesystem exFAT1:
mount: unknown filesystem type 'exfat'
I had to install fuse-exfat as detailed according to [2].
# yum install fuse-exfat
Redirecting to '/usr/bin/dnf install fuse-exfat' (see 'man yum2dnf')

Last metadata expiration check: 0:00:02 ago on Fri May 12 07:06:52 2017.
Dependencies resolved.
================================================================================
Package        Arch       Version             Repository                  Size
================================================================================
Installing:
fuse-exfat     x86_64     1.2.5-1.fc25        rpmfusion-free-updates      40 k

Transaction Summary
================================================================================
Install  1 Package

Total download size: 40 k
Installed size: 71 k
Is this ok [y/N]: Y
Downloading Packages:
fuse-exfat-1.2.5-1.fc25.x86_64.rpm              335 kB/s |  40 kB     00:00    
--------------------------------------------------------------------------------
Total                                           118 kB/s |  40 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Installing  : fuse-exfat-1.2.5-1.fc25.x86_64                              1/1
Mounting after that worked flawlessly.
# mount /dev/sdb1 mydrive
FUSE exfat 1.2.5

References

[1] Wikipedia - exFAT
https://en.wikipedia.org/wiki/ExFAT
[2] Mounting EXFAT formatted pendrives in fedora linux
https://coderwall.com/p/nvwgea/mounting-exfat-formatted-pendrives-in-fedora-linux

Thursday 13 July 2017

Enumerations in EclipseLink

I have a field in the database that does not match with an Enumeration.

So I needed to do a little conversion in EclipseLink, and I didn't know how.

Below is the answer on that one.

References

EclipseLink - @ObjectTypeConverter
https://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Basic_JPA_Development/Mapping/Basic_Mappings/Default_Conversions_and_Converters/ObjectTypeConverter
EclipseLink - EclipseLink/Examples/JPA/EnumToCode
https://wiki.eclipse.org/EclipseLink/Examples/JPA/EnumToCode

Thursday 6 July 2017

Lambdas, New IO, and parsing textfiles in a hurry.

Okay, so I needed to do some parsing of a file containing URLs (which I "wget"-ted) and moving the retrieved files to proper locations.

I decided to write a quick Java program to do this instead of messing around with scripting languages or a Linux bash shell.

It worked very well, and I am rather pleased with the result and Java 8.

It contains the following "new/newer/not-very-old" stuff:
  • a lambda
  • a stream (of Strings)
  • a method reference (used as a lambda)
  • the java.nio.file package (New IO)
One small note: lambdas implement an interface. In this case the forEach requires a lambda that implements the Consumer interface. The Consumer interface does not specify an IOException. Therefore, I am required to catch it here and rethrow it unchecked.

References

[1] Java SE 8 - Official Javadoc
https://docs.oracle.com/javase/8/docs/api/

Thursday 29 June 2017

UML - What do those Arrows Mean?

In PlantUML:
@startuml
abstract class Animal
interface Behaviour
interface Prey
Animal <|-- Lion : inheritance
Behaviour <|.. Animal : realization/implementation
Lion -left-* Pride : composition
interface Grouping
Grouping <|.. Pride
Prey -left-O Lion : aggregation
Water <-down- Lion : uni-directional
Habitat -up- Lion : bi-directional
@enduml
implementation/realization
a dotted line with a closed, unfilled arrow means realization (or implementation). The arrow points to the interface.
inheritance
Inheritance is indicated by a solid line with a closed, unfilled arrowhead pointing at the super class
aggregation
a solid line with an unfilled diamond at the class which uses the other class
composition
a solid line with an filled diamond at the class which contains the other class
bi-directional association
A bi-directional association is indicated by a solid line between the two classes. In the example, the Lion lives in his Habitat, but the Habitat benefits in some way from the Lion as well.
uni-directional association
A uni-directional association is indicated by a solid line between the two classes. The class that knows nothing of the other class, has an open arrowhead pointing to it. A Lion uses Water, but not the other way around.

References

[1] Wikipedia - Class diagram
http://en.wikipedia.org/wiki/Class_diagram
[2] IBM Developer Works - UML basics The class diagram
https://www.ibm.com/developerworks/rational/library/content/RationalEdge/sep04/bell/
What's the difference between Aggregation and Composition?
http://randomthoughtsonjavaprogramming.blogspot.nl/2013/11/whats-difference-between-aggregation.html

Thursday 22 June 2017

MessageBodyWriter not found!

I got the following (unhelpful) message in my server log, when I changed some of my Java classes that are translated to JSON (and vice versa).
Severe: MessageBodyWriter not found for media type=application/json, type=class java.util.ArrayList, genericType=java.util.List

Turns out that I added a specific constructor to one of my Java classes, effectively removing the unspecified Default Constructor that Java always adds.

This default constructor is however essential to the proper working of JSON-Java mapping.

Thursday 15 June 2017

REST-EJB AccessLocalException causes BadRequest(500) instead of Unauthorized(401)

I am using EJBs as REST Services. It works pretty well. I added security on the EJB, by means of security definitions in the web.xml file and appropriate annotations on the EJB (@DeclareRoles and @RolesAllowed).

Unfortunately, when I try to access the methods in the EJB without being properly authorized, I received a 500 BadRequest. Instead I would really like to have a 401 Unauthorized.

I posted a question on StackOverflow1, but I have found the solution2 in the mean time, which I also posted, and will repost here.

It is possible to add an ExceptionMapper to your Application, which can map between an Exception and an appropriate HTTP Response.

Note

My ApplicationConfig has now been expanded with a
resources.add(mmud.exceptions.EJBAccessExceptionMapper.class);
.

References

[1] StackOverflow - REST-EJB AccessLocalException causes BadRequest(500) instead of Unauthorized(401)
https://stackoverflow.com/questions/44351224/rest-ejb-accesslocalexception-causes-badrequest500-instead-of-unauthorized401
[2] RESTfu­­l Jav­a­ wit­h ­JAX­-­­RS 2.­0­ (Second Edition) - Exception Handling
https://dennis-xlc.gitbooks.io/restful-java-with-jax-rs-2-0-2rd-edition/en/part1/chapter7/exception_handling.html
StackOverflow -
https://stackoverflow.com/questions/3297048/403-forbidden-vs-401-unauthorized-http-responses

Thursday 8 June 2017

Casting JSON Object to TypeScript Class

I have implemented some HTTP service for my Angular App using the explanation at [1]. Now in resource [2] it is mentioned that it is important to provide the JSON Object received from the HTTP Service in the constructor of the data model.

I thought I had found a shortcut. I thought that as long as the JSON object received resembled the structure of the TypeScript class, that I could just cast it to the TypeScript class.

This worked fine, until it didn't, and then I got this huge error in my face.

The problem

The problem started appearing when I defined a method in my TypeScript class. Naturally, this method is not available in the JSON Object, and no manner of Casting is going to make it magically appear there.

You get something like:
ERROR TypeError: item.getItemPriceAsInteger is not a function
    at ItemService.webpackJsonp.71.ItemService.updateItem (http://localhost.com/main.bundle.js:811:67)
    at ItemSettingsComponent.webpackJsonp.183.ItemSettingsComponent.update (http://localhost.com/main.bundle.js:508:28)
    at ItemSettingsComponent.webpackJsonp.183.ItemSettingsComponent.saveItem (http://localhost.com/main.bundle.js:480:14)
    at Object.eval [as handleEvent] (ng:///AppModule/ItemSettingsComponent.ngfactory.js:1663:24)
    at handleEvent (http://localhost.com/vendor.bundle.js:13600:138)
    at callWithDebugContext (http://localhost.com/vendor.bundle.js:14892:42)
    at Object.debugHandleEvent [as handleEvent] (http://localhost.com/vendor.bundle.js:14480:12)
    at dispatchEvent (http://localhost.com/vendor.bundle.js:10500:21)
    at http://localhost.com/vendor.bundle.js:12428:20
    at SafeSubscriber.schedulerFn [as _next] (http://localhost.com/vendor.bundle.js:5549:36)

Solutions

There are several solutions available as described in [3, 4, 5].

Chosen solution

I like the one provided in [6]. It uses TypeScript Decorators7. It can be installed as an npm package, according to [8].

To anyone using Java, the solution provided has an uncanny resemblance to JPA annotated Entities or JAXB annotated classes.

I am going to go ahead and try this one out, and see how it works.

I'll provide an update, once I get some results.

References

[1] Angular Docs - HTTP Client
https://angular.io/docs/ts/latest/guide/server-communication.html
[2] Writing a Search Result
ng-book 2 - The Complete Book on Angular Nate Murray, Felipe Coury, Ari Lerner, Carlos Taborda
[3] StackOverflow - How do I cast a JSON object to a typescript class
https://stackoverflow.com/questions/22875636/how-do-i-cast-a-json-object-to-a-typescript-class
[4] StackOverflow - Angular2 cast a json result to an interface
https://stackoverflow.com/questions/34516332/angular2-cast-a-json-result-to-an-interface
[5] Angular2 HTTP GET - Cast response into full object
https://stackoverflow.com/questions/36014161/angular2-http-get-cast-response-into-full-object
[6] Mark Galae - TypeScript Json Mapper
http://cloudmark.github.io/Json-Mapping/
[7] TypeScript - Decorators
https://www.typescriptlang.org/docs/handbook/decorators.html
Ninja Tips 2 - Make your JSON typed with TypeScript
[8] npm - json-typescript-mapper
https://www.npmjs.com/package/json-typescript-mapper
http://blog.ninja-squad.com/2016/03/15/ninja-tips-2-type-your-json-with-typescript/

Thursday 1 June 2017

Bower

Wow. On the website for bower1, they mention the following quote:
“ ...psst! While Bower is maintained, we recommend yarn and webpack for new front-end projects!”2 3
Damn, it's hard to keep up with the advancements in Front-end Land!

References

[1] Bower - A package manager for the web
https://bower.io/
Yarn - Fast, reliable, and secure dependency management.
https://yarnpkg.com/en/
webpack MODULE BUNDLER
https://webpack.github.io/

flexibleJDBCRealm

I have recently changed my security realm settings, and I thought I'd document them here.

I'm still using the flexibleJDBCRealm1 as I've documented in previous blogs2,3.

In the Glassfish administration console, under Configurations -> server-config -> Security -> Realms -> myRealm, the settings are now as follows.
NameValueDescription
datasource.jndijdbc/mydbthe data source to my database
jaas.contextflexibleJdbcRealm
password.digestSHA-512I have upgraded from SHA1 to SHA2, which seems more secure
password.encodingHEX:128See note below
sql.groupsselect groupid from mmv_groups where name in (?)using a database view, makes it easier to change table layout without effecting the securityrealm
sql.passwordselect password from mmv_users where name in (?)same as above

Note

The SHA-512 encoding always creates 128 characters as the hash.

However, in the source code of the flexibleJDBCRealm, this hash is converted from a byte[] into a hexadecimal string by means of a call "new BigInteger(1, aData).toString(16);".

This effectively means that if the byte[] starts with one or more "0"s, these are removed in the BigInteger call leaving you with a hash that is less than 128 characters.

This is why I need to use "HEX:128", instead of just "HEX".

MariaDB/MySQL

The values are easily verifiable in the database.

I can just do a
SELECT SHA2(usertable.password, 512) from usertable where user='mrbear';

It should yield the exact same result as the encryption function of the flexibleJDBCRealm.

References

[1] FlexibleJDBCRealm
http://flexiblejdbcrealm.wamblee.org/site/
[2] Security Realms in Glassfish
http://randomthoughtsonjavaprogramming.blogspot.nl/2016/04/security-realms-in-glassfish.html
[3] Glassfish Security Realms
http://randomthoughtsonjavaprogramming.blogspot.nl/2014/10/glassfish-security-realms.html
[4] Installation instructions
http://flexiblejdbcrealm.wamblee.org/site/documentation/snapshot/installation.html

Thursday 25 May 2017

"this" in JavaScript/TypeScript

I have been struggling with using "this" in JavaScript, ever since I got into that area of programming.

There are lots of warnings on the web, where programmers who are used to a certain behaviour regarding "this" (Like me) can fall into this trap.

I recently found some really good resources that explain it.

There's one1 that explains it a little regarding "this" in JavaScript.

But as I have been writing in TypeScript, I was looking for an explanation that focuses on TypeScript and helps me find the best solution to deal with this. I found that one in [2].

For example

So I've got some code that could use a bit of a look-over.

Here's the troublesome bit.

TypeScript has an excellent Tutorial, which I've used time and again to write my things. One of the pages I've used is the explanation regarding HTTP which you can find at [3].

In it they mention a "handleError" method, which can handle HTTP errors of the PlayerService. Convenient, so I used it. It works.

Next, I wished for the handleError method in the PlayerService that takes care of HTTP connections to notify the ErrorsService. So naturally, I inject the ErrorsService into the PlayerService.

Unfortunately, in the handleError, the ErrorsService is 'undefined'. (See line 30 in the gist below)

It is explained in reference [2] why this is, but I like the following quote:
“The biggest red flag you can keep in mind is the use of a class method without immediately invoking it. Any time you see a class method being referenced without being invoked as part of that same expression, this might be incorrect.”
Now there are several solutions for this described in [2].

The solution below is what I came up with on my own, and I don't really like it, but it works.

Local Fat Arrow

I prefer the solution called the "Local Fat Arrow", which looks like this:
I love it!

References

[1] Mozilla Developer Network - javascript:this
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Operators/this
[2] Github/Microsoft/TypeScript - 'this'in TypeScript
https://github.com/Microsoft/TypeScript/wiki/'this'-in-TypeScript
[3] ts GUIDE - HTTP CLIENT
https://angular.io/docs/ts/latest/guide/server-communication.html

Thursday 18 May 2017

AdditionalCriteria

Small followup of From Hibernate to Eclipselink1 post.

I am not entirely satisfied about the AdditionalCriteria4 thingy. I find it a chore to have to set a parameter on the EntityManager all the time to enable/disable it.

Biggest issue for me is that parameters set on the EntityManager are required. If they are omitted, an exception is thrown when querying.

Current solution in my software:
Turn the AdditionalCriteria on or off by means of a parameter that needs to be set on the EntityManager.

Looks like this:
Setting the parameter activePersonFilter can be done on the EntityManager as follows:
@PersistenceContext(properties =
{
  @PersistenceProperty(name = "activePersonFilter", value = "0"),
  @PersistenceProperty(name = "sundaydateFilter", value = "")
})
private EntityManager em;
Or
entityManager.setProperty("activePersonFilter", 0);

Other solutions

There are some other solutions.
  1. You can remove the additionalCriteria (set it to "") in a subclass, and use the subclass specifically. See [2].
  2. You can customize any mapping in EclipseLink and add the requirements/conditions that you need. See [3].
  3. I could just decide to create a view on the offending database table. Then create two entities. Sounds very similar to the first option.
  4. I could solve the problem in software. Just have EclipseLink not filter anything. (Which is silly, I don't wish for my ORM to get the 1000 persons in the room from the database, if there are say only three persons active.)
  5. I could remove the collection entirely, and retrieve the required Persons using a NamedQuery. (Which is bogus. I like the ORM to deal with this for me, instead of having to do it myself. It's what the ORM is for.)

Customizing a Mapping

I have recently decided to try to customize the mapping specifically in Entities that have collections containing instances of Person class. That way I have more control. See reference [3] on how this works.

It requires a @Customizer annotation.

For instance, in a Room I only wish to see the active persons.

This requires me to define the PersonsFilterForRoom as follows.
"persons"
the name of the field that contains the collection
"room"
the name of the field in the Entity of the collection
"id"
the name of the field in the Room entity that identifies it
It works pretty good.

Note

I also noticed that this way I could have two (Lazy! That's the important bit!) Collections in the same Entity at the same time referring to the same Person. One will contain all Persons and one will contain only the Active Persons.

This is ideal, for instance for Guilds.

Like so:
This way the customizer PersonsFilterForGuild is designed to only work on the activeMembers collection.

I like it!

References

[1] From Hibernate to EclipseLink
http://randomthoughtsonjavaprogramming.blogspot.nl/2014/07/from-hibernate-to-eclipselink.html
[2] StackOverflow - Disable additional criteria only in some entity relations
http://stackoverflow.com/questions/37419406/disable-additional-criteria-only-in-some-entity-relations
[3] Mapping Selection Criteria
https://wiki.eclipse.org/EclipseLink/Examples/JPA/MappingSelectionCriteria
[4] JPA Extention in EclipseLink - @AdditionalCriteria
https://www.eclipse.org/eclipselink/documentation/2.6/jpa/extensions/annotations_ref.htm#additionalcriteria
Customizing EclipseLink JPA/ORM Relationship Joins
http://onpersistence.blogspot.nl/2008/01/customizing-eclipselink-jpaorm.html

Friday 12 May 2017

Eradicating Non-Determinism in Tests

A small blog this time.

At work we sometimes have serious problems with non-deterministic tests.

Martin Fowler mentioned how this can be prevented or dealt with.1

I also noticed that these non-deterministic tests are (almost...) always in the end-to-end tests (or the functional tests, or however you wish to call them).

Martin Fowler also has something to say about those2

References

[1] MartinFowler - Eradicating Non-Determinism in Tests
https://martinfowler.com/articles/nonDeterminism.html
[2] MartinFowler - TestPyramid
https://martinfowler.com/bliki/TestPyramid.html

Thursday 4 May 2017

REST-assured

I am a card-carrying member of the NLJUG0, which provides Java Magazine (not the Oracle one) six times per year.

One of the issues contained an article about REST-assured1.

I have been using SoapUI5 to test my REST services, and that works fine. It's a nice graphical userinterface for me to fiddle with parameters and urls and HTTP requests and even write tests.

I am aware that it is probably possible to integrate SoapUI into my Build Pipeline, but I was really looking for something different. Something more in the line of programming, which is of course my forte. Something I could use in my unit-tests.

REST-assured was exactly what I needed and let me tell you, it's great!

Usage

I will provide an example of how I use it.

As you can see, REST-assured is a very nice DSL (Domain Specific Language) and reads easily.

Some explanation of the above:
log().ifValidationFails()
I wish to log stuff, if the validation/test fails, so I can find out what is wrong. The output looks like
param(name, value)
for setting parameters at the end of the url, like ?stuff=value
pathParam(tag, value)
replaces {tag} in your url with the values. Convenient!
request methods
in the example above, we are using the PUT HTTP Request.
As it is used for testing, it is possible to verify the values afterwards. In the above this is visible as we expect to receive a 204 (NO_CONTENT).

We can extract the response, as is done above, to verify for example the json payload (if there is one) or get cookie values.

In the above example it is essential for the followup calls that we get the JSESSIONID cookie out of the request.

In subsequent REST calls, it is obvious that we need to send along the same JSESSIONID cookie.

See for more information reference [4].

Some notes

I tried to send parameters, but a POST defaults to FORM parameters in the body, but I already have a BODY. But using "queryParam" instead of "param" fixes this problem.

I do enjoy using the "prettyPrint" method on a Response, to properly format a JSON body and dump it to standard output and see what I get. It's nice.

Getting some values out of your JSON formatted response body does require some serious work, though. Needs more research.

I am not entirely sure, I do not enjoy using http status codes like 200 or 204. I prefer something more readable like "NO_CONTENT", but I suppose I can deal with it myself. No biggy.

Update 14/05/2017: I'm also slightly sorry to find out that rest-assured includes Hamcrest. I prefer AssertJ at the moment myself.

Postscriptum

The article in Java Magazine also mentioned WireMock3.

Though I do not use it, it seems excellent for testing the other side of the communications, if you need to test a client that communicates with a server via rest calls.

References

[0] NLJUG
http://www.nljug.org/
[1] REST-assured
Teije van Sloten Java Magazine | 01 2017
[2] GitHub - Java DSL for easy testing of REST services
https://github.com/rest-assured/rest-assured
[3] WireMock
http://wiremock.org/
[4] GitHub - RestAssured Usage
https://github.com/rest-assured/rest-assured/wiki/usage
[5] SoapUI
https://www.soapui.org/
Testing REST Endpoints Using REST Assured
https://semaphoreci.com/community/tutorials/testing-rest-endpoints-using-rest-assured
RFC2616 - HTTP status codes
https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html



Thursday 27 April 2017

Cucumber @After en @Before Hooks

We're using Cucumber at work to write tests, end-to-end-tests that access the user interface of the web application using Selenium.

I recently added an @After hook to a class that contained my StepDefinitions.

However, this @After hook was also called by all other scenarios1, which was not my intention.

As a matter of fact, that @After I added was executing similar code as an @After in another StepDefinition class. I verified that both @After annotated methods were executed for each and every scenario, and they were.

So I decided to move all @After annotated methods into a "GlobalStepDefinition" class, and collaps all of them into one method.

Incidentally, reference [3] shows why we should not have many of these end-to-end tests.

References

[1] GitHub Issues - Before and After methods invoked for unused step definition classes #1005
https://github.com/cucumber/cucumber-jvm/issues/1005
[2] Cucumber - Polymorphic Step Definitions
https://cucumber.io/blog/2015/07/08/polymorphic-step-definitions
[3] MartinFowler.com - TestPyramid
https://martinfowler.com/bliki/TestPyramid.html

Sunday 23 April 2017

Problems with Resolution and My Monitor in Fedora Core 25

Well, my monitor always has been a bit of a problem child, but it worked, so I didn't mind.

I let it bounce once on the floor, but besides some slight discolouring in the lower-right corner, it was fine.

It reports EDID settings that are completely crap, but I got used to ignoring those, using xrandr.

XRandr settings that work for me

The following settings work:
xrandr --newmode "1920x1440" 339.50  1920 2072 2280 2640  1440 1443 1447 1514 -
xrandr --addmode VGA-0 1920x1440
xrandr --newmode "1600x1200" 235.00  1600 1728 1896 2192  1200 1203 1207 1262 -
xrandr --addmode VGA-0 1600x1200
xrandr --newmode "1280x1024"  159.50  1280 1376 1512 1744  1024 1027 1034 1078
xrandr --addmode VGA-0 1280x1024
xrandr --output VGA-0 --mode 1920x1440

Problem

Then I upgraded to Fedora Core 25, and my monitor showed me a handsome 1024x768, which was a disappointment to say the least. (I'm used to 1920x1440.)

Using xrandr gave me the cryptic error message:
bash-4.3$ xrandr --output XWAYLAND0 --mode "1920x1440"
xrandr: Configure crtc 0 failed
After some research I noticed that Fedora Core 25 is the first one to use Wayland1 as the default.

Solution

Switching back to the old Xorg2 fixed my problem.

Checking graphics card

bash-4.3$  lspci -nnk |grep -A 3 -i vga
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper XT [Radeon HD 5770] [1002:68b8]
        Subsystem: ASUSTeK Computer Inc. Device [1043:0344]
        Kernel driver in use: radeon
        Kernel modules: radeon

References

[1] Wayland Desktop
https://wayland.freedesktop.org/
[2] Fedora Project - Switching back to Xorg
https://fedoraproject.org/wiki/Changes/WaylandByDefault
Fedoraforum.org - how to install amd/ati driver on fedora 25?
http://forums.fedoraforum.org/showthread.php?t=312919
AskFedora - How to add a custom resolution to Weyland Fedora 25?
https://ask.fedoraproject.org/en/question/99867/how-to-add-a-custom-resolution-to-weyland-fedora-25/
ArchLinux - Forcing modes and EDID
https://wiki.archlinux.org/index.php/Kernel_mode_setting#Forcing_modes_and_EDID
Bugzilla Redhat - My Bugreport
https://bugzilla.redhat.com/show_bug.cgi?id=1443761

Saturday 15 April 2017

Keyset pagination

In the past I have used the MySQL equivalent of pagination. In other words, the splitting up of a ResultSet into pages of a fixed number of entries, by means of using SQL1.

It looks like the following:
SELECT * FROM tbl LIMIT 5,10;  # Retrieve rows 6-15
For compatibility with PostgreSQL, MySQL also supports the LIMIT row_count OFFSET offset syntax, which I've used in the past.

Performance

Performance is a key point here, as MySQL requires the retrieval of the results in order to determine where the offset starts.

If the table is large, retrieval of pages at the end of the table are going to be extremely slow.

Solution

A better way to deal with this, is to not use an offset, but use the key of the last row of the previous page, and use that in the query for the next page.

Obviously this only works if the resultset is sorted.

For more references that explain this a lot better, see [2] and [3].

References

[1] MySQL 5.7 - 14.2.9. SELECT Syntax
https://dev.mysql.com/doc/refman/5.7/en/select.html
[2] Use the Index, Luke! - We need tool support for keyset pagination
http://use-the-index-luke.com/no-offset
[3] Use the Index, Luke! - Paging Through Results
http://use-the-index-luke.com/sql/partial-results/fetch-next-page

Thursday 6 April 2017

Try Git

To anyone who is absolutely new to the exciting new world of Git1.

There seems to be a little website where you can try Git2, working in a (very very) limited sandbox environment.

What is Git?

If you wish to know what Git is, there are loads of interesting articles on teh interwebs that explain it very well.

But I did find the following explanation in the README provided with the source tar-ball:
The name "git" was given by Linus Torvalds when he wrote the very
first version. He described the tool as "the stupid content tracker"
and the name as (depending on your mood):

 - random three-letter combination that is pronounceable, and not
   actually used by any common UNIX command.  The fact that it is a
   mispronunciation of "get" may or may not be relevant.
 - stupid. contemptible and despicable. simple. Take your pick from the
   dictionary of slang.
 - "global information tracker": you're in a good mood, and it actually
   works for you. Angels sing, and a light suddenly fills the room.
 - "goddamn idiotic truckload of sh*t": when it breaks

References

[1] Git --distributed-is-the-new-centralized
https://git-scm.com/
[2] Try Git
https://try.github.io/

Thursday 30 March 2017

Setting session timeout in Glassfish

People complained that their sessions timed-out too quickly in Glassfish.

I checked and it is set to 30 minutes (default 1800 seconds), just a tad too little.

Increased it to 2 hours (7200 seconds).

Just went to Configurations - Web Container - Session Properties - Session Timeout.

It changes the domain.xml:
<session-properties timeout-in-seconds="7200"></session-properties>

Problem

Of course, this completely and utterly failed to work in my case.

It turns out I already had a session timeout specified in the web.xml.
<session-config>
  <session-timeout>
    30
  </session-timeout>
</session-config>
The session timeout in the web.xml is specified in minutes.

You can also specify it in the glassfish-web.xml file.1
<session-config>
    <session-properties>
        <property name="timeoutSeconds" value="600"/>
        <property name="enableCookies" value="false"/>
    </session-properties>
</session-config>

Precedence

You do need to check which setting takes precedence in your application. It's not clear from the documentation.

References

[1] Glassfish 4.0 Application Deployment Guide
https://glassfish.java.net/docs/4.0/application-deployment-guide.pdf
iT Geek Help - Glassfish web container tuning settings
http://itgeekhelp.blogspot.nl/2009/03/glassfish-web-container-tuning-settings.html
StackOverflow - How to set session timeout in glassfish-web.xml configuration file?
http://stackoverflow.com/questions/33067985/how-to-set-session-timeout-in-glassfish-web-app-glassfish-web-xml-configurat