Six Stages of Debugging

Found this recently:

1. That can’t happen.
2. That doesn’t happen on my machine.
3. That shouldn’t happen.
4. Why does that happen?
5. Oh, I see.
6. How did that ever work?

References

Six Stages of Debugging

http://plasmasturm.org/log/6debug/

New feature in IntelliJ

In the User Interface in the new version of IntelliJ¹, version 2016.3², a new feature popped up called "Parameter hints"³.

My colleague at work has some strong feelings about it.

He feels that the feature provides the lazy programmer with an excuse not to provide properly typed parameters in a method definition (too many String/Boolean/Long and not enough OrderedItem for example). It also would allow the lazy programmer to create methods with 5 parameters or more, as the IDE helps you identify what they are.

Uncle Bob⁴ mentions that anything over three requires some serious soul searching.

My colleague and I both feel that a few of the methods in our software are suffering from an excess of parameters. Even worse, that a lot of these parameters are Booleans that function as switches to alter the behaviour of the method in some way. A code smell, to be sure.

Some solutions, although this blog post does not actually focus on it, is using either a configuration object that contains the parameters⁴ or perhaps a builder.

References

[1] IntelliJ IDEA the Java IDE

https://www.jetbrains.com/idea/

[2] What's New in IntelliJ IDEA 2016.3

https://www.jetbrains.com/idea/whatsnew/

[3] Parameter Hints in IntelliJ IDEA 2016.3 - YouTube

https://www.youtube.com/watch?v=ZfYOddEmaRw&feature=youtu.be

[4] Clean Code, page 40

Robert C. Martin

[5] Introduce parameter object

https://sourcemaking.com/refactoring/introduce-parameter-object

Git and Subversion at Work

At my work we are still using Subversion, and some of my colleagues have started using Git locally, coupled to the remote subversion repository.

I am intrigued in how to get this working on my workstation and this contains some of my thoughts on it.

Advantages

• since we moved to a virtual svn server, it is becoming impossible to do something worthwhile like request the svn history of a file or use "Annotate" in IntelliJ and receive a result within a reasonable time period. It currently takes 15 minutes or longer. A local git repository alleviates this problem immensely.
• the creation of branches can be done locally, which is a big help if I need to work on multiple issues at the same time, and I need to switch repeatedly.
• a local git repository helps me to perform multiple checkins related to the same problem, without inconveniencing any of my colleagues. I can even decide to merge my different checkins together and submit them to subversion as one big changeset.
• the merging is said to be painless and conflicts are very very few, even with rather complex checkins. One of my experiences with Subversion in combination with IntelliJ, is that IntelliJ sometimes is unable to merge it properly and changed code (or duplicated code) is just added to the top of the code file.

Disadvantages

• retrieving the entire svn repository using git (which is the default) is taking a long time (several days). I let it run, but I decided to switch to selecting only a few branches that interest me.
• I still am trying to get to grips with the fact that just checking in isn't enough. After I check stuff into my local repository, I still need to merge it to the master branch and then submit it to subversion.
• it was confusing for a moment, when I noticed that my changes in one branch were automatically visible when I switched to another branch. But this is perfectly normal and happens when you have changes that are yet uncommitted.
• it takes a bit of effort to keep everything straight in my head. Especially in which branch I am currrently working.

JIRA, Git and IntelliJ

I do like the seamless integration between JIRA, IntelliJ and Git. I can just stay in IntelliJ, and search for specific task in JIRA. IntelliJ will automatically create a Git branch with the appropriate JIRA task number as well as a new Change Set.

It is even possible to automatically move the JIRA task to "In Progress", though for now I do not make use of this functionality as I would like to have a little bit more control.

Conclusion

I like git a lot. In the past I was able to submit my changes into Subversion at the touch of a button. The setup now of a local git and a master branch and a subversion remote repository does require several extra steps to check anything in.

But those added steps are worth the effort, compared to all the extra capabilities I gain.

References

[1] git website - git-svn Bidirectional operation between a Subversion repository and Git

https://git-scm.com/docs/git-svn

Obscure little problems

Just a small snippit for now, as I am pressed for time.

It took a little time to find this extremely trivial problem. I overlooked it several times whilst reading through the code.

public void setHarbourCode(HarbourCodeTO harbourCodeTO) {
  this.harbourCode = harbourCode;
}

TO stands for Traffic Object, as this small function is part of a Web interface.

Workshop Continuous Delivery with Docker

Last Monday, on the 14th of November, I got a chance to attend a little workshop on Docker organised by OV Software. It was at the NH Hotel near the facilities of OV Software in The Hague. It is a little far from my place of residence, but I could tag along with a colleague of mine, which helped.

They did a great job of preparing the workshop properly. A VirtualBox image was provided (https://goo.gl/XUM898). The downloaded file can be "imported" (File -> Import Appliance) into the VirtualBox platform (version minimum is 5.0).

Unfortunately, my labtop is not one of the greatest, and the amount of memory required is too much for it. I had to borrow a labtop from work. If this keeps up, I'll have to buy something better for myself one of these days.

Also, unfortunately, I was unable to attend a previous workshop of which this workshop is a followup, so I did miss some background on Docker, and this was my first foray into this new area. Thankfully, a lot can be found on the internet^{1 2}, and I already had plenty of experience with Linux, Java, Maven, and just software development in general.

I had small problems getting it to run, for example:

• I had Vx extentions turned off in the BIOS of my labtop. I needed to turn them on.
• PAE - Physical Address Extentions was turned off. It should be on.
• Somehow, when importing the image, I had turned the machine to 32bits. It's a Linux 64bits image.
• It seems it sometimes helps to turn off your USB in the Virtual Machine settings.

The workshop was provided by an employee of OV Software, Jeroen Peeters, who is an excellent and calm speaker.

One of the first things where I went wrong is thinking that Docker is a kind of Virtualisation. It is not. Docker runs within the same Operating System and the containers also all run in the same Operating System. The images, however, can contain everything the container needs to properly run the image. So, they are simply processes running under the same kernel with the same kernel libraries at their disposal, but it is true that these processes are a little better isolated from the rest of the system.

There are plenty of alternatives to Docker, but Docker seems to be the main menu these days:

• LXC
• Rocket, tooling for developers and systemmanagement
• Lmctfy
• OpenVZ
• Canonical LXD, a Ubuntu thing but more isoltation, so more Virtualisation as it also runs system images

To be more specific, the workshop was regarding Continuous Delivery^{3 4} with Docker.

During the workshop we set up the following Docker containers inside Docker running on Linux Lite 3.0 LTS (Ubuntu based):

• a container of GitLab that held a git repository of a Java Program
• a container of Jenkins
• a maven container (for inside Jenkins)
• a tomcat container
• a load balancer container (Haproxy)
• a testx container⁶

So a webhook in the GitLab repo made sure that jenkins was notified if the source code had changed. Jenkins would therefore start a build pipeline, that would build the application and deploy it on Tomcat and would start testing it using TestX⁵ (a frontend tester tool sauce over protractor and Selenium).

The github repository containing the Java application can be found at https://github.com/jeroenpeeters/docker-workshop-java-todolist/. It is a simple TODO list registration tool.

All this is running in one virtual server. For clustering and the like, you could have a look at Kubernetes, the google solution for running Docker on several machines. Complexity is quite high. But there are also other solutions.

It looks very interesting if you are interested in using Continuous Delivery. I do know that System Management (where I work) is very apprehensive about it. I think they fear to lose control on what Developers can do. We are already working with it in some fashion, but it needs to be improved.

References

[1] Docker

https://www.docker.com/

[2] Docker - Installation on Fedora

https://docs.docker.com/engine/installation/linux/fedora/

[3] Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation

https://www.amazon.com/dp/0321601912?tag=contindelive-20

[4] MartinFowler.com - ContinuousDelivery

http://martinfowler.com/bliki/ContinuousDelivery.html

[5] TestX

https://github.com/testxio/testx

[6] TestX Docker Image

https://hub.docker.com/r/testx/protractor/

Nginx - Docker for Java Developers

https://www.nginx.com/resources/library/docker-for-java-developers/

The Docker Book - Containerization is the new virtualization

https://www.dockerbook.com/

Jenkins

http://www.jenkins.io

Simpson's paradox

“There are three kinds of lies: lies, damned lies, and statistics.”

During my travels across the Internet, I recently stumbled upon a paradox that exemplifies the quote¹ above.

It is called the Simpson's Paradox² and I had never heard of it before.

From what I can gather, it means that data that is aggregated might point in the exact opposite direction compared to the same data partitioned by a certain type/relation.

It depends on the situation which of the two should be preferred, in order to extract proper conclusions from the data.

The Wikipedia entry in the references has some very good examples of Simpson's Paradox and should be required reading for any budding statistician.

References

Wikipedia - Lies, Damned lies and statistics

https://en.wikipedia.org/wiki/Lies,_damned_lies,_and_statistics

wikipedia - Simpson's Paradox

https://en.wikipedia.org/wiki/Simpson%27s_paradox

JFall 2016 - Writeup

“Knowledge is the only treasure that increases when you share.”

- African saying.

This blog post is my writeup and evaluation of the sessions I attended at J-Fall 2016¹.

CQRS and Event Sourcing with Lagom

Unfortunately I was rather late in attending this seminar. It caused me some problems with understanding the matter. I gather it is about event sourcing and CQRS with Lagom.

Hands-on Lab: 12 things you can do better with Java8 - Jakub Marchwicki

Wow, there was a severe shortage of places to sit (properly) and headphones. He provided a github repo². I decided to leave early because of these issues. I hope to get some info from the github repo.

Sharing 2 years experience using Scala in a real project by a Java team

It did mean I could attend this session for a bit. Unfortunately, my knowledge of Scala is limited, but it was possible to follow the talk, except for the details.

Though there are some rumours that the takeoff of Scala isn't as big as it once was, the real project proved that with excellent programmers you can build a better application in Scala than in Java.

It does take some serious getting used to, though.

Java9 and the impact on Maven projects

A bit dull. Seems there are only a few little problems with using Java 9. Big problems were fixed right quickly.

Note: new jar format, special classes for different JDKs in different directories. src/main/java8 src/main/java9 etc. This is called Forward compatibility.

Modularity dependencies are hard.

Java - The Ecosystem Awakens

Sharat Shander mentioned that Java is now 21 years old already. The three top (new!) speakers of Jfall 2016 are invited to JavaOne.

Rise of the Machines - Automate your development

Sven Peters, Atlassian Evangelist, had a very entertaining seminar about Bots in Software development in general, and how they can make our lives easier. It all started with the advent of Hudson, the automated building environment, if I may call it that.

Coding Bots

JIRA should follow our versioning systems "Don't branch from a red branch". "Invite right reviewers" "Two approvals minimum before merge is possible"

Test Bots

Developers don't write bad code on purpose -> Developers are trying to solve problems. Calling Dr. Code -> Hall of Shame/Hall of Fame Hallelujah - Functional Test RunManager Flaky Test Detector -> test in Quarantaine -> create issue

Ops Bots

No more throwing stuff over the wall pagerduty : Faster feedback loop Own your code -> from end to end

Service Bots

Hercules -> scans (log) files and has system knowledge. Confluence has machinereadable bit for hercules. Hercules suggests that page. Hercules learns from response of user/firstlinesupport.

Robots can only do so much. They are very good at automating simple tasks.

Creating customer value is an art, however, and is hard to automate. Therefore humans will always be required.

Treat your bots well.

Delivering Better and Faster Microservices and Mobile Apps with the Cloud

Bruno Borges was very engaging in telling the tale. A shame that the Technology in the room didn't actually help, especially the wireless.

Software helps companies to Innovate.

Oracle has a presence on github⁵.

Documentation avoidance for developers - Peter Hilton

Peter Hilton³ was tremendously entertaining in explaining how we can write as little documentation as possible, and get away with it.

The half life of a team is 3.1 years. After that period, half your team has moved on to something else, and the knowledge they have is gone. This in stark contrast to code, which has a half life of about 15 years.

What you need is constructive laziness, which means thinking hard on how to do as little as possible.

Excuses may help, things like "It's on the Wiki". Tell a person who is leaving the team to write everything down before he's gone doesn't work. But asking questions and posting the answers into a wiki will help.

Pair programming works. A person who writes horrible/complex code will write horrible/complex documentation.

Better naming and type safety helps.

Spotify and Github have a very well documented API.

A good term is "minimum viable documentation".

Java EE Next

David Delabassee mentioned the release deadlines for Java EE 8 (2017) and Java EE 9 (2018).

They seem to be more geared towards microservices and asynchronous behaviour.

JAX-RS 2.1

server sent events

Servlet 4.0

HTTP/2 drop-in replacement server push

CDI 2.0

more geared towards outside an application server

JDF 2.3

small changes

JSON-B 1.0

JAXB for JSON.

References

[1] J-Fall 2016

http://www.nljug.org/jfall/2016/

[2] GitHub.com - JFall Java 8 Lab

https://github.com/kubamarchwicki/jfall-java8-lab

[3] Peter Hilton

http://hilton.org.uk

[4] Tom Preston-Werner - Readme driven development

http://tom.preston-werner.com/2010/08/23/readme-driven-development.html

[5] GitHub - Oracle

https://github.com/oracle

The Martingale using Lambdas

I created a small blog post some time ago regarding the Martingale System of Gambling.

In it, a programming example is available, which could be rewritten using the Lambda style in Java 8.

I am going to attempt doing just that and post my results below.

ForEach

When I was following the MOOC course of Oracle¹, Simon Ritter quite emphatically mentioned trying not to use the foreach method when not required.

But let us try it now, as a seemingly perfectly reasonable first step on a slippery slope to hell.

What we need is a stream of random numbers, and then run the foreach on it.
So far, so good.

Still doesn't look very much better, and perhaps even a little bit worse.

Mapping

Let's try to make it better.

For this we are going to use an "Account" class. Like this:

This class will be used in the Lambda, like so:

Conclusion

Lambdas can make your code look a lot cleaner. However, they can never replace all the loops in your code, as witnessed in the example above.

I'd really like to use collect or reduce instead of the forEach in the example, but I don't think I can.

But if anyone has any suggestions on how to fix it, please tell me.

For the source code to the "Bet" class, check out https://gist.github.com/maartenl/a804f4ac435f491b57c7ae810d5fd577.

Peek and Debugging

Peek is very valuable if you wish to do some debugging of your new Lambdas.

Using the following:

new Random()
           .ints(0, 37)
           .peek(System.out::println)
           .mapToObj(x -> Bet.getBet(x))
           .peek(System.out::println)
           .forEach(x
                   ->
                   {
                     if (x == Bet.RED)
                     {
                       account.add();
                     } else
                     {
                       account.subtract();
                     }
           });

You get some nice output to see what's happening:

10
BLACK
29
BLACK
11
BLACK
19
RED
26
BLACK
34
RED
34
RED

References

[1] Oracle - Announcing: JDK 8 MOOC: Lambdas and Streams!

https://blogs.oracle.com/javatraining/entry/announcing_jdk_8_mooc_lambdas

[2] The Java™ Tutorials > Collections > Aggregate Operations - Reduction

https://docs.oracle.com/javase/tutorial/collections/streams/reduction.html

Wikipedia - Roulette

https://en.wikipedia.org/wiki/Roulette

Potential heap pollution via varargs parameter

Reifiable versus Non-Reifiable

Type erasure is an important "feature" of Generics in Java. It means generics are not available at runtime, as they are effectively removed during compiling.

The reference in [1] has a much better explanation.

To quote:

“A reifiable type is a type whose type information is fully available at runtime.”

“Non-reifiable types are types where information has been removed at compile-time by type erasure.”

Generics versus Arrays

In Java, generics are non-reifiable and arrays are reifiable.

Problems can occur when we combine these two together.

Combining these two together can happen when using the varargs construction in Java.

The reason for this is that the varargs way of using method parameters is translated within the method as a array.

This can cause Heap pollution² when combined with Generics.

As the compiler doesn't know when this happens (it depends on how the method deals with it), it throws out the warning.

Hence the need for the @SafeVarargs³ annotation for those methods where the software designers are certain the problem does not occur.

References

[1] Non-Reifiable Types

http://docs.oracle.com/javase/tutorial/java/generics/nonReifiableVarargsType.html

[2] 9.6.3.7. @SafeVarargs

http://docs.oracle.com/javase/specs/jls/se7/html/jls-9.html#jls-9.6.3.7

[3] Oracle JavaDoc - SafeVarargs

http://docs.oracle.com/javase/7/docs/api/java/lang/SafeVarargs.html

StackOverflow - Potential heap pollution via varargs parameter

http://stackoverflow.com/questions/12462079/potential-heap-pollution-via-varargs-parameter

The Joel Test

I found a link regarding a job opportunity at a Banking establishment¹. (I don't know how long that link is going to stay current, sorry about that.)

They included a little list of the software practices that they follow. Apparently it is called the "Joel Test"².

So I thought I'd mention it here, for those interested about comparing companies.

References

[1] StackOverflow - Mid-Level Java Developer at Leading Sustainable Bank

http://stackoverflow.com/jobs/122285/mid-level-java-developer-at-leading-sustainable-triodos-bank?med=clc&ref=small-sidebar-tag-themed-java

[1] The Joel Test: 12 Steps to Better Code by Joel Spolsky

http://www.joelonsoftware.com/articles/fog0000000043.html

Window Shades Buttons

At work, we have window shades on the outside of the office building that we work in. Inside there are buttons (visible in the picture on the left) that we can use to regulate the lowering and the raising of the window shades.

As a software engineer there's a major mismatch regarding the buttons. A zoomed in version is available below:

The buttons are binary (on or off). There's two buttons, one for up and one for down. Purely functionally it means there are four different options for the buttons to be in:

up is off and down is on

the window shade will travel down until it covers the entire window

up is on and down is off

the window shade will move up, until the entire window is free

up is off and down is off

the window shade will not move

up is on and down is on

the window shade will not move

Clearly there are two combinations possible, that cause the same result. Namely, the window shade will not move.

So on some mornings, I need to turn "down" on to drop the shades and on some days I need to turn "up" off to drop the shades. And I frequently get it wrong.

This extra difficulty could have been eliminated with a three state button, that has the default state of "will not move" and "up" and "down" for special states.

I know it seems trivial, but it is the little things that get on some peoples nerves the most.

Specification By Example

“The formulation of the problem is often more essential than its solution, which may be merely a matter of mathematical or experimental skill.”

- Albert Einstein

The image on the left is a Dutch proverb, meaning "Setting a good example to follow". It exemplifies the Specification by Example idea.

In short, I managed to follow a one-day course on Specification By Example¹, by InfoSupport² at my place of work (which was convenient) along with a collection of my colleagues.

It seems that in the past there were different terms for the same thing, for example ATDD (Acceptance Test Driven Development) and BTDD (Behaviour Test Driven Development). Martin Fowler shared all these under the heading of Specification by Example.

The main goal here is to create living documentation, documentation that evolves along with the software and is therefore always up to date. It also provides a common domain language that can be used by everyone in the company, be it business analysts, software developers, project managers, stakeholders, product owners, etc.

Once proper examples are created using this domain language, these examples can function as the templates for the creation of tests. At work we use Gherkin and Cucumber⁴ to get these examples written down.

Along with the course I was provided with the book³ on Specification By Example, and I look forward to reading it.

References

[1] Trainingen - Driving Development by Example - InfoSupport

https://training.infosupport.com/trainingen/driving-development-by-example

[2] InfoSupport

http://www.infosupport.com/

[3] Specification by Example - How Successful Teams Deliver the Right Software

Gojko Adzic

[4] Wikipedia - Cucumber (software)

https://en.wikipedia.org/wiki/Cucumber_(software)

Impact Mapping

https://www.impactmapping.org/

Jeff Patton, Product Manager, Agile, Lean, UX and Product Design Evangelist

http://jpattonassociates.com/

Final project for DEV201x Introduction to TypeScript on edX.org

I have just now completed the course on typescript¹ available at edX².

The good part about the course is that it is a decent introduction into TypeScript, and also provides information on why typescript was invented in the first place. However, that is basically all it is. To become adept at TypeScript, besides knowing TypeScript, it is essential to have a pretty good grasp of JavaScript.

It was a good course, and helped me a lot, but it is a small course indeed.

edX vs. Coursera

This is also the first time that I got to try edX² as I previously only had experience with Coursera³. I don't think it is fair to compare the two, as it very much depends on which course you take. I do think the web interface of edX, in total, could use a little more polishing up, as some of the user interface quirks were hard to understand.

Coursera, I think, is a bit more polished and provides a better experience.

Final Project

The Final Project consisted of a small web application that displays information regarding Painters. I basically did the bare minimum, as I find it hard to find the time to do anything constructive after working hours. (What with life and everything).

What helps is the little IDE provided for free by Microsoft called Visual Studio Code⁴.

The animated gif below shows the user interface of the little web application written in TypeScript⁵.

Click on it for a bigger and better view.

References

1. Microsoft: DEV201x Introduction to TypeScript

https://courses.edx.org/courses/course-v1:Microsoft+DEV201x+1T2016/info

2. edX | Free online courses from the world's best universities

https://courses.edx.org/

3. Coursera : Online Courses From Top Universities

https://www.coursera.org/

4. Microsoft - Visual Studio Code

http://code.visualstudio.com/

5. Typescript Language

http://www.typescriptlang.org/

J-Fall 2016

Registration for J-Fall 2016 has recently been opened.

It will take place on Thursday, November 3th 2016 from 08:00 onwards. Once again it takes place at the CineMec Cinema at 150 Laan der Verenigde Naties, 6716 JE Ede. It is the biggest Java Conference of the Netherlands and is organised by the NLJUG.

I have chosen to follow the following sessions:

• No excuses: switch to Kotlin - Thijs Suijten
• Hands-on Lab: 12 things you can do better with Java8 - Jakub Marchwicki
• How Kids Code and How We Know - Efthimia Aivaloglou
• Hands-on Lab: Adopt-A-JSR: aan de slag met Java 9 (JSR 379) - Maurice de Chateau & Eelco Meuter
• Documentation avoidance for developers - Peter Hilton
• One application to rule them all - Tom Eugelink

Hope to see you there!

References

[1] NLJUG - Nederlandse Java Users Group

http://www.nljug.org

[2] J-Fall 2016

http://www.nljug.org/jfall/2016/

Running java programs on the web

StackOverflow, the website I use frequently when I have questions about the art of Software Design, recently started a new branch called Documentation¹.

It is in Beta at the moment, but it seems already to have some valid content².

Ideone

The Documentation section of StackOverflow seems to make extensive use of Ideone³, which I had never heard of before now.

From their website:

“Ideone is an online compiler and debugging tool which allows you to compile source code and execute it online in more than 60 programming languages.”

Besides the unfortunate large amount of adds surrounding your HelloWorld code, it seems to work pretty well, if you quickly want to try something in the Programming language of Your Choice.

References

1. StackOverflow - Documentation - The Tour

http://stackoverflow.com/tour/documentation

2. StackOverflow - Documentation - Java Language - Arrays

http://stackoverflow.com/documentation/java/99/arrays/404/creating-and-initializing-arrays#t=201609011104443665439

3. ideone.com

https://ideone.com/

I'm Back!

Sorry for the extended time I've been away. Truth is there is neither much to report on, and there's a noticable lack of motivation from my end, besides which I am very busy with other things in my life.

And I was on vacation.

I'm back now, and I hope to add some more articles to this blog in the coming weeks.

Regards, Mr. Bear.

Creating special F dependent on class A

Current situation:

New situation required:

We don't wish to touch the intervening classes, as they are called from different parts of the application for different reasons.

Solutions:

1. provide a boolean flag all up/down the chain
2. provide a "nicer" boolean flag, for example an enum or a Factory that determines which f should be created
3. use a static flag in f2, in class a2 just set the flag appropriately -> completely unthreadsafe.
4. use the ThreadLocal storage to store the boolean in a/a2 and read it in f/f2.
5. in e -> call up the stacktrace and check for the existence of class a2.
6. in e -> throw exception, call up the stacktrace and cehck for the existence of class a2.

However, the underlying problem is of course, that the two classes are too far apart, and there is too much tight coupling between the intervening classes and the rest of the system.

Also, the whole problem might go away with Functional Programming, but I'm not sure about that.

What do the experts say?

This Video Will Make You Angry

A friend of mine had found the little video on youtube visible below.

I thought it worth while to share.

Replacing ... With Boolean Expression

I found my IDE (IntelliJ) mentioning casually to me that I should replace this:

return municipality != null ? countSupply(municipality) > 0 : false;

With this:

return municipality != null && countSupply(municipality) > 0;

Trivial, I know, but most impressive of the IDE to catch that.

In the mean time, I'll patiently wait for the next version of the IDE that can actually do my programming for me.

Until that day, I'd better get back to work.

Deprecating Optional.get()

I found this link, and it struck a chord:
http://royvanrijn.com/blog/2016/04/deprecating-optional-get/.

Typescript

Our architect recently let us know that there is an Introduction to TypeScript¹ available on the edX Website².

Naturally, given that the TypeScript I am using at work is still very hard for me to understand, and given the fact that I wish to start using Angular 2 (which uses Typescript extensively) at home, I found this to be fascinating.

It was only a matter of minutes for me to enroll.

The course starts on May the 16th and is given by Microsoft experts and apparently the inventor of the TypeScript language.

I am quite looking forward to it and will let you know my thoughts at the end of the course.

References

[1] edX - Introduction to TypeScript

https://www.edx.org/course/introduction-typescript-microsoft-dev201x-1

[2] edX - Free online courses from the world's best universities

https://www.edx.org/

Microsoft Virtual Academy

https://mva.microsoft.com/

Typescript Tutorial - Quick Start

http://www.typescriptlang.org/Tutorial

A little Architecture Lesson

I found this blogpost:

http://blog.cleancoder.com/uncle-bob/2016/01/04/ALittleArchitecture.html

I think I still have a lot to learn.

java.lang.IllegalArgumentException: Comparison method violates its general contract!

I got the error message in the title whilst using Flyway to run database scripts at work¹.

I find it very satisfying that sorting methods in the JDK can detect if your comparer is not obeying the contract.

There are a lot of errors that cause the contract to be violated according to [2].

Here are the rules in short as described in [3]:

• sign(compare(x,y)) == -sign(compare(y,x))
• (compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0 (transitivity)
• compare(x, y)==0 implies that sgn(compare(x, z))==sgn(compare(y, z))

Sounds like an excellent starting point for some good JUnit Tests.

References

[1] GitHub - Flyway - Issue 1249

https://github.com/flyway/flyway/issues/1249

[2] StackOverflow - “Comparison method violates its general contract!”

http://stackoverflow.com/questions/8327514/comparison-method-violates-its-general-contract

[3] Oracle Javadoc - Interface Comparator>T>

https://docs.oracle.com/javase/7/docs/api/java/util/Comparator.html

Constructors, Javadoc and Inheritance

I recently had the problem that I defined some very good javadoc documentation on a constructor in one of my generic classes that is inherited a lot.

The documentation, however, is never propagated to subclasses, because constructors are not inherited¹.

It is too bad that I already have to duplicate Constructors in my subclasses, but now I have to duplicate the documentation as well?

Right now, the best solution I can think of is to add a "@see" link to the appropriate super constructor. Like so:

There is a feature request/bug² defined to expand the {inheritedDoc} to also work on constructors. But it has not seen any love for a long time.

References

[1] StackOverflow - Why is inheritedDoc not defined on constructors?

http://stackoverflow.com/questions/14848999/why-is-inheriteddoc-not-defined-on-constructors

[2] JDK-4810216 : Allow {@inheritDoc name} for constructors

http://bugs.java.com/bugdatabase/view_bug.do?bug_id=4810216

Java is pass-by-value!

Java is always pass by value, with no exceptions, ever.

From the JLS¹:

When the method or constructor is invoked (§15.12), the values of the actual argument expressions initialize newly created parameter variables, each of the declared type, before execution of the body of the method or constructor.

What this means is that the parameter is actually a copy of the value you provided. Hence, Java is always pass by value.

It is the same for the programming language C³.

It is the same for JavaScript⁴.

Confusion

The following quote from the JLS² causes a bit of confusion in the discussion:

An object is a class instance or an array.

The reference values (often just references) are pointers to these objects, and a special null reference, which refers to no object.

So, yes, Java does use pointers (to objects). However, these pointers are passed-by-value! This is why you can change the contents of an object but never switch the object for a different object.

References

[1] JLS SE 8 - 8.4.1. Formal Parameters

http://docs.oracle.com/javase/specs/jls/se8/html/jls-8.html#jls-8.4.1

[2] JLS SE 8 - 4.3.1. Objects

https://docs.oracle.com/javase/specs/jls/se8/html/jls-4.html#jls-4.3.1

[3] Dennis Kubes - Is C Pass by Value or Reference?

http://denniskubes.com/2012/08/20/is-c-pass-by-value-or-reference/

[4] What's The Pointy - JavaScript does not have "pass by reference"

http://whatsthepointy.blogspot.nl/2013/11/javascript-does-not-have-pass-by.html

JavaDude.com - Java is Pass-by-Value, Dammit!

http://javadude.com/articles/passbyvalue.htm

StackOverflow - Is Java “pass-by-reference” or “pass-by-value”?

http://stackoverflow.com/questions/40480/is-java-pass-by-reference-or-pass-by-value

Security Realms in Glassfish

I recently ran into the problem that the Roles I declared in my application were never assigned to the users I defined, when deployed in my Glassfish server. The log provided me with the following:

2016-03-30T13:13:43.560+0200] [glassfish 5.0] [WARNING] [] [javax.enterprise.system.core.security] [tid: _ThreadID=96 _ThreadName=AutoDeployer] [timeMillis: 1459336423560] [levelValue: 900] [[
No Principals mapped to Role [player].]]

Upon closer inspection, the problem became quite clear.

A more in-depth explanation on how security works in Glassfish can be found in the Java EE 6 Tutorial at [1].

Within a realm (security policy domain) you can define groups and users. Users can be assigned to groups.

Now, there needs to be a mapping between the security as defined in your application, and the security as defined in your application server. There is a bit of a separation between the two, because you need to be able to effortlessly deploy your application to different application servers without having to change your applications security settings.

Even if the realm of the application server happens to be configured differently.

scope	security items	Annotations	deployment descriptor
within your application	roles	@DeclareRolls, @RolesAllowed	web.xml
within your application server	users and groups	-	glassfish-web.xml

Manual mapping

Manual mapping between groups/users and your applications defined roles, is by using an application server specific deployment descriptor. The one glassfish uses is the WEB-INF/glassfish-web.xml and is included in your application war.

Schematically it looks like the following:

An example of the glassfish-web.xml deployment descriptor:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE glassfish-web-app PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1 Servlet 3.0//EN" "http://glassfish.org/dtds/glassfish-web-app_3_0-1.dtd">
<glassfish-web-app error-url="">
  <context-root>/myNewWebapp</context-root>
  <security-role-mapping>
    <role-name>admin</role-name>
    <group-name>admin</group-name>
  </security-role-mapping>
  <security-role-mapping>
    <role-name>god</role-name>
    <principal-name>root</principal-name>
    <group-name>god</group-name>
  </security-role-mapping>
  <security-role-mapping>
    <role-name>player</role-name>
    <group-name>player</group-name>
  </security-role-mapping>
 <class-loader delegate="true"/>
  <jsp-config>
    <property name="keepgenerated" value="true">
      <description>Keep a copy of the generated servlet class' java code.</description>
    </property>
  </jsp-config>
</glassfish-web-app>

In the example above most mappings are between groups and roles. There is one exception, which is the principal (user) "root". He is directly mapped to the "god" role, as he should be.

Automatic mapping

One of the handy features if you wish to use it, is the option for the glassfish server to automatically try to map groups and roles, based on similar names. This can be done in the glassfish administration gui, by going to server-config and selecting security.

It can completely replace the need for a glassfish-web.xml deployment descriptor.

References

[1] Java EE 6 Tutorial - Working with Realms, Users, Groups and Roles

https://docs.oracle.com/javaee/6/tutorial/doc/bnbxj.html

JSESSIONID and SoapUI

As the HTTP protocol is stateless, there has always been the question of 'How do we add state?'.

Nowadays this has been "solved" (notice the quotation marks there) by storing the state in the server, and assigning all this state to a "SessionID". The user entering the website gets a newly generated SessionID which is transmitted over and over again, so the server can associate the appropriate State with the appropriate User.

Usually the SessionID is sent along as a cookie, sometimes it is transmitted in the request as a request parameter.

In Java EE and JEE application servers, the cookie is called the "JSESSIONID". You can view an example in the headers shown below.

Retrieve Session id

In the first request a user does for a certain website, there is no cookie yet regarding the session.

PUT /resources/game/Elisa/logon?password=secret HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (X11; Fedora; Linux i686; rv:45.0) Gecko/20100101 Firefox/45.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/json; charset=utf-8
X-Requested-With: XMLHttpRequest
Referer: http://localhost:8080/
Cookie: COOKIE_SUPPORT=true; GUEST_LANGUAGE_ID=en_US; LOGIN=6d61617274656e5f6c407961686f6f2e636f6d; REMEMBER_ME=true; SCREEN_NAME=416b713230756641726f6d414441412b304452616b513d3d; _ga=GA1.1.23915379.1454272946
Connection: keep-alive
Content-Length: 0

A new session is created, and a cookie is sent along to the browser of the user with the session id. It is visible in the response below with the name of JSESSIONID.

HTTP/1.1 204 No Content
Server: GlassFish Server Open Source Edition  5.0
X-Powered-By: Servlet/3.1 JSP/2.3  (GlassFish Server Open Source Edition  5.0  Java/Oracle Corporation/1.8)
Set-Cookie: JSESSIONID=29ee5541e005eacdbe4291e5e138; Path=/resources; HttpOnly
Date: Sat, 26 Mar 2016 11:11:11 GMT

From then on the cookie is sent upon each request the browser of the user sends:

GET /resources/private/Elisa/mail?offset=0&_=1459034166678 HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (X11; Fedora; Linux i686; rv:45.0) Gecko/20100101 Firefox/45.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
X-Requested-With: XMLHttpRequest
Referer: http://localhost:8080/
Cookie: JSESSIONID=536aea5126fb63309b2a7609da51; COOKIE_SUPPORT=true; GUEST_LANGUAGE_ID=en_US; LOGIN=6d61617274656e5f6c407961686f6f2e636f6d; REMEMBER_ME=true; SCREEN_NAME=416b713230756641726f6d414441412b304452616b513d3d; _ga=GA1.1.23915379.1454272946
Connection: keep-alive

HttpOnly

A session id is very vulnerable to man-in-the-middle attack. One slight way to mitigate this is by using the tag "HttpOnly", as shown above.

This means, the cookie is set by the server during a http response, and is read by the server during a http request, and everyone else is not allowed access.

This makes it hard to read the cookie for example using JavaScript/Crosssite scripting.

UML

Perhaps some UML will make things clearer.

SoapUI

Store session id

The idea here is to add a Groovy script as a Test step after the response from the server (with the Set-Cookie). The script retrieves the Session ID from the response headers and stores it in a SoapUI properties cache on testcase, testsuite or project level.

def header = testRunner.testCase.getTestStepByName("logon").httpRequest.response.responseHeaders["Set-Cookie"]
header=header.toString()
start=header.indexOf("JSESSIONID=")
end=header.indexOf("; Path=")
testCaseProperty= header.substring(start,end)
testRunner.testCase.testSuite.setPropertyValue( "JSESSIONID", testCaseProperty )
testRunner.testCase.testSuite.project.setPropertyValue( "JSESSIONID", testCaseProperty )

Use Session id

Now, the idea is to use this stored session id in subsequent http requests in other test steps and even other test cases.

You can define a new header in your subsequent http requests to send the session id along as a cookie, mimicking how a website works:

REST

I will go into REST Services, which are not stateful, in a future blog post. Basically it would mean that we need to send all information required for the REST Service, each and every time we access the REST Service. The state is effectively stored on the client side.

References

QA Strategies Best Testing practices and Automation tips - SOAPUI - Fetch session id from response header

http://qastrategies.blogspot.nl/2012/05/soapui-fetch-session-id-from-response.html

Time is running out, don't lose it. - Set Authentication WebService and Set Authentication Cookies in SoapUi

http://mariemjabloun.blogspot.nl/2014/11/set-webservice-authentication-and-set.html

SoapUI Cookie management

https://siking.wordpress.com/2013/07/25/soapui-cookie-management/

SoapUI - Working with Properties

https://www.soapui.org/functional-testing/properties/working-with-properties.html

CAFE BABE

CAFE BABE is the first four bytes in any Java compiled class file, in Hexadecimal format. It identifies the file as a Java class file.

James Gosling explained why.

    We used to go to lunch at a place called St Michael’s Alley. According to local legend, in the deep dark past, the Grateful Dead used to perform there before they made it big. It was a pretty funky place that was definitely a Grateful Dead Kinda Place. When Jerry died, they even put up a little Buddhist-esque shrine. When we used to go there, we referred to the place as Cafe Dead. Somewhere along the line, it was noticed that this was a HEX number. I was re-vamping some file format code and needed a couple of magic numbers: one for the persistent object file, and one for classes. I used CAFEDEAD for the object file format, and in grepping for 4 character hex words that fit after “CAFE” (it seemed to be a good theme) I hit on BABE and decided to use it. At that time, it didn’t seem terribly important or destined to go anywhere but the trash can of history. So CAFEBABE became the class file format, and CAFEDEAD was the persistent object format. But the persistent object facility went away, and along with it went the use of CAFEDEAD – it was eventually replaced by RMI.

References

The Magic Word in Java: CAFEBABE

https://dzone.com/articles/the-magic-word-in-java-cafebabe

Humour in Javadoc

I found this little gem that made me chuckle from Gavin King.

/**
 * A copy of the EE5 standard InvocationContext API.
 * We do this because some poor souls are still using
 * J2EE. Pray for them.
 * 
 * @author Gavin King
 *
 */
public interface InvocationContext
{
   public Object getTarget();
   public Map getContextData();
   public Method getMethod();
   public Object[] getParameters();
   public Object proceed() throws Exception;
   public void setParameters(Object[] params);
}

Lightfish

Had a small issue with installing Lightfish onto my Glassfish. I noticed this issue before, but forgot to write down the solution somewhere (that's how it goes).

[2016-03-06T13:20:51.109+0000] [glassfish 4.1] [WARNING] [poolmgr.create_resource_error] [javax.enterprise.resource.resourceadapter.com.sun.enterprise.resource.allocator] [tid: _ThreadID=224 _ThreadName=AutoDep
RAR5038:Unexpected exception while creating resource for pool DerbyPool. Exception : javax.resource.spi.ResourceAllocationException: Connection could not be allocated because: java.net.ConnectException : Error connecting to server localhost on port 1527 with message Connection refused.]]

Ahhh, found the solution again, the Derby database server is not installed/started automatically:

asadmin start-database

This's what I wanted to see:

[2016-03-06T13:34:06.500+0000] [glassfish 4.1] [INFO] [NCLS-DEPLOYMENT-02035] [javax.enterprise.system.tools.deployment.autodeploy] [tid: _ThreadID=224 _ThreadName=AutoDeployer] [timeMillis: 1457271246500] [levelValue: 800] [[
[AutoDeploy] Successfully autodeployed : /home/glassfish/glassfish4/glassfish/domains/domain1/autodeploy/lightfish.war.]]

References

[0] Adam Bien - Lightfish

http://lightfish.adam-bien.com/

[1] GitHub - Lightfish

https://github.com/AdamBien/lightfish

[2] StackOverflow - Derby Pool ping fails with java.net.ConnectException in Glassfish

http://stackoverflow.com/questions/27747479/derby-pool-ping-fails-with-java-net-connectexception-in-glassfish

Youtube - Project LightFish--Java EE Telemetry For GlassFish

https://www.youtube.com/watch?v=9nU6Oubx0tQ

Abstraction Considered Harmful

One of the more thought-provoking pieces I have read in a long time, and one I happen to fully agree with, having seen myself what too much Abstraction can lead to:

http://bravenewgeek.com/abstraction-considered-harmful/

I like the quote:

“duplication is far cheaper than the wrong abstraction”

More, Less and Most

I found the following gem in [1]:

“less is more, but more more than more is, so more is less less, so use more less if you want less more.”

— Joost Kremers²

“If less is more than more, most is more than less.”

— Slackware Linux Essentials²

References

[1] Stackoverflow - What are the differences between most, more and less?

http://unix.stackexchange.com/questions/81129/what-are-the-differences-between-most-more-and-less

[2] Slackware Linux Essentials - 10.2 Pagers: more, less, and most

http://www.slackbook.org/html/file-commands-pagers.html

Front end update: Angular 2.0 & HTTP/2

I have taken up learning Angular, yet it seems that a new angular is being prepared, Angular 2¹. Angular 2 seems to be a complete rewrite, and has a very different architecture. For one thing, it can work with ECMAScript 6 and TypeScript.

Luckily, I managed to be at a small session regarding Angular 2 at Quintor, today, Thursday the 18th of February 2016.².

As a matter of fact, I am writing this blog in the train home.

HTTP/2

The speaker was Martijn van Campenhout of Quintor. The first presentation was regarding the new HTTP/2 protocol (officially released in May 2015), which is rapidly replacing the old HTTP/1.1 version (released in 1996). It is based on the SPDY implementation developed by Google in 2010 and officially submitted for standardisation.

Statistics shown by Mozilla indicate that an decrease of pageload times of 2.2 seconds, results in an increase of Downloads by about 15%.^{3 4}

Old tips and tricks

The old tips and tricks that are used to deal with the defficiencies of HTTP/1.1 and the lack of bandwith and troubles with latency were:

• combining files as much as possible
• inlining everything
• minifying everything
• domain sharding
• using sprite files
• define critical CSS and JS files
• using GZIP

These different tactics are used to make sure the Webbrowser has to do a minimum number of requests and does not need to get a lot of data to the user.

It seems the Chrome browser development tools has excellent support for simulating bad connections or large latency, using a variety of set network connection types.

Another common tactic was to display/render only those parts of the web page that are visible to the user. Once the user starts scrolling, the parts that scroll into the viewscreen of the user are downloaded and rendered. This is called "the Fold"⁵, and is a common DTP term.

There was a most impressive demonstration, where a picture consisting of about 100 smaller pictures was being shown, with a one second delay (latency) between the request and the response for a smaller picture. The difference between HTTP/1.1 and HTTP/2 was very big. Where the Browser only allows 8 concurrent connections with HTTP/1.1 causing a severe bottleneck, a single connection using HTTP/2 was more than enough to show the entire picture in about a second.

The tactics used for HTTP/2 can be summed up as follows:

Multiplexing

different Streams of data (files, css, images, html, javascript) across the same HTTP connection

Priority

allowing a setting of priority based on content. For example that the big logo should be loaded first, and the little icons at the bottom of the page last.

Binary Headers

easier and quicker to parse, fixed length, unambiguous

HPACK

In the past headers were not compressed, this has now changed.

Server push

server remembers the combination of files requested. One file request will trigger the others automatically.

This is also the first time that I heard the term CRIME, "Compression Ratio Info-leak Made Easy". Basically it boils down to a man-in-the-middle-attack, adding stuff to a message to see if it increased the compressed file. If it does not, the item added already was part of the file, and he has learned something valuable regarding the message.

The HTTP/2 is already incorporated in Apache server, NGINX. Servlet specification 4.0 of Java EE 8 will have support for it (JSR-369).

Criticisms

There were some criticisms regarding HTTP/2, though.

It seems the HTTP/2 might just be available only using HTTPS. The configuring of priority and server push needs to be done on the backend right now and cannot be controlled by the webdeveloper. This causes a tight coupling between front end and back end, which most people (including me) might not like.

All these optimalisations mean absolutely nothing, if your resources (html files, images, etc) are retrieved from many different servers.

Note: the effect of HTTP/2 is also that the old tips and tricks are not only useless, but can actually hamper the responsiveness of the website. An example of this is caching.

Angular 2

The speaker, Rachèl Heimbach, went quite fast through his presentation, which means we had time for two presentations in stead of one. On the other hand, it did require an inordinate amount of attention and knowledge of Angular/JavaScript/TypeScript to follow.

Angular 2 uses both ECMAScript 6 and TypeScript.

Some of the most important features⁶ of ECMAScript 6 that will return when you are attempting Angular 2:

• class
• import
• export
• private namespaces (without export and import, everything is by default off limits)
• default values (used when parameters are omitted in the function call)
• lambdas (no longer do we require the use of "var self = this;".

Benefits of using TypeScript in angular 2 are as follows:

• safe typing, makes refactoring and debugging a whole lot easier
• @Decorators, makes injection and adding behaviour generically a whole lot easier

Instead of the familiar services, factories and controllers of Angular 1, Angular 2 has simplified this a bit. Angular 2 basically consists of only Web components (that make up your pages/views/etc) and singleton injectors (that can take the place of your controllers, factories and services.) As you can see there is no longer a distinction necessary.

Some more enhancements:

• ChangeDetection
• ViewEncapsulation (with a shadow dom)
• Provide function
• ngUpgrade thing

It is possible to upgrade an Angular 1 application piece-by-piece or add Angular 2 components to it. The other way around is also possible.

Notes

HTTP/2 seems to be more and more widely supported. Angular 2 is still in the beta phase, however, it seems to be fairly stable. Unfortunately, it is currently not recommended for production use. However, the presentation did point out that in order to make the task of porting your application from Angular 1 to Angular 2 less onerous, it might be a good bet to start by designing the new parts of your Angular 1 application according to the Angular 2 style.

A question was raised on whether or not you should use Angular 2 at all. It seems Google will support Angular 1 for as long as there is a market for it. I heard that they measure the traffic to the two websites (Angular 1 and Angular 2) to determine interest. I got the impression from the presentation that Angular 1 has slowly been moving towards the new Angular 2 way of doing things.

Performance wise, Angular 2 is a step in the right direction. Keeping new Angular 2 in mind, whilst working on an Angular 1 application seems to be a very safe bet. If the need will arise in the future that a port is required, at least it will not be as herculean a task as first thought.

References

[1] Angular 2

https://angular.io/

[2] Quintor - Front end update: Angular 2.0 & HTTP/2

https://www.quintor.nl/sessie-front-end-update/

[3] Webpage testing

http://www.webpagetest.org

[4] Can I use?

http://www.caniuse.com

[5] Wikipedia - Above the fold

https://en.wikipedia.org/wiki/Above_the_fold

[6] ES6 Features

http://www.es6-features.org

My First Lambda

I just implemented my first lambda¹. My java code is now officially only Java 8 and up compliant.

My pom was changed as follows:

<project.source.version>1.8</project.source.version>
<project.target.version>1.8</project.target.version>

Original code

The original code looked like this:
Pretty straight forward stuff.

Adding a Lambda

Then we received two change requests, that could be resolved by re-using the method above.

But this time, not everyone was required to see the message. In other words, the list of active players needed to be filtered.

Enter the Predicate².

Let us say the filtering needs to be done, by those who wish to be kept in the loop regarding roleplaying events. These users contain an Ooc (Out-of-Character) flag.

Calling this can be done using a Lambda, like so:

sendWall("[OOC: " + aUser.getName() + "] " + message + "\r\n", p -> p.getOoc());

Streams

To make things a little more complicated, you can make use of the new Streams^{3 4} provided in Java 8.

In the example below, I take a stream from the List collection provided by getActivePlayers, filter it by the predicate, and run the writeMessage on each found user. That last one is called a "terminator" as it terminates the stream, i.e. it "does something".

A lot less code than the previous for loop. It seems more complicated, but I guess it just requires me to get used to it.

It also causes me to create another Lambda, as the terminator.

The Old Way

In the old way, calling this method sendWall was done using an inner class, and it looked as follows:

References

[1] Java^TM Tutorials - Lambda Expressions

https://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html

[2] JavaDoc - Interface Predicate<T>

https://docs.oracle.com/javase/8/docs/api/java/util/function/Predicate.html

[3] Java^TM Tutorials - The Collection Interface

https://docs.oracle.com/javase/tutorial/collections/interfaces/collection.html

[4] Processing Data with Java SE 8 Streams, Part 1

http://www.oracle.com/technetwork/articles/java/ma14-java-se-8-streams-2177646.html

Validation of Beans

Just a small utility function that prints out readable messages on what is wrong with a certain bean (like for example a JPA Entity).

Got a little miffed with the fact that my javax.validation.ConstraintViolationException that is thrown, never shows the exact problem. I always have to drill down into the Exception to find the message in order to fix the problem.

I hope it helps someone.

See for more information about bean validation JSR 303.

References

The Java EE 6 Tutorial - Using Bean Validation

http://docs.oracle.com/javaee/6/tutorial/doc/gircz.html

The Java EE 6 Tutorial - Validating Persistent Fields and Properties

https://docs.oracle.com/cd/E19798-01/821-1841/gkahq/index.html

Liferay Themes

This is a followup of the blog entry about Liferay^{1 2 3}.

As I wished to tailor the website layout of my Liferay instance, I started looking at themes. What I found was a bit of a surprise for me, as it seems themes for Liferay needs to be compiled/build using ant or maven.

There is a big body of documentation regarding the creation of Themes, but the procedure seems a bit brittle. It uses ant, needs to be able to reach the files of the classic theme, is dependent of the type of Application Server you are using, and a bit of other stuff.

Below are some of the error messages I got and had to solve before I got anything that worked.

Creating a Theme with Ant

• Download the Plugins SDK that compiles new themes (and plugins) using Ant.
  
• Create a properties file like build.<username>.properties
  [glassfish@localhost glassfish]$ cat liferay-plugins-sdk-6.2/build.glassfish.properties
  app.server.type=glassfish
  app.server.parent.dir=/home/glassfish
  app.server.glassfish.dir=${app.server.parent.dir}/glassfish4/glassfish
  javac.compiler=modern
• In this directory, there should be a directory themes, where we can create our new theme⁶.
  
  [glassfish@localhost themes]$ ./create.sh mine "Mine"
  Parallel execution with configuration on demand is an incubating feature.
  :createTheme
  
  BUILD SUCCESSFUL
  
  Total time: 2.377 secs
• Changed the build.xml in mine-theme, to use 'classic' as the parent theme.
  
  <property name="theme.parent" value="classic" />
  
  If you create your own theme with the Plugins SDK, place the icon in themes/Your_theme/docroot/_diffs/images/favicon.ico if you want to have your own favicon to bookmark.
  
• Then "ant war"
  
• Then "ant deploy" (or just copy the war into glassfish4/deploy)

The first problem I ran into was a problem with the xml file.

[2015-09-19T20:45:06.136+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=140 _ThreadName=Thread-8] [timeMillis: 1442695506136] [levelValue: 800] [[
20:45:06,136 INFO  [AutoDeployer][HotDeployEvent:145] Plugin mine-theme requires marketplace-portlet]]

[2015-09-19T20:45:06.142+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=140 _ThreadName=Thread-8] [timeMillis: 1442695506142] [levelValue: 800] [[
20:45:06,136 INFO  [AutoDeployer][HotDeployImpl:217] Deploying mine-theme from queue]]

[2015-09-19T20:45:06.149+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=140 _ThreadName=Thread-8] [timeMillis: 1442695506149] [levelValue: 800] [[
20:45:06,142 INFO  [AutoDeployer][PluginPackageUtil:1016] Reading plugin package for mine-theme]]

[2015-09-19T20:45:06.193+0000] [glassfish 4.1] [INFO] [] [javax.enterprise.web] [tid: _ThreadID=140 _ThreadName=AutoDeployer] [timeMillis: 1442695506193] [levelValue: 800] [[
WebModule[null] ServletContext.log():Initializing Spring root WebApplicationContext]]

[2015-09-19T20:45:06.204+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=140 _ThreadName=Thread-8] [timeMillis: 1442695506204] [levelValue: 800] [[
20:45:06,204 INFO  [AutoDeployer][ThemeHotDeployListener:98] Registering themes for mine-theme]]

[2015-09-19T20:45:06.236+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=140 _ThreadName=Thread-8] [timeMillis: 1442695506236] [levelValue: 800] [[
20:45:06,228 ERROR [AutoDeployer][ThemeLocalServiceImpl:278] com.liferay.portal.kernel.xml.DocumentException: Error on line 8 of document  : Element type "theme" must be followed by either attribute specifications, ">" or "/>". Nested exception: Element type "theme" must be followed by either attribute specifications, ">" or "/>".
com.liferay.portal.kernel.xml.DocumentException: Error on line 8 of document  : Element type "theme" must be followed by either attribute specifications, ">" or "/>". Nested exception: Element type "theme" must be followed by either attribute specifications, ">" or "/>".
at com.liferay.portal.xml.SAXReaderImpl.read(SAXReaderImpl.java:429)
at com.liferay.portal.xml.SAXReaderImpl.read(SAXReaderImpl.java:447)
at com.liferay.portal.kernel.xml.SAXReaderUtil.read(SAXReaderUtil.java:187)
at com.liferay.portal.service.impl.ThemeLocalServiceImpl._readThemes(ThemeLocalServiceImpl.java:472)
at com.liferay.portal.service.impl.ThemeLocalServiceImpl.init(ThemeLocalServiceImpl.java:272)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:115)
at com.liferay.portal.spring.transaction.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:62)
at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:51)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:56)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
at com.liferay.portal.spring.aop.ServiceBeanAopProxy.invoke(ServiceBeanAopProxy.java:175)
at com.sun.proxy.$Proxy269.init(Unknown Source)
at com.liferay.portal.service.ThemeLocalServiceUtil.init(ThemeLocalServiceUtil.java:118)
at com.liferay.portal.deploy.hot.ThemeHotDeployListener.doInvokeDeploy(ThemeHotDeployListener.java:103)
at com.liferay.portal.deploy.hot.ThemeHotDeployListener.invokeDeploy(ThemeHotDeployListener.java:49)
at com.liferay.portal.deploy.hot.HotDeployImpl.doFireDeployEvent(HotDeployImpl.java:227)
at com.liferay.portal.deploy.hot.HotDeployImpl.fireDeployEvent(HotDeployImpl.java:96)
at com.liferay.portal.kernel.deploy.hot.HotDeployUtil.fireDeployEvent(HotDeployUtil.java:28)
at com.liferay.portal.kernel.servlet.PluginContextListener.fireDeployEvent(PluginContextListener.java:164)
at com.liferay.portal.kernel.servlet.PluginContextListener.doPortalInit(PluginContextListener.java:154)
at com.liferay.portal.kernel.util.BasePortalLifecycle.portalInit(BasePortalLifecycle.java:44)
at com.liferay.portal.kernel.util.PortalLifecycleUtil.register(PortalLifecycleUtil.java:74)
at com.liferay.portal.kernel.util.PortalLifecycleUtil.register(PortalLifecycleUtil.java:58)
at com.liferay.portal.kernel.util.BasePortalLifecycle.registerPortalLifecycle(BasePortalLifecycle.java:54)
at com.liferay.portal.kernel.servlet.PluginContextListener.contextInitialized(PluginContextListener.java:116)
at org.apache.catalina.core.StandardContext.contextListenerStart(StandardContext.java:5394)
at com.sun.enterprise.web.WebModule.contextListenerStart(WebModule.java:743)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:5932)
at com.sun.enterprise.web.WebModule.start(WebModule.java:691)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1041)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:1024)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:747)
at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:2286)
at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1932)
at com.sun.enterprise.web.WebApplication.start(WebApplication.java:139)
at org.glassfish.internal.data.EngineRef.start(EngineRef.java:122)
at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:291)
at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:352)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:500)
at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:219)
at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:491)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:539)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:535)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:534)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:565)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:557)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:556)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1464)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1300(CommandRunnerImpl.java:109)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1846)
at org.glassfish.deployment.autodeploy.AutoOperation.run(AutoOperation.java:164)
at org.glassfish.deployment.autodeploy.AutoDeployer.deploy(AutoDeployer.java:597)
at org.glassfish.deployment.autodeploy.AutoDeployer.deployAll(AutoDeployer.java:484)
at org.glassfish.deployment.autodeploy.AutoDeployer.run(AutoDeployer.java:412)
at org.glassfish.deployment.autodeploy.AutoDeployer.run(AutoDeployer.java:403)
at org.glassfish.deployment.autodeploy.AutoDeployService$1.run(AutoDeployService.java:233)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Caused by: org.dom4j.DocumentException: Error on line 8 of document  : Element type "theme" must be followed by either attribute specifications, ">" or "/>". Nested exception: Element type "theme" must be followed by either attribute specifications, ">" or "/>".
at org.dom4j.io.SAXReader.read(SAXReader.java:482)
a]]

[2015-09-19T20:45:06.236+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=140 _ThreadName=Thread-8] [timeMillis: 1442695506236] [levelValue: 800] [[
t org.dom4j.io.SAXReader.read(SAXReader.java:365)
at com.liferay.portal.xml.SAXReaderImpl.read(SAXReaderImpl.java:426)
... 82 more]]

Seems the problem was Quotes (") around the name of the theme "Mine" in the docroot/WEB-INF/liferay-plugin-package.properties file. Removed quotes, and things get picked up now.

Creating a Theme with Maven

mvn archetype:generate \
    -DarchetypeArtifactId=liferay-theme-archetype \
    -DarchetypeGroupId=com.liferay.maven.archetypes \
    -DarchetypeVersion=6.1.0 \
    -DartifactId=sample-theme \
    -DgroupId=com.liferay.sample \
    -Dversion=1.0-SNAPSHOT

I tried with Maven^{4 5} instead of Ant to develop a theme, but I couldn't rightly get it to work. Needs some time to work on it.

Uploading a theme

So there's a war file created containing the files. You can deploy the file into the application server.

There are several ways to do this. You can upload it to glassfish4/glassfish/domains/domain1/autodeploy.

But I find that in some cases the theme is not recorded by Liferay. Liferay listens to specific directories for new war files that are theme related. In my case the additional directory could well be glassfish4/deploy, because glassfish4 is the "home" of Liferay in my case.

You can verify if the theme was successfully picked up in your glassfish server log. You should see something lke:

[2016-01-11T14:38:26.668+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523106668] [levelValue: 800] [[
14:38:26,667 INFO  [com.liferay.portal.kernel.deploy.auto.AutoDeployScanner][AutoDeployDir:204] Processing mine-theme-6.2.0.1.war]]

[2016-01-11T14:38:26.673+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523106673] [levelValue: 800] [[
14:38:26,673 INFO  [com.liferay.portal.kernel.deploy.auto.AutoDeployScanner][ThemeAutoDeployListener:51] Copying themes for /home/glassfish/glassfish4/deploy/mine-theme-6.2.0.1.war]]

[2016-01-11T14:38:26.723+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523106723] [levelValue: 800] [[
14:38:26,722 INFO  [com.liferay.portal.kernel.deploy.auto.AutoDeployScanner][BaseDeployer:863] Deploying mine-theme-6.2.0.1.war]]

[2016-01-11T14:38:27.119+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523107119] [levelValue: 800] [[
Expanding: /home/glassfish/glassfish4/deploy/mine-theme-6.2.0.1.war into /tmp/20160111143826861]]

[2016-01-11T14:38:27.563+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523107563] [levelValue: 800] [[
Copying 1 file to /tmp/20160111143826861/WEB-INF]]

[2016-01-11T14:38:27.568+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523107568] [levelValue: 800] [[
Copying 1 file to /tmp/20160111143826861/WEB-INF]]

[2016-01-11T14:38:27.581+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523107581] [levelValue: 800] [[
Copying 1 file to /tmp/20160111143826861/WEB-INF/classes]]

[2016-01-11T14:38:27.589+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523107589] [levelValue: 800] [[
Copying 1 file to /tmp/20160111143826861/WEB-INF/classes]]

[2016-01-11T14:38:27.606+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523107606] [levelValue: 800] [[
Copying 1 file to /tmp/20160111143826861/WEB-INF]]

[2016-01-11T14:38:27.760+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523107760] [levelValue: 800] [[
14:38:27,760 INFO  [com.liferay.portal.kernel.deploy.auto.AutoDeployScanner][BaseDeployer:2391] Modifying Servlet 2.4 /tmp/20160111143826861/WEB-INF/web.xml]]

[2016-01-11T14:38:27.860+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523107860] [levelValue: 800] [[
Building war: /tmp/20160111143827761]]

[2016-01-11T14:38:28.307+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523108307] [levelValue: 800] [[
Deleting directory /tmp/20160111143826861]]

[2016-01-11T14:38:28.331+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=112 _ThreadName=Thread-8] [timeMillis: 1452523108331] [levelValue: 800] [[
14:38:28,330 INFO  [com.liferay.portal.kernel.deploy.auto.AutoDeployScanner][ThemeAutoDeployListener:57] Themes for /home/glassfish/glassfish4/deploy/mine-theme-6.2.0.1.war copied successfully. Deployment will start in a few seconds

[2016-01-11T14:38:29.675+0000] [glassfish 4.1] [INFO] [NCLS-DEPLOYMENT-02027] [javax.enterprise.system.tools.deployment.autodeploy] [tid: _ThreadID=221 _ThreadName=AutoDeployer] [timeMillis: 1452523109675] [levelValue: 800] [[
Selecting file /home/glassfish/glassfish4/glassfish/domains/domain1/autodeploy/mine-theme.war for autodeployment]]

[2016-01-11T14:38:30.239+0000] [glassfish 4.1] [INFO] [] [javax.enterprise.system.tools.deployment.common] [tid: _ThreadID=221 _ThreadName=AutoDeployer] [timeMillis: 1452523110239] [levelValue: 800] [[
visiting unvisited references]]

[2016-01-11T14:38:31.349+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=221 _ThreadName=Thread-8] [timeMillis: 1452523111349] [levelValue: 800] [[
14:38:31,348 INFO  [AutoDeployer][HotDeployEvent:145] Plugin mine-theme requires marketplace-portlet]]

[2016-01-11T14:38:31.350+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=221 _ThreadName=Thread-8] [timeMillis: 1452523111350] [levelValue: 800] [[
14:38:31,349 INFO  [AutoDeployer][HotDeployImpl:217] Deploying mine-theme from queue]]

[2016-01-11T14:38:31.350+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=221 _ThreadName=Thread-8] [timeMillis: 1452523111350] [levelValue: 800] [[
14:38:31,350 INFO  [AutoDeployer][PluginPackageUtil:1016] Reading plugin package for mine-theme]]

[2016-01-11T14:38:31.404+0000] [glassfish 4.1] [INFO] [] [javax.enterprise.web] [tid: _ThreadID=221 _ThreadName=AutoDeployer] [timeMillis: 1452523111404] [levelValue: 800] [[
WebModule[null] ServletContext.log():Initializing Spring root WebApplicationContext]]

[2016-01-11T14:38:31.414+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=221 _ThreadName=Thread-8] [timeMillis: 1452523111414] [levelValue: 800] [[
14:38:31,413 INFO  [AutoDeployer][ThemeHotDeployListener:98] Registering themes for mine-theme]]

[2016-01-11T14:38:33.507+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=221 _ThreadName=Thread-8] [timeMillis: 1452523113507] [levelValue: 800] [[
14:38:33,507 INFO  [AutoDeployer][ThemeHotDeployListener:113] 1 theme for mine-theme is available for use]]

[2016-01-11T14:38:33.736+0000] [glassfish 4.1] [INFO] [AS-WEB-GLUE-00172] [javax.enterprise.web] [tid: _ThreadID=221 _ThreadName=AutoDeployer] [timeMillis: 1452523113736] [levelValue: 800] [[
Loading application [mine-theme] at [/mine-theme]]]

[2016-01-11T14:38:33.992+0000] [glassfish 4.1] [INFO] [] [javax.enterprise.system.core] [tid: _ThreadID=221 _ThreadName=AutoDeployer] [timeMillis: 1452523113992] [levelValue: 800] [[
mine-theme was successfully deployed in 4,148 milliseconds.]]

[2016-01-11T14:38:33.998+0000] [glassfish 4.1] [INFO] [NCLS-DEPLOYMENT-02035] [javax.enterprise.system.tools.deployment.autodeploy] [tid: _ThreadID=221 _ThreadName=AutoDeployer] [timeMillis: 1452523113998] [levelValue: 800] [[
[AutoDeploy] Successfully autodeployed : /home/glassfish/glassfish4/glassfish/domains/domain1/autodeploy/mine-theme.war.]]

Setting your theme

See Control Panel -> Configuration -> Portal settings -> Display Settings and below there's Look & Feel.

Also, there's this place: Control Panel -> Sites -> Your site -> Site Pages -> Look and Feel + Current Theme. You can select another theme from a list below in the same configuration page.

Your theme is actually a Plugin, and can be found among the Theme Plugins. Navigate to Control Panel -> Apps -> Plugins Configuration. Select the tab "Theme Plugins". Your theme should be visible in the list.

If not, you are likely to receive something like this in your logs^{7 8}:

[2015-12-31T23:38:41.633+0000] [glassfish 4.1] [INFO] [] [] [tid: _ThreadID=29 _ThreadName=Thread-8] [timeMillis: 1451605121633] [levelValue: 800] [[
23:38:41,633 WARN [http-listener-1(4)][ThemeLocalServiceImpl:156] No theme found for specified theme id mine_WAR_minetheme. Returning the default theme.]]

I don't know why this happened in my case, but I found a reference in the Liferay database to the old mine-theme. Even though I thought I got rid of it permanently, but removing it from the autodeploy directory. If I remove it from the glassfish autodeploy directory, it is also removed from the Application Server. Liferay is listening to what happenes on the autodeploying front, detects the deletion, and starts cleaning up after itself as well.

Apparently, there still was an entry left hanging. I had to update the database manually to reset a database tablerow back to "classic".

Adapting your theme

It is possible to adapt your theme, by replacing images and css and html files judiciously.

The interesting bit of theming is that you specify the parent theme. So, you've got "styled" and "unstyled" and the default theme called "classic" to choose from. But it would also be easy to create for example a "Christmas" theme, that is derived from the theme that you normally use. So it's kind of an Inheritance structure, similar to Object Oriented Programming.

I also like the fact that in your new theme you specify the "changes" compared to the parent theme in the _diffs folder. This way, when a parent theme changes (upon a new release of Liferay Portal), you automatically get the new updates, and if you do not change these defaults in your theme, they are just build into the target war upon the next release.

If you wish to start making bigger changes, there is the possibility of using the Liferay Application Display Templates⁹ (ADT).

Liferay uses Apache Velocity¹⁰ as its templating engine.

Adding JQuery to the Liferay portal can be done as described at [11].

References

[1] Randomthoughts - Liferay

http://randomthoughtsonjavaprogramming.blogspot.nl/2015/12/liferay.html

[2] Liferay

http://www.liferay.com/

[3] Wikipedia - Liferay

https://en.wikipedia.org/wiki/Liferay

[4] Liferay Company Blogs - Creating Liferay Themes with Maven

https://www.liferay.com/web/mika.koivisto/blog/-/blogs/creating-liferay-themes-with-maven#_33_message_13069757

[5] Liferay Developer Network - Developing Liferay Theme Plugins with Maven

https://dev.liferay.com/develop/tutorials/-/knowledge_base/6-2/developing-liferay-theme-plugins-with-maven

[6] Liferay Developer Network - Creating a Theme Project in the Plugins SDK

https://dev.liferay.com/develop/tutorials/-/knowledge_base/6-2/creating-a-theme-project-in-the-plugins-sdk

[7] StackOverflow - Error occures when deploying theme to liferay portal. “No theme found for specified theme id .”

http://stackoverflow.com/questions/11721220/error-occures-when-deploying-theme-to-liferay-portal-no-theme-found-for-specif

[8] Techspace - No theme found for specified theme id abc_WAR_xyztheme

http://parasjain.net/2012/07/11/no-theme-found-for-specified-theme-id-abc_war_xyztheme/

[9] LiferaySavvy - List of Velocity Variables in Liferay

http://www.liferaysavvy.com/2015/02/list-of-velocity-variables-in-liferay.html

[10] Wikipedia - Apache Velocity

https://en.wikipedia.org/wiki/Apache_Velocity

[11] Liferay Company Blogs - Using jQuery (or any Javascript library) in Liferay 6.0

http://www.liferay.com/web/nathan.cavanaugh/blog/-/blogs/using-jquery-or-any-javascript-library-in-liferay-6-0