Improper Use of Softly Asserts

I don't really like Softly Asserts, and I notice that a lot of developers at work seem to use it all the time.

And since it's Christmas today (Merry Christmas!), let's get into the spirit of the season by testing if there is a Miracle on 34th Street.

We are going to use SoftlyAssertions with the following test written below:

And it looks fine if the tests run properly.

But when the tests fail, likely as not we see NullPointerExceptions instead of useful asserts.

Let's try to fail the test, by removing the town.

Street thirtyfourthStreet = new Street("34th Street", null);

And lo and behold, we get this:

java.lang.NullPointerException at com.mrbear.testingtests.SoftlyAssertionTest.testMiracleAt34thStreet(SoftlyAssertionTest.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:571)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:707)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:979)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.privateRun(TestRunner.java:648)
at org.testng.TestRunner.run(TestRunner.java:505)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
at org.testng.SuiteRunner.run(SuiteRunner.java:364)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1187)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1116)
at org.testng.TestNG.runSuites(TestNG.java:1028)
at org.testng.TestNG.run(TestNG.java:996)
at com.intellij.rt.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:66)
at com.intellij.rt.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:110)

I really would like for SoftAssertions to only be used during checking of simple properties within an object, thanks!

And not this cross-boundary stuff.

Java Alternatives in Fedora Core - Followup

This is a followup of the blogpost Java Alternatives in Fedora Core.

The instructions in the blogpost also hold for Java 13.

[root@localhost ~]# dnf install java-13-openjdk
[root@localhost ~]# dnf install java-13-openjdk-devel

Unfortunately, there was a snag with Maven, so I thought I'd document it here.

Maven

Maven stubbornly remained firmly pointing to java 8, whatever alternative I picked in my Fedora Core.

[mrbear@localhost ~]$ mvn -version
Apache Maven 3.5.4 (Red Hat 3.5.4-5)
Maven home: /usr/share/maven
Java version: 1.8.0_232, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.fc30.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.3.11-200.fc30.x86_64", arch: "amd64", family: "unix"

Well, trying to compile a java application with version 13 in the pom.xml, is NOT going to work when your Maven is using openjdk 8.

Really no surprise there.

You get:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project mrbearapp: Fatal error compiling: invalid target release: 13 -> [Help 1]

It turns out I really need to set JAVA_HOME?

Editing .bashrc to add the following:

export JAVA_HOME=/usr/lib/jvm/java-13

Now I get:

[mrbear@localhost ~]$ mvn -version
Apache Maven 3.5.4 (Red Hat 3.5.4-5)
Maven home: /usr/share/maven
Java version: 13.0.1, vendor: N/A, runtime: /usr/lib/jvm/java-13-openjdk-13.0.1.9-2.rolling.fc30.x86_64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.3.11-200.fc30.x86_64", arch: "amd64", family: "unix"

All is as it should be, and we are merrily on our way.

Installing Bootstrap with NPM

I thought it would be a good idea to write this stuff down. You can easily find sources on the web that indicates the same instructions.

Bootstrap contains some css files and some js files.

Bootstrap also uses some optional dependencies.

Make sure NPM is the latest

npm install -g npm

Make sure your Bootstrap¹ is updated and/or installed with npm.

npm install bootstrap --save

You will probably see some warnings regarding bootstrap:

npm WARN bootstrap@4.3.1 requires a peer of jquery@1.9.1 - 3 but none is installed. You must install peer dependencies yourself.
npm WARN bootstrap@4.3.1 requires a peer of popper.js@^1.14.7 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/webpack-dev-server/node_modules/fsevents):

Install the jquery² if you like.

It provides Bootstrap with some additional behaviour that can be initiated by the websurfer.

npm install jquery --save

Install popper.js³ if you like.

It's for popovers and tooltips and things.

npm install popper.js --save

I also use bootswatch⁴ for the amazing Bootstrap themes that they have.

npm install bootswatch --save

Angular

As I am using Angular⁵, I need to add it so Angular adds it to the app.

Add in angular.json:

"styles": [
   "node_modules/bootstrap/dist/css/bootstrap.min.css"
   "src/styles.css"
],
"scripts": [
   "node_modules/popper.js/dist/popper.min.js",
   "node_modules/jquery/dist/jquery.min.js",
   "node_modules/bootstrap/dist/js/bootstrap.min.js"
]

This is the way I have added Bootstrap, but there are a lot of other ways to install Bootstrap into an Angular app.

I could have added the links to the css and the javascript in the index.html file instead, but this way it gets packaged and minified into the app.

The blog post on [5] helped me a lot.

The way I am using, is not necessarily the right way. Here are some other ways to use Bootstrap with Angular.

use NgBootstrap⁶

I personally chose not to go this route, as it means spilling a lot of Bootstrap-specific stuff into my typescript code.

use ngx-bootstrap⁷

Basically has the same ideas as NgBootstrap

References

[1] Bootstrap - Getting started 3.4

https://getbootstrap.com/docs/3.4/getting-started/

[2] JQuery

https://jquery.com/

[3] Popper.js

https://popper.js.org/

[4] Bootswatch - Themes for Bootstrap

https://bootswatch.com/help/

[5] Using Bootstrap with Angular

https://medium.com/codingthesmartway-com-blog/using-bootstrap-with-angular-c83c3cee3f4a

[6] NgBootstrap

https://ng-bootstrap.github.io/

[7] ngx-bootstrap

https://github.com/valor-software/ngx-bootstrap

Uploading a file in Angular

I wanted my users to have the ability to upload an image via de Angular App that I provide for them.

So let me explain the process.

First, of course, there's the HTML that needs to be added to support File Uploads.

You can add these to your app.component.html file.

Then, once it is submitted it needs to be handled by your Angular app, in the app.component.ts file.

Then it gets transmitted to your rest service on the backend.

And that stashes it into the database using JPA.

All done.

References

StackOverflow - Angular File Upload

https://stackoverflow.com/questions/47936183/angular-file-upload

StackOverflow - Angular 5: How to upload an image

https://stackoverflow.com/questions/48343853/angular-5-how-to-upload-an-image?noredirect=1&lq=1

AUTO flush

So, to my sorrow, I recently found out that the old product uses hibernate.cfg.xml and the new product we're developing uses persistence.xml.

So, whilst developing for the new product, I ran into issues.

Issues where a query sent to the database, would trigger a flush of all my dirty changes in my Session to the database.

This is because the Session is running with FlushModeType.AUTO, which is the default.

Problem was,... I was at the moment still building an entity to be persisted, and it wasn't ready yet.

Apparently I now have to be very careful in which sequence I do things.

Suffice to say, I am not amused.

Either that, or I switch over to FlushModeType.COMMIT, which means I can query the database all I want, and the flushing of my dirty entities in the session will take place only when I commit.

This is very convenient, unfortunately the disadvantage is that I might not see the dirty data in the database during my queries. But I feel this is well worth it.

References

Vlad Mihalcea - How does AUTO flush strategy work in JPA and Hibernate

https://vladmihalcea.com/how-does-the-auto-flush-work-in-jpa-and-hibernate/

Vlad Mihalcea - How do JPA and Hibernate define the AUTO flush mode

https://vladmihalcea.com/how-do-jpa-and-hibernate-define-the-auto-flush-mode/

GitHub - Jakarta Persistence project

https://github.com/eclipse-ee4j/jpa-api

HTTP Caching Headers

I recently wanted to add some rather aggressive caching, to prevent my server getting hit too often.

Naturally, I wanted this caching to take place on the client, i.e. the webbrowser.

So, there have traditionally been advantages and disadvantages to caching.

Advantage: the server doesn't get hit again, and your website feels more responsive.

Disadvantage: it is possible to see outdated cached data.

Invalidating cache entries is basically an entire study of its own.

In my case I took the obvious route: the data stays the same forever.

This facilitates my caching wonderfully.

response.addHeader("Cache-Control","private, max-age=31536000");

This stores the item in the cache for one year (31536000 seconds).

Private means caching in the client browser, public means that it might also be elligible for caching in all other intermediaries between de browser and the server.

The blogs in the references are very clear.

References

[1] Heroku Dev Center - Increasing Application Performance with HTTP Cache Headers

https://devcenter.heroku.com/articles/increasing-application-performance-with-http-cache-headers

MDN Webdocs - HttpHeaders - Cache Control

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control

MDN Webdocs - HTTP caching

https://developer.mozilla.org/en-US/docs/Web/HTTP/Caching

Devoxx 2019 - wrapup

I just thought I'd mention the sessions I saw, and the sessions that are on my "todo" list.

That which is Seen

Rust for Java Developers - by Paul Watson, Andy Bell

It's always a good thing to learn a new programming language. Rust is similar to C, with an interesting take on no-garbage-collectors and all sorts of goodies. Compiles to native.

The Web is on 🔥 F.I.R.E. 🔥 !!! - by Noël Macé

Progressive Web Apps, new standards that are not standard yet, but already available in Chrome or Chrome Canary.

Broken Buildtools and Bad Behaviors; The Maven Story - by Robert Scholte

An interesting take on what to do and what not to do with Maven.

Money, Money, Money, can be funny with JSR 354 - by Werner Keil

A bit boring, but only until you realize what kind of freakish money and money rules there are out there. Already it's available in standard Java, and there's no good reason not to use it.

Solving Memory Leaks in the JVM - by Kirk Pepperdine

just as it says, a very good explanation also about how the memory management in Java works.

Java 9 and Beyond - by Venkat Subramaniam

Hey, it's Venkat, always good.

Evolving a Pragmatic, Clean Architecture - A Craftsman's Guide - by Victor Rentea

a lot of the lessons seem hauntingly familiar. Some pieces were unknown. Some things I had doubts about.

Welcome to LWorld: The current state of value types in Java - by Tobi Ajila

a very good entry-level and advanced lecture about the new value inline types

Collections Corner Cases - by Stuart Marks

An awesome talk about things I didn't know, but have been using for ages.

What's coming in Scala 3 - by Josh Suereth, James Ward

Very interesting and advanced, must be easier for people who have more familiarity with Scala.

Why We Hate Java Serialization And What We're Doing About It - by Stuart Marks, Brian Goetz

Some ideas about extending Serialization to how it should have been. Right now only a rough plan.

A Gentle Introduction to Angular Schematics - by Matt Raible

Basically the stuff you can use to adapt the Angular CLI to what you want it to do. Think "templates" on speed.

The time relativity principle - by Jakub Marchwicki

Useful if only for the fact that we get up to some strange shenanigans when it comes to timezones

Ask the Java Architect - by Brian Goetz, Stuart Marks

Always fascinating to find out the reasons behind the new designs for Java.

Advancements in Angular - by Jeremy Elbourn

Fascinating talk on the internals by a core member of Google Angular team

Is your AI plotting to kill you? Why AI and Machine Learning is dangerous - by Ben Vermeersch

A nice talk for the tin-foil hat people, and people interested in finding out the weaknesses of AI en how to beat AI at their own game.

How to get properly hacked! - by Julien Topçu

informative, but most things I already knew. The xml entities hack was new for me, though.

From Java to Golang, the journey of the nexuzhealth team - by Dylan Meeus

fascinating talk about the pitfalls of moving to Golang. A little short on information on how to convert large legacy systems.

Java Language Futures: Late 2019 Edition - by Brian Goetz

Always a pleasure.

That which is Unseen

Exploring Collectors - by Venkat Subramaniam

It's Venkat!

Top Tips for Terrible Tech Talks - by Chet Haase

Heard good things about it.

Project Loom: Helping Write Concurrent Applications on the Java Platform - by Ron Pressler

Because Project Loom sounds interesting.

More Than a Query Language: SQL in the 21st Century - by Markus Winand

Learn things about SQL that you didn't know existed.

The 7 biggest mistakes we made in 10 years of running a major open source project. - by Roel Spilker, Reinier Zwitserloot

Colleagues liked it.

The past, present and future of the Java type system - by Alasdair Collinson

As Venkat put it, will we ever see a List<int>?

RSA is Dead. Long Live BB84! - by James Birnie

Because encryption is cool.

Beyond ByteBuffers - by Brian Goetz

ByteBuffers are bad? Really?

Memory footprint of a Java process - by Andrei Pangin

Perhaps could be interesting.

References

YouTube - Devoxx

https://www.youtube.com

Devoxx.be

https://devoxx.be/

Bootstrap and Darkmode

Recently (well, not that recently) an issue was submitted to my hobby project that the background, which was very bright white, was a little off putting. Apparently people are surfing on my website at night.

Since I have a new monitor¹, I must say that the new monitors indeed transmit a staggering amount of light.

Of course, this means something has to be done about it (apparently) and it's called Darkmode².

Personally, I find that a bright screen enables me to keep working in the dead of night without becoming tired, but that's just me. And probably not entirely healthy.

Anyway, I am using Bootstrap³, so I was looking for a Bootstrap Theme to fix this problem.

A lot of really FANTASTIC themes for Bootstrap can be found on Bootswatch⁴.

I choose Cyborg⁵ from their selection.

The switching

Next is how to provide the websurfer with a choice between the two modes, normal mode and darkmode.

You can find a lot of different ways of doing it, just surfing around.

Now, obviously, I do not wish to add all sorts of classes to my html pages. I just want to replace the appropriate links to the .css files in my html.

I have two different setups in my current website, static pages through a templating engine and two SPA using Angular.

Static pages

For the static pages, the problem is actually non-existant. I just add the appropriate css- and javascript-links in my webpage based on a Darkmode cookie setting, and away we go.

Angular

For Angular, I must confess I used a very low-tech solution, that some of you might not like.

Basically there is a index.html in my Angular app, that contains the link to the css file and I just replace the link using javascript.

I do this in the ngOnInit() of the central Angular component.

As follows:

References

[1] My new monitor

https://randomthoughtsonjavaprogramming.blogspot.com/2019/07/a-new-monitor.html

[2] Wikipedia - Light-on-dark color scheme

https://en.wikipedia.org/wiki/Light-on-dark_color_scheme

[3] Bootstrap

https://getbootstrap.com/

[4] Bootswatch

https://bootswatch.com/

[5] Bootswatch - Cyborg

https://bootswatch.com/cyborg/

Devoxx 2019

Hello, there!

I am currently at Devoxx 2019, in Antwerpen for the entire week.

I'll try and write some blogposts on it.

Freemarker and java.time

I noticed that Freemarker¹ does not yet support the Java SE 8 Date and Time². I noticed this, because I tried it. It just tries to parse the toString() and it fails.

There's already blogs about it.

So I had to add a little dependency to provide this, courtesy of Jakob Vad Nielsen³.

<dependency>
    <groupId>no.api.freemarker</groupId>
    <artifactId>freemarker-java8</artifactId>
    <version>1.1.5</version>
</dependency>

Of course, it's not native support, you still have to call specific methods to parse this. But it works in my case.

It means I have to change some of my templates around, as described below:

Published Date ${blog.createDate?datetime}

Instead should be:

Published Date ${blog.createDate.format()}

References

[1] FreeMarker Java Template Engine

https://freemarker.apache.org/

[2] Java Technical Details - Java SE 8 Date and Time

https://www.oracle.com/technical-resources/articles/java/jf14-date-time.html

[3] GitHUb - Java library adding support for the Java 8 java.time api to FreeMarker.

https://github.com/lazee/freemarker-java-8

Git Worktrees

Since git 2.8 it is possible to work with "Worktrees". These are extra maps you can use to checkout a branch. This is instead of changing your current working directory contents by checking out a different branch (and by extention messing up your build targets and artifacts and stuff).

At work, unfortunately, we are using Fedora Core 25 (a little old, but we are sure to update any day now).

So I decided to install git from source¹. Apparently we're already up to version 2.23.0

At home, I do not use worktrees², and I managed to avoid it at work too. But the situation has progressed in such a fashion that the branches are starting to differ soo much that a reset maven profiles/clean/rebuild/redeploy is required when switching branches, costing me valuable development time.

Creating a new worktree from the master

git worktree add -b feature_branch ../new_folder

Creating a new worktree from the master from an existing branch

git worktree add ../new_folder feature_branch

Use --detach instead of -b branch to create a worktree without a branch.

Gebruik --detach ipv -b branch om een losgekoppelde worktree aan te maken zonder een branch.

Removal of a worktree

rm -rf ../new_folder && git worktree prune

It all seems easy enough.

For some reason the new git also enables me to simply checkout the svn repo (we're still using SVN at the moment) without any pains like in the past (for example bombing out because it takes too long). Perhaps something has changed in our SVN setup, I don't know.

References

[1] Git - see kernel.org link on the page

https://git-scm.com/download/linux

[2] Git - manpage worktree

https://git-scm.com/docs/git-worktree

StackOverflow - What would I use git-worktree for?

http://stackoverflow.com/questions/31935776/what-would-i-use-git-worktree-for

Atomic Object - Parallelize Development Using Git Worktrees

https://spin.atomicobject.com/2016/06/26/parallelize-development-git-worktrees/

Atlassian - Six cool features of the Git 2.x series

https://developer.atlassian.com/blog/2015/10/cool-features-git-2.x/

GAUI.SI - Git worktree feature

https://www.gaui.is/git-worktree-feature/

Hello, production

Recently saw this blogpost¹ among incoming tweets, and it's good enough to mention here.

References

[1] Pete Hodgson - Hello, production

https://blog.thepete.net/blog/2019/10/04/hello-production/

Ref: Filtering a Stream of Optionals in Java

Recently got a little annoyed that I always have to combine a filter with Optional::isPresent with a map with Optional::get.

A quick search on the Internets, found a good resource about how this can be done, and in the future should be done.

See the references. Let me know if you have something better.

References

Baeldung - Filtering a Stream of Optionals in Java

https://www.baeldung.com/java-filter-stream-of-optional

Automating Importing Let's Encrypt Certificates into Payara/Glassfish

Well, I already made a blogpost on letsencrypt certificates in payara¹.

In this blogpost I mentioned a python script that automatically renews certificates for payara². Unfortunately, it's not quite a perfect fit for me, as I do not run payara on port 80, which the script requires.

My setup has an nginx running on port 80, that basically forwards all requests to the https version of the website.

So I could simply turn off nginx, re-certify and turn nginx back on.

This simple bash script is as follows:

Running it will give you:

$ source renew_letsencrypt_certificates.sh
---- Starting.
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/www.mrbear.org.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for www.mrbear.org
Waiting for verification...
Cleaning up challenges

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/www.mrbear.org/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/www.mrbear.org/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
---- Adding certificates...
/home/payara/payara5/glassfish/domains/production/config/keystore.jks
Private key with alias [s1as] added to keystore /home/payara/payara5/glassfish/domains/production/config/keystore.jks.
Command add-pkcs8 executed successfully.
---- Disabling http listener...
server.network-config.network-listeners.network-listener.http-listener-2.enabled=false
Command set executed successfully.
---- Enabling http listener...
server.network-config.network-listeners.network-listener.http-listener-2.enabled=true
Command set executed successfully.
---- Done.

And your certificate is renewed. I am most pleased that this is soo easy in Payara/Glassfish.

In order to properly automate this, the master password must be stored somehow, and it's possible to do this via the --passwordfile=passwordfile.txt commandline parameter.

I found a good blogpost on how to do this at .Lost in Coding³.

And also the blogpost on [4] to get me started on how to do this.

Verifying that the keystore contains the appropriate stuff, can be done as follows:

# keytool -list -keystore /home/payara/payara5/glassfish/domains/production/config/keystore.jks
Enter keystore password:
Keystore type: jks
Keystore provider: SUN

Your keystore contains 2 entries

glassfish-instance, Jan 22, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): WZ:B6:4N:8V:AT:YP:QC:9N:VT:HA:WI:NQ:B[:V8:0W:YT:B8:AW:YT:VQ
s1as, Sep 25, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): N8:5Y:AN:SV:N8:9W:4T:Y7:9W:V9:W4:VT:8Y:WV:4N:W4:VT:FB:8S:4E

To get more information, try:

keytool -list -v -keystore /home/payara/payara5/glassfish/domains/production/config/keystore.jks

Or for a specific alias:

keytool -list -v -keystore /home/payara/payara5/glassfish/domains/production/config/keystore.jks -alias s1as

References

[1] Enabling SSL in Payara with certificate from Let's Encrypt.

https://randomthoughtsonjavaprogramming.blogspot.com/2019/01/enabling-ssl-in-payara-with-certificate.html

[2] github - Let's encrypt script to retrieve and upload keys/certificates to Payara

https://gist.github.com/ratcashdev/1b09877d37e02ef5170bf9e60c377f34

[3] .Lost in Coding - Configure Payara Server passwords in Docker

https://ondro.inginea.eu/index.php/configure-passwords-in-payara-server-and-glassfish/

[4] Payara Blog - TLS certificates with Payara Server and Let's Encrypt

https://blog.payara.fish/configuring-ssl/tls-certificates-with-payara-server-and-lets-encrypt

Devvalley Java Meetup

Two sessions were being given at the java dev meeting on the 26th of September 2019, at OpenWeb^{1 2} in Nieuwegein.

I just thought I'd write some of the things down that were being explained.

Reactive Streams

The first session, provided by Tim van Eijndhoven, was very interesting.

One of the limitations of the Stream API of Java, is that there is no implementation for "back pressure", i.e. the consumer of the stream cannot indicate to the producer of the stream the chunk size of the work it can handle.

Several implementations of reactive streams are already available.

• RxJava (I noticed it looks a lot like RxJs, and no wonder. It is both based on Reactive Extentions³.)
• Project Reactor⁴
• Akka Streams⁵

RxJava

It still targets java 6, which makes it very suitable for android development, where it is very popular.

It's used by the Vert.x framework.

Project Reactor

A newcomer available from Pivotal. It's built on Java 8, on version 3 and is used by spring.

Has a feature to pass context.

Akka Streams

Akka Streams is from Lightbend, and is part of Akka. A framework built around Scala.

It seems to be quite different compares to the other two implementations.

There is an Alpakka Library, containing integrations with other systems, for example ElasticSearch, MongoDb, AWS APIs, etc.

Reactive Streams in Java⁶

Apparently there is already an active JEP (JEP 266) to get Reactive Streams into Java 9.

The different interfaces are called Flow.Publisher, Flow.Subscriber, Flow.Subscription and Flow.processor.

The existing implementations mentioned above all already have functionality to proxy their classes into the Java 9 interfaces.

Some notes

The Publisher is responsible for making decisions regarding what data to send to the Subscribers. For instance to skip data or to send all old data as soon as the subscriber subscribes.

And of course because of the "back pressure" thing, a publisher will have to keep track of how many items has been sent to the subscribers, and how many the subscriber can still receive before the publisher needs to wait for the next request.

The difference seems to be between cold streams and hot streams.

Where cold streams, a new subscriber will receive the same stream, starting from the beginning.

Where hot streams, data that has passed, will not be sent to new subscribers. Ideal for quickly changing values, where the recent value is the most valuable.

Testing is hard, because it's asynchronous. All the frameworks provide ways of adding unit tests.

Frameworks provide performance via:

• macro fusion: replacing subsequent operators with a single operator
• micro fusion: share resources or internal structure between operators.

The reactive stream interfaces are easy, but hard to get right frameworkwise.

The learning curve is pretty steep, though.

There are two points, usually, in the stream where you can decide to fiddle with thread-usage.

Supercharged Microservices

The second speaker was Dave Franken.

The second session was regarding the use of Kotlin⁷, and GraalVM⁹ and Quarkus⁸ to supercharge your microservices, making them both small (memory wise), with very fast bootup times (seriously very fast!), and can be developed quickly in Kotlin.

That is basically all I have to say about that.

References

[1] Devvalley 2019 - Open source meetup of the future

https://devvalley.openweb.nl/

[2] Devvalley Java Meetup

https://www.meetup.com/nl-NL/Devvalley/events/261802586/

[3] Reactive

http://reactivex.io/

[4] Project Reactor

https://projectreactor.io/

[5] Akka Streams

https://doc.akka.io/docs/akka/current/stream/index.html

[6] JEP 266: More Concurrency Updates

https://openjdk.java.net/jeps/266

[7] Kotlin Programming Language

https://kotlinlang.org/

[8] Quarkus - Supersonic Subatomic Java - A Kubernetes Native Java stack tailored for GraalVM & OpenJDK HotSpot, crafted from the best of breed Java libraries and standards

https://quarkus.io/

[9] GraalVM Run Programs Faster Anywhere

https://www.graalvm.org/

Stream to Optional

Was looking for a function that returns an Optional containing a value if it was the only value in the stream (size of stream is 1) and an empty Optional in every other case.

I came up with the following:

The limit that is put upon it, might make matters really performant, but perhaps not in the case of parallel streams.

See the javadoc of limit() on why this is.

Google Doc Smart Quotes

I recently had to look up how to disable Smart Quotes in Google Drive.

I found the solution after a few seconds googling¹.

Just thought I'd paste it in here, in case I cannot find it again.

Option 1

Disable Smart Quotes by clicking on "Tools" -> "Preferences -> turn off "Smart quotes".

Option 2

Leave Smart Quotes enabled. For straight quotes, immediately press Ctrl+Z to Undo the AutoFormat.

The post had some additional info on how to enter smart quotes, even though smart quotes was disabled, but that didn't work for me, and I didn't need it.

References

[1] Google Groups - Change Smart Quotes To Dumb Quotes

https://groups.google.com/forum/#!topic/microsoft.public.word.formatting.longdocs/tyKtBNWlZss

Converting an Optional to a List

Just a quick small snippet on how to convert a Optional to a List.

Hope it helps someone, who's new at Optionals.

JOpt Simple

Just a small note regarding a library I use.

According to the website¹:

JOpt Simple is a Java library for parsing command line options, such as those you might pass to an invocation of javac.

References

JOpt Simple

http://jopt-simple.github.io/jopt-simple/

A New Monitor

The times move fast. Must faster than I do, when it comes to computer hardware.

As mentioned in the last blog, I would reveal what kind of monitor I have been using for all this time.

Above is a picture of my new setup.

The Old Monitor

An Iiyama original. The Vision Master Pro 454 HM903DT B

It was quite a good monitor at the time and it did manage to perform fine for decades.

The New Monitor

The new monitor is visible at the top of the blog. It's a HP¹ Z2n G2 27 inch monitor and it is awesomeness.

Some notes:

the stand

very sturdy heavy metal adjustable stand

incredible contrast

especially compared to my old monitor. I had to up the anti-aliasing on my games, as the jaggies were very noticeable, now that the contrast is awesome.

easy setup

just a little difficult to find the buttons.

the brightness

the brightness needs some getting used to, especially if the screen is a bright white. Especially in the evenings, the brightness needs to be turned down.

size/weight

it's obviously a huge difference in size and weight. The desktop (the real one) has a lot more real estate now. Getting used to having more room to put the keyboard.

energy

I expect a lot of energy savings with the new monitor, as well as a reduction in generated heat.

operating system

zero problems in all my operating systems in getting the highest resolution. Almost didn't have to do anything at all.

Conclusion: with the new graphics card and the new monitor I sit and stare in silent admiration at the beautiful images that my games can conjure up for me.

References

HP Website - HP Z27n G2 27 Monitor

https://store.hp.com/NetherlandsStore/Merch/Product.aspx?id=1JS10AT&opt=ABB&sel=MTO

New Second Hand Graphics Card

I tend to postpone upgrading my computers until it either breaks down or it gets too cramped in performance/memory/etc. This tends to take a long time.

So a colleague of mine was upgrading his computer, and I managed to get his old graphics cards.

Seeing as I am on a budget, I was looking into what the difference in performance were for second hand graphics cards.

A lot of graphics cards are replaced because there are better ones, even though the old cards still work perfectly fine.

And he just happens to buy the latest and greatest, which is still a pretty good deal after 7 years, compared to what I buy.

My original graphics card

It's a AMD Radeon HD 5700 Series.

"New" graphics card

It's a Nvidia GeForce 770.

Notes

Some notes of mine.

type

you cannot tell any more what kind of graphics card it is from just looking at the physical hardware. With any luck you can find the manufacturer, but that's it. There's just not more room or not important.

external connectors

So, no more VGA connector. And of course my old monitor only has a VGA Connector. (Don't ask, I'll write about it in the next blog about how ancient it is). It gave me an excuse to upgrade my monitor.

internal power cable for PCI Express cards

The old graphics card had only one internal power connector, a 6 pin. The new graphics card had both a 6 pin and an 8 pin. Time to look in the attic for my spare cables box.

noise

it's rather depressing that the new graphics card has three fans and is more quiet than my old graphics card which has only one fan.

heaviness

with all the fans, and the cooling fins, this new graphics card is a seriously heavy beast in pure weight. I hope my motherboard can cope with it.

plastic

instead of screws, my case has plastic easy-to-use click-and-go thingies to fasten cards. Of course, plastic is the first thing to become brittle after several years. No surprises there.

All in all, I am extremely happy. Except for a few little surprises mentioned above, I had the entire setup working relatively quickly.

References

NVidia - GeForce GTX 770

https://www.geforce.com/hardware/desktop-gpus/geforce-gtx-770/specifications

Deleting a domain in Payara/Glassfish

Back from vacation.

Just quickly putting this in my notes:

$ cd payara5
$ cd bin
$ ./asadmin delete-domain
Enter the value for the domain_name operand> domain1
Domain domain1 deleted.
Command delete-domain executed successfully.

Converting DVD to MP4 using Linux

I recently wished to convert some of my childrens DVDs to MP4, so I could put them on their individual tablets.

I've dabbled with this in the past, on Linux, and it has always been quite a chore to do.

Everything from obscure commandline commands and GUIs that provided a lot of options that were basically meaningless to me (without proper research).

Of course, this caused a lot of grief. Sound not synchronised to the video, or the entirely wrong soundtrack, or converted only the trailers, etc.

But recently I found Handbrake¹, and I was much impressed. A nice gui that only takes a little while to learn, and only for special cases (say for example if you do not wish to have the default English soundtrack).

It has sensible defaults.

The only issue was the installation, as it requires, of course, libcss and all that.

References

[1] Handbrake

https://handbrake.fr/

Unit Testing vs. Integration Testing

I was talking with a colleague on the way home, when we started talking about his troubles with his bike.

In short, he could turn the pedals like crazy, and the bike was not moving an inch.

That's when I realized that this is a perfect example of the difference between Unit Testing and Integration Testing.

In short:

unit test

write a unit test, to see if we can properly turn the pedals in the proper direction.

Conclusion: Success! Software works fine. In fact it works extremely well. We achieved a performance increase of over 500%! Yay!

integration test

write an integration test, to see if when we turn the pedals, the bicycle will move.

Conclusion: Failure! Pedals turn fine, but the bicycle doesn't move.

It turns out the integration between the component "pedals" and the component "wheels" was busted.

It totally shows that though Unit tests may all pass, it is not a guarantee that your software is actually doing what it should.

Lesson learned.

Abuse of Lambdas

So, sometimes when you have an existing software structure, with lots of domain rules inside, it becomes difficult to change. Especially when there is a lot of cohesion between your classes.

This is the problem of tight coupling, as opposed to loose coupling¹.

I recently encountered this in our software.

And of course, I needed to change something deep inside.

Now, I didn't need to change anything about the domain logic. It was very good and working as intended.

But I needed a change purely for technical reasons. In this case, the domain logic was used in a Batch Processing on Hibernate Entities, and the domain logic was (in this case, not normally) generating a lot of Entities, and they were all persisted in the Session Context, which naturally grew. And then the dirty check and the session closing takes a long time, etc, etc. See reference [2] on why this's bad.

So I was looking for a way to persist entities, without adding them to all the collections with the CascadeType without changing the domain logic too much, and also keep the default behaviour as standard.

I didn't want to use inheritance, as it was a persisted Hibernate Entity.

So, I came up with the following hack:

Quite simply, I can now add behaviour to an Entity when I need it.

For example:

As me and my colleague say "A seemingly innocent first step on a slippery slope to hell."

Of course, this kind of abuse has always been possible, but it is now easier to do than ever (without having to create an entire class for the priviledge).

References

[1] Wikipedia - Loose coupling

https://en.wikipedia.org/wiki/Loose_coupling

[2] Vlad Mihalcea - The best way to do batch processing with JPA and Hibernate

https://vladmihalcea.com/the-best-way-to-do-batch-processing-with-jpa-and-hibernate/

MERGE SQL Statement

I recently had to do a insert on the database, if the records weren't there, and an update if it was.

Performance was a problem.

Colleague of mine had the same problem when using an insert and update statement with an (exists (select 1)) on 2.5 million records.

After 5 minutes, 5000 rows were done. It would have taken 41 hours to fix them all.

After a merge¹ statement, the entire thing was done in 2.5 minutes.

So, I had to get to grips with the merge statement as well. I hadn't used it before.

There's plenty of tutorials on how it works.

Oracle allows a DELETE statement in the MERGE statement, which seems not to be defined in the SQL Standard.

References

[1] Wikipedia - Merge(SQL)

https://en.wikipedia.org/wiki/Merge_(SQL)

[2] Oracle PL/SQL Documentation - MERGE

https://docs.oracle.com/cd/E11882_01/server.112/e41084/statements_9016.htm#SQLRF01606

Proper use of Optional

Using Optional is hard in the beginning.

Especially for first-timers one has a tendency to use the wrong methods.

Take the following sequence of refactoring for example:

First without any kind of Optional:

A first bad attempt at an optional:

It looks like the person just... kind of removed the Optional back to a situation that he knew and was more familiar with.

Let's try this again:

Now this is a good try. It is very common to see the two Optional class methods being used together. isPresent() and get().

But we can do better.

It's often the same: we wish to do something, only when there's a value available.

Like so:

Of course, it's sometimes better to create a method reference:

But I do find this proliferates the number of methods, but it's an acceptable side effect.

We finally arrive at something concise and easy to read, once you get the hang of it.

Antisamy - false positive security issue

Some time ago, I received a security notification from Github regarding a vulnerability^{1 2} in Antisamy.

It took me a while to find out that it was a false positive.

But the issue in [1] did mention that Antisamy seems to be lacking a maintainer. A worthwhile replacement was suggested as being HTML Sanitizer^{3 5 6}.

I started using the HTML Sanitizer in my project and I like it. The fact that I don't need an XML configuration file is a plus.

There's a StackOverflow answer on why and how of the two projects⁴.

Only thing missing are the policy configuration files from AntiSamy, which are not present in any way in HTML Sanitizer, so you either have to write your own or get them from here⁷.

The file is called HTMLSanitizerAntiSamyPolicy.java.

References

[1] Github - Antisamy issues

https://github.com/nahsra/antisamy/issues/32

[2] CVE details - CVE-2018-1000643

https://www.cvedetails.com/cve/CVE-2018-1000643/

[3] OWASP Java HTML Sanitizer Project

https://www.owasp.org/index.php/OWASP_Java_HTML_Sanitizer_Project

[4] Java: Owasp AntiSamy vs Owasp-java-html-sanitize

https://stackoverflow.com/questions/28577738/java-owasp-antisamy-vs-owasp-java-html-sanitize/29259874

[5] GitHub - OWASP/java-html-sanitizer

https://github.com/OWASP/java-html-sanitizer

[6] MailingList - HTML Sanitizer

https://lists.owasp.org/pipermail/owasp-leaders/2011-March/004883.html

[7] GitHub - ESAPI/esapi-java-legacy - New file to use Java HTML Sanitizer that implements old AntiSamy poli…

https://github.com/ESAPI/esapi-java-legacy/commit/d48e5a6f07601322c44c113058526eb133b777a5#diff-72ca1caf39f169db1ab83d2af2ec6cc3l

Refactoring into Streams

So, as often happens, I encountered the following code at work:

It's basically a zipped file, containing other zipped files, containing supermarket sales item data. This little bit of code just decides if a file should be parsed.

SonarLint quite rightly complained "Refactor this method to reduce its Cognitive Complexity".

And I quite quickly came up with the thought that it's really a list, where we can stop evaluating the list after the first match.

So, it becomes something like:

But, of course we have streams now:

It looks better, no?

It's a trivial example, but I like it.

Addendum

A colleague mentioned that I should do a bit of this differently. Move some more stuff into the enum.

The result becomes something like this:

Moving knowledge into the Enum, seems to be a better fit here.

Some comments on blog comments

So, for a long time I've had comments enabled on this blog for anyone and without moderation.

And I did get a fair amount of spam comments about other websites, especially those polite bastards giving all that training in Chennai.

Stuff like this:

I have read your blog its very attractive and impressive. I like it your blog.

Java Training in Chennai Java Training in Chennai | Core Java Training in Chennai

And I've decided to turn on moderation to the max.

This means that any comments will be moderated by me, at my convenience.

So, if you're comment does not show up on the article in question right away, my most sincere apologies.

I will try and keep up.

Regards,

Did you ever do this?

So I was working on a class which was too big. Time to pull it apart into helper classes (in the same package). Then I noticed a number of methods were protected.

Now protected means it can be both accessible from subclasses as well as classes in the same package. (That last part is often forgotten by me.)

Which means when pulling things apart into classes, I can simply call these protected methods. Yay!

Unfortunately, these protected methods were defined in a superclass and the superclass was declared in... you guessed it... a different package.

The ugly hack that solved this is the following:

Of course, I deleted this, because to my mind overriding a method, just to increase the visibility is a huge code-smell. (Or isn't it?)

But I was wondering if anyone else ever did this? (due to time pressures and not being able to change the super class, etc.)

And what would be a more pretty work around, if you have one?

Angular

The blog about Angular JS (http://blog.angularjs.org/) has moved (last year already) and is now, as it should be The Angular Blog.

It can be found at https://blog.angular.io/.

On Rewriting Software

I came across the reference¹ below on "rewriting" software, and it struck a cord with me.

Turns out that, as is so often the case, the fact where "you should never ever do that"² has its successful exceptions.

References

[1] Lessons from 6 software rewrite stories

https://medium.com/@herbcaudill/lessons-from-6-software-rewrite-stories-635e4c8f7c22

[2] Joel Spolsky - Things You Should Never Do, Part I

https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/

Adventures in Shell Scripts

A very simple and short blogpost today, sorries.

A colleague decided to automated some simple task by means of a shell script.

This script caused the following error:

rm: cannot remove 'tmp/*': No such file or directory

I quite quickly (after adding -f to defeat the are-you-sure-prompt) found out that double quotes indicated that it should take stuff literally. In this case the script was trying to actually delete the file tmp/*, and not finding it.

The shell did not expand the * properly when encased in double quotes.

A simple example for you to try if this is the case in your environment would be:

echo *

Instead of

echo "*"

Installing MariaDB 10 on CentOS 7

I have always used the stock install of MariaDB that came with CentOS 7, but I noticed recently that it is still only on 5.5 (at the time of writing, 5.5.60).

I never had any issues with it, until I tried to upgrade my Liferay CMS system. Apparently there are definite changes that are not incorporated into the 5.5.

I wished to moved to 10.3, which is the latest stable branch, and I can apparently do that (as indicated on the mariadb website¹) by creating a MariaDB repository for DNF and then installing as usual.

Of course, first uninstall the 5.5 version.

References

[1] MariaDB - Installing MariaDB 10 on CentOS 7 / RHEL 7

https://mariadb.com/resources/blog/installing-mariadb-10-on-centos-7-rhel-7/

MariaDB - Moving to MariaDB Backup

https://mariadb.com/resources/blog/moving-to-mariadb-backup/

Backup & Restore of Payara Server

How do I backup and restore my domain in Payara (or glassfish for that matter.)

$ ./asadmin backup-domain --backupDir ~/mmud_backups/payara_backups domain1

It ends up in ~/mmud_backups/payara_backups/domain1/domain1_2019_01_27_v00001.zip

$ ./asadmin restore-domain --filename ~/mmud_backups/payara_backups/domain1/domain1_2019_01_27_v00001.zip --backupdir ~/mmud_backups/payara_backups domain1
Restored the domain (domain1) to ~/payara5/glassfish/domains/domain1
Command restore-domain executed successfully.

References

Payara - Backup & Restore of Payara Server

https://blog.payara.fish/backup-restore-of-payara-server

Payara - How to upgrade Payara Server 5

https://blog.payara.fish/how-to-upgrade-payara-server-5-reloaded

Nginx - Redirecting to HTTPS

For automatic http->https redirection it is recommended to use Apache or Nginx in front of Payara to do the redirection, see [1].

I decided on using nginx for this^{2 3 4}.

Before nginx, it looks like this:

$ curl -s -D - http://www.mrbear.org -o /dev/null
HTTP/1.1 200 OK
Server: Payara Server  5.184 #badassfish
X-Powered-By: Servlet/4.0 JSP/2.3 (Payara Server  5.184 #badassfish Java/Oracle Corporation/1.8)
Content-Type: text/html;charset=ISO-8859-1
Transfer-Encoding: chunked
X-Frame-Options: SAMEORIGIN

Configuration is found in /etc/nginx/nginx.conf.

I changed it into something very simple:

server {
      listen      80 default;
      access_log off;
      error_log off;
      return      301 https://$server_name$request_uri;
}

Double checking:

# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Enabling automatic start and also starting nginx.

# systemctl enable nginx
# systemctl start nginx

So, let's try seeing what the webserver now responds with:

$ curl -s -D - http://www.mrbear.org -o /dev/null
HTTP/1.1 301 Moved Permanently
Server: nginx/1.12.2
Date: Mon, 28 Jan 2019 16:28:59 GMT
Content-Type: text/html
Content-Length: 185
Connection: keep-alive
Location: https://www.mrbear.org/

$ curl -s -D - http://www.mrbear.org/this/is/some/url.jsf -o /dev/null
HTTP/1.1 301 Moved Permanently
Server: nginx/1.12.2
Date: Mon, 28 Jan 2019 16:31:06 GMT
Content-Type: text/html
Content-Length: 185
Connection: keep-alive
Location: https://www.mrbear.org/this/is/some/url.jsf

References

[1] Payara - How to Secure Payara Server with Apache

https://blog.payara.fish/how-to-secure-payara-server-with-apache

[2] How To Nginx Redirect All HTTP Request To HTTPS Rewrite 301 Rules

https://www.cyberciti.biz/faq/linux-unix-nginx-redirect-all-http-to-https/

[3] How to do an Nginx redirect

https://bjornjohansen.no/nginx-redirect

[4] Creating NGINX Rewrite Rules

https://www.nginx.com/blog/creating-nginx-rewrite-rules/

DigitalOcean - How To Install Nginx on CentOS 7

https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-centos-7

Angular App Upgrade

For simple Angular¹ apps, it just might make more sense to rewrite them, than to upgrade them.

This might seem easier as I haven't done anything with my Angular app lately, it's still on Angular 4 (4.0.0) and Angular 7 (7.2.0) is out by now.

Added benefit is that I see firsthand the different changes, and I can blog about it.

Let's see what has changed, and how I am going to replace certain elements.

General setup

Creating the App

Start by creating the application.

$ ng new my-app

Bootstrap

Adding bootstrap can be done using [2].

$ npm install bootstrap --save

Added the bootstrap css to the styles in angular.json.

"styles": [
  "src/styles.css",
  "node_modules/bootstrap/dist/css/bootstrap.min.css"
],

I am not planning on using NgBootstrap², I think it's just a wrapper.

Material Icons

I need me some icons⁴, without getting *everything*.

$ npm install --save material-icons

Added the icons css to the styles in angular.json.

"styles": [
  "src/styles.css",
  "node_modules/bootstrap/dist/css/bootstrap.min.css",
  "node_modules/material-icons/iconfont/material-icons.css"
],

Miscellaneous settings

I usually make some small changes in the angular.json. These are documented here.

"outputPath": "../webapp",

Components and Services

It's possible to generate a skeleton for components and services we wish to design.

$ ng generate component errors
$ ng generate service player

JSON-typescript-mapper

I used to use the json-typescript-mapper as displayed below, as it helps to convert a incoming json response into a real live TypeScript class. It works by adding approproate @JsonProperty annotations.

I have since removed this, as I find that the new Angular has the possibility of getting a "Typed" JSON response.

Although, since at the bottom is it still JavaScript, it means I cannot have any business logic in my "JSON" TypeScript classes, as per the usual the casting of type A to type B is simply ducktyping in action.

I do run the risk of creating an anemic entity model, though.

$ npm install --save json-typescript-mapper

The Differences

This is the important part.

Angular CLI

It seems I originally built the app using CLI version 1.0.6. Now I am using 7.2.1.

Generating code

It is now possible to generate guard, interface and enum with the CLI.

angular.json

The file angular.json is new, and replacing the old .xxxx

test files moved to src directory

Test files are now in /e2e/src/app.e2e-spec.ts

creating docs

Need to remember that the old file had a way to generate docs.In scripts of the package.json

"docs": "npm run typedoc -- --options typedoc.json --exclude '**/*.spec.ts' ./src/",
"typedoc": "typedoc"

Tslint and TypeScript syntax

• TsLint says: single quotes are better than double quotes.
• always use the === instead of ==, unless you're sure.
• most of the stuff I use locally should be const instead of let
• tslint actually complains sometimes if I am specific. For example with error: boolean = false;

Http

The HttpModule has been replaced/renamed to HttpClientModule and is moved from @angular/http to @angular/common/http.

Injectable annotation

has a new "providedIn" property.

REST service calls

The calling of a rest service has been simplified, from:

public updateGuild(guild: Guild): Observable<any> {
  let headers = new Headers({
    'Content-Type': 'application/json',
    'Accept': 'application/json'
  });
  let options = new RequestOptions({ headers: headers });
  return this.http.put(this.getGuildUrl(), guild, options)
    .catch((n) => this.handleError(n));
}

To:

public updateGuild(guild: Guild): Observable<any> {
  return this.http.put(this.getGuildUrl(), guild);
}

Luckily the actual code didn't need much changing.

Build and Release

Building for production

$ ng build --prod --base-href "/karchanpersonal/"

Miscellaneous

Dates

For some reason, the dates returned by the Rest services are in a format that is not recognised by JavaScript.

2010-03-03T05:54:09Z[UTC]

The [UTC] at the end is weird.

An ugly hack that removed the [UTC] with string.replace works.

References

[1] Angular

https://angular.io/

[2] Adding Bootstrap to Angular

https://www.talkingdotnet.com/add-bootstrap-4-to-angular-6-application/

[3] NgBootstrap - Getting started

https://ng-bootstrap.github.io/#/getting-started

[4] NPM - Material Icons

https://www.npmjs.com/package/material-icons

Git Branches

Just a little something for me to remember how to create branches and how to switch over to them.

There are plenty of resources available explaining how to do this, so I just thought I'd write some references down here.

$ git branch testing

creates a new branch called testing. Notice that this does not automatically make your new branch the current HEAD.

$ git log --oneline --decorate

seeing the branches in your log (as well as tags and the origins. Nice.)

54718edb (HEAD -> master, tag: v2.0.0, origin/master, origin/HEAD, v2.0.1) /* notes here. */
4c8c3bb9 /* moved the entire karchanangular into karchanpersonal. It compiles into the webapp directory. */
bb4bb432 /* upgrade from angular 4 to angular 7 */
12b528ed /* removed old directories that are no longer in use. Clean up. */
ead93fc3 /* changed images table, now has an autogenerated ID, plus an index on the pair owner and url. */

$ git checkout testing

switching to a branch

$ git checkout -b testing

shorthand for both creating a new branch called testing and checking it out immediately.

$ git checkout master ; git merge testing

merges the changes in the testing branch over into the master.

$ git checkout testing ; git merge master

merge the changes from your master into your testing branch.

$ git branch -d testing

delete a branch

$ git status

also shows you what branch you are on.

$ git show-branch

shows you all existing branches, and the commit that goes with that point in which it was branched.

References

GitBook - Git Branching

https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell

Git - Branch Manpage

https://git-scm.com/docs/git-branch

TutorialsPoint - Git Managing Branches

https://www.tutorialspoint.com/git/git_managing_branches.htm

Enabling SSL in Payara with certificate from Let's Encrypt.

In most cases the steps here are exactly the same as for the Glassfish application server¹.

So, it might be a bit superfluous, but in the interest of being complete.

Bear in mind that when you change the master password, this password is also used to re-encrypt all your keystores!

Letsencrypt in cacerts.jks

I'm still using the Keystore Explorer for this. When you open up the cacerts.jks file, you'll see Let's Encrypt is already in there.

It's called 'letsencryptisrgx1 [jdk]'.

Add private key to keystore.jks

1. "Open an existing keystore" in Keystore Explorer.
2. Choose keystore.jks. Put in your master password to access the keystore.
3. Deleted the old "s1as" Key Pair.
4. Import Key Pair
5. Select format PKCS#8
6. Encrypted private key checkbox should be unchecked
7. PKCS#8 Private key file should be privkey.pem
8. certificates file could be fullchain.pem
9. enter the alias "s1as".
10. enter a password to seal the deal.
11. close and save
12. restart Payara domain
13. done.

Interesting.

When attempting to restart the Payara domain, I used asadmin, and it immediately noticed that my keystore.jks file was changed. It showed me the certificate and asked:

Do you trust the above certificate [y|N]-->
Enter admin user name>
Enter admin password>

Restart your listener

Instead of having to restart your entire Payara Domain, apparently it is also possible to just turn your http-listeners on and off.

That's what this little bit here is all about.

The command "asadmin list-http-listeners", shows you the listeners.

$ ./asadmin list-http-listeners
Enter admin user name> admin
Enter admin password for user "admin">
http-listener-1
http-listener-2
admin-listener
Command list-http-listeners executed successfully.

In order to examine the http listeners fully, it is best to list the different properties, like so:

$ ./asadmin list server.network-config.protocols.protocol.*
Enter admin user name> admin
Enter admin password for user "admin">
server.network-config.protocols.protocol.admin-http-redirect
server.network-config.protocols.protocol.admin-http-redirect.http-redirect
server.network-config.protocols.protocol.admin-listener
server.network-config.protocols.protocol.admin-listener.http
server.network-config.protocols.protocol.admin-listener.http.file-cache
server.network-config.protocols.protocol.http-listener-1
server.network-config.protocols.protocol.http-listener-1.http
server.network-config.protocols.protocol.http-listener-1.http.file-cache
server.network-config.protocols.protocol.http-listener-1.ssl
server.network-config.protocols.protocol.http-listener-2
server.network-config.protocols.protocol.http-listener-2.http
server.network-config.protocols.protocol.http-listener-2.http.file-cache
server.network-config.protocols.protocol.http-listener-2.ssl
server.network-config.protocols.protocol.pu-protocol
server.network-config.protocols.protocol.pu-protocol.port-unification
server.network-config.protocols.protocol.pu-protocol.port-unification.protocol-finder.admin-http-redirect
server.network-config.protocols.protocol.pu-protocol.port-unification.protocol-finder.http-finder
server.network-config.protocols.protocol.sec-admin-listener
server.network-config.protocols.protocol.sec-admin-listener.http
server.network-config.protocols.protocol.sec-admin-listener.http.file-cache
server.network-config.protocols.protocol.sec-admin-listener.ssl
Command list executed successfully.

We can see which ones have ssl enabled, by using get:

$ ./asadmin get server.network-config.protocols.protocol.http-listener-2.security-enabled
server.network-config.protocols.protocol.http-listener-2.security-enabled=true
Command get executed successfully.
$ ./asadmin get server.network-config.protocols.protocol.http-listener-1.security-enabled
server.network-config.protocols.protocol.http-listener-1.security-enabled=false
Command get executed successfully.
$ ./asadmin get server.network-config.protocols.protocol.admin-listener.security-enabled
server.network-config.protocols.protocol.admin-listener.security-enabled=false
Command get executed successfully.
$ ./asadmin get server.network-config.protocols.protocol.sec-admin-listener.security-enabled
server.network-config.protocols.protocol.sec-admin-listener.security-enabled=true
Command get executed successfully.

As you've noticed above, the admin-listener has defined a redirect to sec-admin-listener.

Let's turn http-listener-2 off and on again. See if he picks up the new certificate.

./asadmin set server.network-config.network-listeners.network-listener.http-listener-2.enabled=false
Command get executed successfully.
./asadmin set server.network-config.network-listeners.network-listener.http-listener-2.enabled=true
Command get executed successfully.

Yes! We have a green lockicon in my Firefox bar! Success!

For some reason this little thing, also caused the new certificate to be picked up by the admin listener of the administration console.

Automation

Apparently it's possible to automate the whole thing, and there's a Python script in the Payara application server called payara5/bin/letsencrypt.py that let's you do that.

There's more information available on the website of Certbot on how to automate it, see in the references below.

References

[1] Enabling SSL in Glassfish with certificate from Let's Encrypt.

https://randomthoughtsonjavaprogramming.blogspot.com/2019/01/enabling-ssl-in-glassfish-with.html

Payara Blog - Securing Payara Server with Custom SSL Certificate

https://blog.payara.fish/securing-payara-server-with-custom-ssl-certificate

Payara Blog - Configuring SSL/TLS Certificates with Payara Server and Let's Encrypt

https://blog.payara.fish/configuring-ssl/tls-certificates-with-payara-server-and-lets-encrypt

Github.com - PAYARA-1061 LetsEncrypt integration script

https://github.com/payara/Payara/pull/2727

Certbot - User Guide

https://certbot.eff.org/docs/using.html

Keystore Explorer

https://keystore-explorer.org/

Enabling SSL in Glassfish with certificate from Let's Encrypt.

I wished to use the certificates of Let's Encrypt¹ for my website/glassfish.

I installed certbot using the manual found on the certbot website².

Make sure you are not running a website at the time, because the challenge of Let's Encrypt to verify you own the domain, is done by running a small webserver.

Getting certificates

[root@localhost certificates]# certbot certonly
Saving debug log to /var/log/letsencrypt/letsencrypt.log

How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Spin up a temporary webserver (standalone)
2: Place files in webroot directory (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 1
Plugins selected: Authenticator standalone, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Please enter in your domain name(s) (comma and/or space separated)  (Enter 'c'
to cancel): www.mrbear.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for www.mrbear.org
Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/www.mrbear.org/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/www.mrbear.org/privkey.pem
   Your cert will expire on 2019-01-12. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

Converting PEM

So now I have the necessary PEM files. Now to add them to my keystore and cacerts.

[root@mrbear config]# keytool -import -v -trustcacerts -alias letsencrypt -file /etc/letsencrypt/live/www.mrbear.org/fullchain.pem -keystore cacerts.jks -storepass itsasecret
... lots of text...
#9: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: B6 90 77 77 F6 3B DF 0C   C3 29 25 B5 56 29 EB CF  ..ww.;...)%.V)..
0010: 5D FD 3B 07                                        ].;.
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore
[Storing cacerts.jks]

I received a warning.

Warning: The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore keystore.jks -destkeystore keystore.jks -deststoretype pkcs12".

I did just that.

[root@mrbear config]# keytool -importkeystore -srckeystore keystore.jks -destkeystore keystore.jks -deststoretype pkcs12                                                                                           
Enter source keystore password:  
Entry for alias godaddy successfully imported.
Entry for alias godaddy2 successfully imported.
Entry for alias glassfish-instance successfully imported.
Entry for alias s1as successfully imported.
Import command completed:  4 entries successfully imported, 0 entries failed or cancelled

Warning:
Migrated "keystore.jks" to Non JKS/JCEKS. The JKS keystore is backed up as "keystore.jks.old".
[root@mrbear config]#

I got a java.security.cert.CertificateParsingException: signed fields invalid when trying to import the Let's Encrypt keys.

I didn't know how to resolve it, so I decided to go with KSE - KeyStore Explorer³.

Verify key

openssl rsa -in /etc/letsencrypt/live/www.mrbear.org/privkey.pem -check

Glassfish Admin console and HTTPS

I had some issues with the admin console which is also behind https.

There's a stackoverflow⁴ that helped me.

Running renew

In order to renew my keys with Let's Encrypt, all I need to do is run "cert renew" apparently.

I get the message that he cannot validate my domain. Apparently he needs to spin up a webserver again.

[root@mrbear ~]# certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/www.mrbear.org.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for www.mrbear.org
Waiting for verification...
Cleaning up challenges
Resetting dropped connection: acme-v02.api.letsencrypt.org

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/www.mrbear.org/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/www.mrbear.org/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Still using KSE, which is awesome, to change keystore.jks.

Steps taken:

1. Deleted the old "s1as" Key Pair
2. Import Key Pair
3. Select format PKCS#8
4. Encrypted private key checkbox should be unchecked
5. PKCS#8 Private key file should be privkey.pem
6. certificates file could be chain.pem
7. enter the alias "s1as".
8. enter a password to seal the deal.
9. close and save
10. restart Glassfish
11. done.

References

[1] Let's Encrypt

https://letsencrypt.org/

[2] Certbot

https://certbot.eff.org/

[3] SourceForge - Keystore-Explorer

http://keystore-explorer.sourceforge.net/

[4] StackOverflow - Glassfish V4 ssl admin no longer works

https://stackoverflow.com/questions/34935725/glassfish-v4-ssl-admin-no-longer-works/34952975

Reminders of what vanilla was like

A colleague found a... lenghty... article¹ about what World of Warcraft - Vanilla was.

Ye Be Warned.

[1] MMO-Champion - Reminders of what vanilla was like

https://www.mmo-champion.com/threads/2330240-Reminders-of-what-vanilla-was-like