Friday 25 November 2016

Obscure little problems

Just a small snippit for now, as I am pressed for time.

It took a little time to find this extremely trivial problem. I overlooked it several times whilst reading through the code.

public void setHarbourCode(HarbourCodeTO harbourCodeTO) {
  this.harbourCode = harbourCode;
}
TO stands for Traffic Object, as this small function is part of a Web interface.

Thursday 17 November 2016

Workshop Continuous Delivery with Docker

Last Monday, on the 14th of November, I got a chance to attend a little workshop on Docker organised by OV Software. It was at the NH Hotel near the facilities of OV Software in The Hague. It is a little far from my place of residence, but I could tag along with a colleague of mine, which helped.

They did a great job of preparing the workshop properly. A VirtualBox image was provided (https://goo.gl/XUM898). The downloaded file can be "imported" (File -> Import Appliance) into the VirtualBox platform (version minimum is 5.0).

Unfortunately, my labtop is not one of the greatest, and the amount of memory required is too much for it. I had to borrow a labtop from work. If this keeps up, I'll have to buy something better for myself one of these days.

Also, unfortunately, I was unable to attend a previous workshop of which this workshop is a followup, so I did miss some background on Docker, and this was my first foray into this new area. Thankfully, a lot can be found on the internet1 2, and I already had plenty of experience with Linux, Java, Maven, and just software development in general.

I had small problems getting it to run, for example:
  • I had Vx extentions turned off in the BIOS of my labtop. I needed to turn them on.
  • PAE - Physical Address Extentions was turned off. It should be on.
  • Somehow, when importing the image, I had turned the machine to 32bits. It's a Linux 64bits image.
  • It seems it sometimes helps to turn off your USB in the Virtual Machine settings.

The workshop was provided by an employee of OV Software, Jeroen Peeters, who is an excellent and calm speaker.

One of the first things where I went wrong is thinking that Docker is a kind of Virtualisation. It is not. Docker runs within the same Operating System and the containers also all run in the same Operating System. The images, however, can contain everything the container needs to properly run the image. So, they are simply processes running under the same kernel with the same kernel libraries at their disposal, but it is true that these processes are a little better isolated from the rest of the system.

There are plenty of alternatives to Docker, but Docker seems to be the main menu these days:
  • LXC
  • Rocket, tooling for developers and systemmanagement
  • Lmctfy
  • OpenVZ
  • Canonical LXD, a Ubuntu thing but more isoltation, so more Virtualisation as it also runs system images

To be more specific, the workshop was regarding Continuous Delivery3 4 with Docker.

During the workshop we set up the following Docker containers inside Docker running on Linux Lite 3.0 LTS (Ubuntu based):
  • a container of GitLab that held a git repository of a Java Program
  • a container of Jenkins
  • a maven container (for inside Jenkins)
  • a tomcat container
  • a load balancer container (Haproxy)
  • a testx container6

So a webhook in the GitLab repo made sure that jenkins was notified if the source code had changed. Jenkins would therefore start a build pipeline, that would build the application and deploy it on Tomcat and would start testing it using TestX5 (a frontend tester tool sauce over protractor and Selenium).

The github repository containing the Java application can be found at https://github.com/jeroenpeeters/docker-workshop-java-todolist/. It is a simple TODO list registration tool.

All this is running in one virtual server. For clustering and the like, you could have a look at Kubernetes, the google solution for running Docker on several machines. Complexity is quite high. But there are also other solutions.

It looks very interesting if you are interested in using Continuous Delivery. I do know that System Management (where I work) is very apprehensive about it. I think they fear to lose control on what Developers can do. We are already working with it in some fashion, but it needs to be improved.

References

[1] Docker
https://www.docker.com/
[2] Docker - Installation on Fedora
https://docs.docker.com/engine/installation/linux/fedora/
[3] Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation
https://www.amazon.com/dp/0321601912?tag=contindelive-20
[4] MartinFowler.com - ContinuousDelivery
http://martinfowler.com/bliki/ContinuousDelivery.html
[5] TestX
https://github.com/testxio/testx
[6] TestX Docker Image
https://hub.docker.com/r/testx/protractor/
Nginx - Docker for Java Developers
https://www.nginx.com/resources/library/docker-for-java-developers/
The Docker Book - Containerization is the new virtualization
https://www.dockerbook.com/
Jenkins
http://www.jenkins.io

Thursday 10 November 2016

Simpson's paradox

“There are three kinds of lies: lies, damned lies, and statistics.”
During my travels across the Internet, I recently stumbled upon a paradox that exemplifies the quote1 above.

It is called the Simpson's Paradox2 and I had never heard of it before.

From what I can gather, it means that data that is aggregated might point in the exact opposite direction compared to the same data partitioned by a certain type/relation.

It depends on the situation which of the two should be preferred, in order to extract proper conclusions from the data.

The Wikipedia entry in the references has some very good examples of Simpson's Paradox and should be required reading for any budding statistician.

References

Wikipedia - Lies, Damned lies and statistics
https://en.wikipedia.org/wiki/Lies,_damned_lies,_and_statistics
wikipedia - Simpson's Paradox
https://en.wikipedia.org/wiki/Simpson%27s_paradox

Thursday 3 November 2016

JFall 2016 - Writeup

“Knowledge is the only treasure that increases when you share.”
- African saying.

This blog post is my writeup and evaluation of the sessions I attended at J-Fall 20161.
CQRS and Event Sourcing with Lagom
Unfortunately I was rather late in attending this seminar. It caused me some problems with understanding the matter. I gather it is about event sourcing and CQRS with Lagom.
Hands-on Lab: 12 things you can do better with Java8 - Jakub Marchwicki
Wow, there was a severe shortage of places to sit (properly) and headphones. He provided a github repo2. I decided to leave early because of these issues. I hope to get some info from the github repo.
Sharing 2 years experience using Scala in a real project by a Java team

It did mean I could attend this session for a bit. Unfortunately, my knowledge of Scala is limited, but it was possible to follow the talk, except for the details.

Though there are some rumours that the takeoff of Scala isn't as big as it once was, the real project proved that with excellent programmers you can build a better application in Scala than in Java.

It does take some serious getting used to, though.

Java9 and the impact on Maven projects

A bit dull. Seems there are only a few little problems with using Java 9. Big problems were fixed right quickly.

Note: new jar format, special classes for different JDKs in different directories. src/main/java8 src/main/java9 etc. This is called Forward compatibility.

Modularity dependencies are hard.

Java - The Ecosystem Awakens
Sharat Shander mentioned that Java is now 21 years old already. The three top (new!) speakers of Jfall 2016 are invited to JavaOne.
Rise of the Machines - Automate your development
Sven Peters, Atlassian Evangelist, had a very entertaining seminar about Bots in Software development in general, and how they can make our lives easier. It all started with the advent of Hudson, the automated building environment, if I may call it that.
Coding Bots
JIRA should follow our versioning systems "Don't branch from a red branch". "Invite right reviewers" "Two approvals minimum before merge is possible"
Test Bots
Developers don't write bad code on purpose -> Developers are trying to solve problems. Calling Dr. Code -> Hall of Shame/Hall of Fame Hallelujah - Functional Test RunManager Flaky Test Detector -> test in Quarantaine -> create issue
Ops Bots
No more throwing stuff over the wall pagerduty : Faster feedback loop Own your code -> from end to end
Service Bots
Hercules -> scans (log) files and has system knowledge. Confluence has machinereadable bit for hercules. Hercules suggests that page. Hercules learns from response of user/firstlinesupport.

Robots can only do so much. They are very good at automating simple tasks.

Creating customer value is an art, however, and is hard to automate. Therefore humans will always be required.

Treat your bots well.


Delivering Better and Faster Microservices and Mobile Apps with the Cloud

Bruno Borges was very engaging in telling the tale. A shame that the Technology in the room didn't actually help, especially the wireless.

Software helps companies to Innovate.

Oracle has a presence on github5.



Documentation avoidance for developers - Peter Hilton

Peter Hilton3 was tremendously entertaining in explaining how we can write as little documentation as possible, and get away with it.

The half life of a team is 3.1 years. After that period, half your team has moved on to something else, and the knowledge they have is gone. This in stark contrast to code, which has a half life of about 15 years.

What you need is constructive laziness, which means thinking hard on how to do as little as possible.

Excuses may help, things like "It's on the Wiki". Tell a person who is leaving the team to write everything down before he's gone doesn't work. But asking questions and posting the answers into a wiki will help.

Pair programming works. A person who writes horrible/complex code will write horrible/complex documentation.

Better naming and type safety helps.

Spotify and Github have a very well documented API.

A good term is "minimum viable documentation".


Java EE Next

David Delabassee mentioned the release deadlines for Java EE 8 (2017) and Java EE 9 (2018).

They seem to be more geared towards microservices and asynchronous behaviour.

JAX-RS 2.1
server sent events
Servlet 4.0
HTTP/2 drop-in replacement server push
CDI 2.0
more geared towards outside an application server
JDF 2.3
small changes
JSON-B 1.0
JAXB for JSON.

References

[1] J-Fall 2016
http://www.nljug.org/jfall/2016/
[2] GitHub.com - JFall Java 8 Lab
https://github.com/kubamarchwicki/jfall-java8-lab
[3] Peter Hilton
http://hilton.org.uk
[4] Tom Preston-Werner - Readme driven development
http://tom.preston-werner.com/2010/08/23/readme-driven-development.html
[5] GitHub - Oracle
https://github.com/oracle