Thursday, 26 November 2015

Code Kata at the Dojo

At my work, the software architect asked if people were interested in getting together at work of an evening and doing a Kata¹.

“Kata² (型 or 形 literally: "form"), a Japanese word, are the detailed choreographed patterns of movements practised either solo or in pairs. The term form is used for the corresponding concept in non-Japanese martial arts in general.”

Well, I'd never done one of those, so I signed up enthusiastically.

We spent a couple of hours in pairs, (xtreme programming, yay!) to do the Kata. In this case it was regarding Bowling Scores^{4 5} provided by Uncle Bob⁸.

It is very interesting, and I decided to spend a little more time in my limited spare time to see what I could come up with. I do notice almost immediately that I always try and capture the domain (in this case, scores, rolls, frames, spares and strikes) into Objects/Classes. It's a challenge to not immediately grab to my standard toolbox. It's also great to get some tests done, before diving into coding.

It is also a challenge to properly read requirements.

I retrieved the git repo from [6] and forked it over in my own github [7] to work on. There's nothing there yet, but that will change once I have something that looks half decent.

References

[1] Wikipedia - Kata (programming): https://en.wikipedia.org/wiki/Kata_%28programming%29
[2] Wikipedia - Kata: https://en.wikipedia.org/wiki/Kata
[3] Wikipedia - Procedural memory: https://en.wikipedia.org/wiki/Procedural_memory
[4] ArticleS. UncleBob. TheBowlingGameKata: http://butunclebob.com/ArticleS.UncleBob.TheBowlingGameKata
[5] CodingDojo - KataBowling: http://codingdojo.org/cgi-bin/index.pl?KataBowling
[6] https://github.com/jphhoeks/dojo: https://github.com/jphhoeks/dojo
[7] https://github.com/maartenl/dojo: https://github.com/maartenl/dojo
[8] Wikipedia - Robert Cecil Martin: https://en.wikipedia.org/wiki/Robert_Cecil_Martin

Thursday, 19 November 2015

Linux Commandline Password Manager

Recently I was looking for a good Password Manager, as the number of passwords I need to maintain is growing steadily (I'm sure everyone knows how it is).

The one I found ("pass² - the standard unix password manager"), the one I am talking about, exemplifies the Unix philosophy⁰. Which is:

Do One Thing and Do It Well.

Write programs to work together.

Write programs to handle text streams, because that is a universal interface.

The password manager actually basically doesn't do anything itself, but delegates to other parts of the Linux system, namely the filesystem, and GPG for encryption, pwgen for the generation of random passwords, xclip for interfacing with the clipboard and (optionally) Git for version control.

It does mean that you have to setup Gnu Private Guard properly, before you can start using "pass", hence the next chapter.

Gnu Private Guard

“GnuPG¹ is a complete and free implementation of the OpenPGP standard as defined by RFC4880 (also known as PGP).”

Creating a key

All keys are stored in .gnupg.

bash-4.3$ gpg2 --full-gen-key
gpg (GnuPG) 2.1.7; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: directory '/home/mrbear/.gnupg' created
gpg: new configuration file '/home/mrbear/.gnupg/gpg.conf' created
gpg: WARNING: options in '/home/mrbear/.gnupg/gpg.conf' are not yet active during this run
gpg: keybox '/home/mrbear/.gnupg/pubring.kbx' created
Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection? 2

I have decided to follow the advice of the website¹ and take 2.

DSA keys may be between 1024 and 3072 bits long.
What keysize do you want? (2048)

I take 2048, it seems fairly default.

Requested keysize is 2048 bits
Please specify how long the key should be valid.
0 = key does not expire
= key expires in n days
w = key expires in n weeks
m = key expires in n months
y = key expires in n years
Key is valid for? (0)
Key does not expire at all
Is this correct? (y/N) Y

GnuPG needs to construct a user ID to identify your key.

Real name: mrbear
Email address: mrbear@localhost.com
Comment: mrbear
You selected this USER-ID:
"mrbear (mrbear) "

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit?

And away we go!

We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: WARNING: some OpenPGP programs can't handle a DSA key with this digest size
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: /home/mrbear/.gnupg/trustdb.gpg: trustdb created
gpg: key 3A4CFDFE marked as ultimately trusted
gpg: directory '/home/mrbear/.gnupg/openpgp-revocs.d' created
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
pub dsa2048/3A4CFDFE 2015-10-14
Key fingerprint = 7FCC 07FA A6BE 32DE A971 2281 587B D827 3A4C FDFE
uid [ultimate] mrbear (mrbear)
sub elg2048/0B94C819 2015-10-14

So, just to check:

bash-4.3$ gpg2 --list-keys
/home/mrbear/.gnupg/pubring.kbx
---------------------------------
pub dsa2048/3A4CFDFE 2015-10-14
uid [ultimate] mrbear (mrbear)
sub elg2048/0B94C819 2015-10-14

Password store

Apparently, initialising the store needs to be done with the name of the key generated above.

bash-4.3$ sudo dnf install pass
...
bash-4.3$ pass init "3A4CFDFE"
mkdir: created directory ‘/home/mrbear/.password-store/’
Password store initialized for 3A4CFDFE

And I wish to use Git, because I'm that kinda guy.

bash-4.3$ pass git init
Initialized empty Git repository in /home/mrbear/.password-store/.git/
[master (root-commit) 77cc085] Add current contents of password store.
1 file changed, 1 insertion(+)
create mode 100644 .gpg-id
[master 6655058] Configure git repository for gpg file diff.
1 file changed, 1 insertion(+)
create mode 100644 .gitattributes

Adding my passwords

For example, my email:

bash-4.3$ pass insert Email/mail.localhost.com
mkdir: created directory ‘/home/mrbear/.password-store/Email’
Enter password for Email/mail.localhost.com:
Retype password for Email/mail.localhost.com:
[master 40d6094] Add given password for Email/mail.localhost.com to store.
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 Email/mail.localhost.com.gpg

Adding parameter --multiline makes it possible to not only add your password but other stuff as well. Handy if the userid/username is not standard my email address.

Showing passwords

bash-4.3$ pass
Password Store
└── Email
└── mail.localhost.com

Generating passwords

bash-4.3$ pass generate Internet/sync.firefox.com 16
mkdir: created directory ‘/home/mrbear/.password-store/Internet’
[master 9b966af] Add generated password for Internet/sync.firefox.com.
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 Internet/sync.firefox.com.gpg
The generated password for Internet/sync.firefox.com is:
\$%R&E%&8^BA/

Some websites do not accept "odd" characters (< > ! @ # $ % ^ & * _) when it comes to passwords. When adding the "[--no-symbols,-n]" option you get a generated password containing only letters, capitals and digits.

Showing passwords and copying to clipboard

bash-4.3$ pass -c Email/mail.localhost.com
Copied Email/mail.localhost.com to clipboard. Will clear in 45 seconds.

Getting your stuff back

As basically this awesome password manager stored the passwords as a bunch of GPG2 encrypted files in .password-store, we should be able to decrypt the files directly.

This would work as follows:

bash-4.3$ gpg2 --decrypt .password-store/Email/mail.localhost.com.gpg
gpg: encrypted with 2048-bit ELG key, ID 0B94C819, created 2015-10-14
"mrbear (mrbear) "
itsasecret!

Syncing the git

As I wish to manage my passwords from multiple computers, there's a need to sync/merge the git repositories⁵ from time to time.

First is, of course, the copying of the GPG keys by copying the .gnupg directory.

scp -r .gnupg mrbear@toby:/home/mrbear

From the second machine, from my homedirectory:

git clone ssh://mrbear@sherlock/home/mrbear/.password-store
Cloning into '.password-store'...

After that it is nothing more but a "git pull" or "git push" whenever I need it.

References

[0] Wikipedia - Unix philosophy: https://en.wikipedia.org/wiki/Unix_philosophy
[1] GnuPG: https://www.gnupg.org/documentation/howtos.html
[2] Password store: http://www.passwordstore.org/
[3] Linux Crypto: Passwords: http://blog.sanctum.geek.nz/linux-crypto-passwords/
[4] StackExchange Unix - I try to add passwords to the pass password managers but my attempts fail: http://unix.stackexchange.com/questions/53912/i-try-to-add-passwords-to-the-pass-password-manager-but-my-attempts-fail-with
[5] Git - Git Basics Working with Remotes: https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes
Fedora Magazine - GPG: a Fedora primer: https://fedoramagazine.org/gnupg-a-fedora-primer/
Fedora Magazine - GPG key management, part 1: https://fedoramagazine.org/gpg-key-management-part-1/

Thursday, 12 November 2015

GUID Partition Table

MBR (Master Boot Record) is an old relic that is still in much use as the standard Partition Table format. It is still the standard when it comes to USB sticks and things like that. GPT (GUID Partition Table) is much better and already exists for the past 5 years³.

Apparently Linux (Fedora² since version 8) supports GPT for a long time. For more information on GPT, see [1], which has some nice diagrams.

Unfortunately, every time I order a new harddrive for one of my computers, it comes equipped standard with the MBR Partition table, and it always takes me a while to figure out why I cannot allocate more than 2 terabytes or assign more than 4 partitions without having to jump through some hoops.

Oh, well.

References

[1] FedoraProject - MBR vs GPT which is better for Fedora: https://ask.fedoraproject.org/en/question/51863/mbr-vs-gpt-which-are-better-for-fedora/
[2] FedoraProject - Features - GUID Partition Table: https://fedoraproject.org/wiki/Features/GUID_Partition_Table
[3] Wikipedia - GUID Partition Table: https://en.wikipedia.org/wiki/GUID_Partition_Table
[4] Linux.com - Using the new guild partition table in linux: https://www.linux.com/learn/tutorials/730440-using-the-new-guid-partition-table-in-linux-good-bye-ancient-mbr-

Thursday, 5 November 2015

J-Fall 2015

“Once you stop learning, you start dying.”

- Albert Einstein

I just got back from J-Fall 2015¹.

Seeing as the location had been changed from Nijkerk to Ede, I was unable to attend the Early Bird sessions, much to my dismay. I did manage to hitch a ride with a colleague of mine, but in the future, I will try and get a train. The connection seems to be quite good and once more there is a shuttle bus riding between the location (Cinemec) and the trainstation.

What with traffic (congestion) and my colleague sleeping late, I did miss the first keynote (The Experimental Enterprise - Keynote ING), only arriving at the destination at around 09:45.

The list of sessions I witnessed:

Pushing the limits of Continuous Delivery - Keynote Quintor

Provided by Rene Boere and Pascal Snippen

A fascinating view on what Quintor uses to provide Continuous Delivery (CD) using Docker, Apache Mesos, Marathon and Consul combined with HAProxy. The use of container ships to emphasise Docker was inspired and probably is starting to be overused by now. The microservices landscape is large and you can no longer see the forest for the trees, test environments are hard to set up, deployments are complex and timeconsuming.

Automation of Continuous Delivery is the answer.

Microservices for Mortals

Provided by Bert Ertman

This lecture could be construed as a big, huge, warning if you wish to use Microservices. It had a metaphore comparing coding practices with Italian food. The old way of coding compares to Spaghetti. Then SOA (Service Oriented Architecture) was introduced and those layered coding practices compare to Lasagna. Now the new Microservices compare more to Ravioli (tight independent containers of code bundled together) with sauce. The main idea behind microservices seems to be the ability to easy adapt to change. There is no hard definition on what microservices are. Apparently, microservices are small enough to just replace them with a completely new implementation, or just run 10 instances of one easily. They do away with a lot of items we are software developers have gotten used to:

synchronous programming models
ACID
code reuse
using abstractions

Instead, because of the characteristics of Microservices, you get:

asynchronous programming models
any constraints need to be taken care of at the application level
code duplication, each micro service is independent of the others, so code reuse between microservices doesn't make sense
separate data storages (one data storage per microservice)
passing data using websockets or binary protocols or messaging (REST is too much overhead, I don't want to even mention XML. It's all fine for a public API though.)
prefer conventions over abstractions (see code reuse)

If you have not been doing Devops, you cannot get started using Microservices without getting into a world of hurt.

Code for failure. Microservices are brittle, the distributed asynchronous models make them so. Design for this! Use fail early, use resilience patterns, use redundancy, etc.

10 Awesome Tips for Enterprise Javascript - Oracle

Provided by Geertjan Wielenga

This lecture provided an excellent overview of the current JavaScript landscape. I especially liked the fact that he mentioned that at the end of the lecture the landscape will probably already have changed. The JavaScript ecosystem is in that much flux.

Nowadays we have multiple devices, all with different screensizes requiring responsive design. One common factor in all these devices, is that all of them possess a browser. And the Single Page Application (SPA) is becoming the internet-enabled "application" of choice. Page navigation is irrelevant when it comes to Single Page Applications, yet for most Java-based Web Frameworks, page navigation is still an important part of their programming model.

Also, resist the hype. Apparently most managers make bad architecture decision based on hype. And you, as a developer, shouldn't want that.

For some applications, Javascript actually doesn't make any sense. The example provided was air traffic controllers. But different architectures can play different roles, even in those applications where Javascript doesn't make any sense. for example:

mobile apps: notifications
web apps: reporting to (upper) management
application itself: what users actually need and use in day-to-day work

NetBeans provided really nice point-and-click programming of the new HTML 5 components. The new HTML 5 components look very snazzy. Apparently, every new HTML 5 component is itself composed of sub-html-components, called the Shadow DOM and can therefore be styled appropriately. CSS has been split up into approximately 50 CSS modules. JavaScript can be used to avoid having to hide HTML using CSS, because hiding certain HTML components still makes the browser get the linked resources contained in those HTML components. Especially on mobiles this can make a huge difference.

In the Java world, the acronym WORA (Write Once Run Anywhere) is very well known. In the JavaScript world its WONTA (Write Once, Never Touch Again). The projects are so small, and the evolution of the Javascript ecosystem is so quick, it makes more sense to re-design and re-write everything from scratch when required.

Oracle has defined a Javascript framework, which is basically a bundling of common javascript libraries, into what is called the "Oracle Javascript Extention Toolkit"².

“Javascript is the assembly language of the web.”

Then there's, if you really do not want to touch JavaScript with a ten-foot pole, the transpilers:

Dukescript: write Java, but also HTML and CSS. Combine those.
GWT/Vaadin: provides a Java backend similar to the Swing toolkit, and translates your code into HTML,CSS and JavaScript for the frontend.
Dart: separate programming language for the web/mobile/desktop.
Typescript: provides static typing to javascript, a sort of superset of JavaScript following ECMA recommendations that is transpiled to JavaScript
Coffeescript: programming language that translates to javascript

The Java renaissance continues - Keynote Oracle

Provided by Sharat Chander

A motivating speech on the fact that Java is still a success primarily because of the community. And a personal appeal to every person there to connect to at least two in the audience during the conference.

Devops - Are you walking or still talking? - Keynote Capgemini

Provided by Remko Reinders

A lecture on Devops, more importantly, what it takes for an organisation to become a success at Devops and what you can do to help your organisation and yourself to become a success as well.

“Most people will talk the talk. Few will walk the walk. Be amongst those few.”

- Steve Maraboli

“Software is eating up the world.”

- Marc Andreessen, Wall Street Journal

Building Asynchronous and Non-Blocking HTTP Applications with Ratpack

Provided by Hubert Klein Ikkink

My first introduction to Ratpack³, which I had never heard of before. It is a small webserver, that can be programmed using Java 8 and Groovy. It uses Netty for the HTTP IO, has a very small footprint, is very fast and is asynchronous. Does not provide JEE, does not follow the Servlet API and can be started by running a Jar file. Spock is used for testing.

If you want a quick webserver to dish out some REST stuff in a distributed environment, this is it. Check his webblog⁴ for more information.

Java modularity, life after Java 9 - Luminis

Provided by Sander Mak & Paul Bakker

“Good fences make good neighbours.”

- Proverb

An excellent talk about the upcoming Jigsaw in Java 9. It provides a faster startup and a smaller footprint, essential for small devices. It compared what little information there is about Jigsaw to the existing implementation of OSGI. Modules are allowed to talk to other modules only by the service contract each module provides. Modules are defined in a module-info.java file. The segregation is looked after by the VM itself and cannot be circumvented. Layers are used to prevent different versions of the same module interfering with each other. Versioning isn't really built into the new system. There is a linking tool called jlink and module maker tool called jmod. Jdebs can provide your program with which modules it needs. Identification of a module is done using a newly defined namespace specific for modules.

Classpath scanning for annotations is going to be a problem. It is going to have to be service-oriented in the future.

Especially application servers are going to need a major overhaul, as well as several annotation-crosscutting concerns frameworks as REST and JPA implementations.

See http://bit.ly/java9demo for more information.

References

[1] NLJUG - J-Fall 2015: http://www.nljug.org/jfall/2015/
[2] Oracle JavaScript Extention Toolkit: http://oraclejet.org
[3] Ratpack: https://ratpack.io/
[4] Mr Haki's Webblog: http://mrhaki.blogspot.nl/

Monday, 2 November 2015

J-Fall 2015 Incoming!

The NLJUG^[1] is once again organizing J-Fall^[2]. Edition 2015 obviously.

It takes place in a new location, as the old location apparently has grown too limiting in number of people allowed. It will be held in CineMec, 150 Laan der Verenigde Naties, Ede on Thursday, the 5th of November 2015. No longer on a Wednesday, that's a change!

Once again I am looking forward to it, to see what I can learn. I shall write up a report on what sessions I visited and found interesting.

References

[1] NLJUG - Nederlandse Java Users Group: http://www.nljug.org
[2] J-Fall 2015: http://www.nljug.org/jfall/2015/