Google OnBoard - Google Cloud

“Almost all the successful microservice stories have started with a monolith that got too big and was broken up.”

- Martin Fowler¹

On the 20th of September 2017, I enjoyed a full day at Eindhoven HighTech Campus listening to Google explain everything about their Google Cloud Platform (GCP)². It's the 63rd or so time this workshop was given. (20.000 people 36 cities)

Of course, I needed to write some of the stuff down, so here's the stuff.

I liked the April fools joke shorts that were interspersed in the lecture.

The problem everyone seems to be facing is that everything needs to be done faster, and better and there needs to be more focus on building business value, instead of having to spent time administrating the hardware.

This can be seen in the fact that the most expensive item in your organisation are your engineers, and they need to be able to work as efficient as possible. Google noticed in the past that time spent doing things that ultimately were of little value can not be regained and is forever lost.

When it comes to the evolution of clouds, you see the following waves happen:

1st wave - colocation

your kit, someone else's building, yours to manage.

2nd wave - virtualized data centers

standard virtual kit, for rent, still yours to manage.

3rd wave - intelligent services automated everything

invest your energy in great apps

Funnily enough, I am noticing this same trend in my home hobbyprojects as well. It can be summarised as follows:

1st wave

I had my own computer, but to install it in a data centre, it needed a rack-mount. My hardware in somebody elses data center.

2nd wave

My computer now is a VPS that I rent.

3rd wave

I started moving small apps over to the Google App Engine.

For a lot of software companies (mine included) we are still only on the second wave. The products of lots of software companies do not require some of the advantages of the cloud and it therefore makes no sense to make the effort.

You can just assemble the google "hexagons" that your application is going to need. It takes a little work to find out what Google has and how it can be best applied to your situation. Because, everybody's situation is different.

Why choose google? If you do use google, you use the netwerk Google created (which is fast), to access other Google users (which use the same network), which are therefore only a hub away.

Googles network is divided in regions and zones. A region might for example be Europe, and a zone might be a data center in a specific region, for example West-1a.

The Google hexagons can be roughly categorized into four main groups, as displayed in the following most important sheet:

StackDriver

StackDriver is a company which was bought up by Google and it specializes in "supporting glue" for different components in an application:

• Logging
• Monitoring
• Debug
• Error Reporting
• Trace

Philips Hue

As an example of what it can do, Philips was there to explain how they have been using the cloud along with their new Smart-bulbs in their home-automation project called Hue.

It is a long day to listen, luckily with the free trial version³ of the Google Cloud Platform, it was possible to follow the two gentlemen along in their path through the different components of the platform.

There were some lighter notes as well. For example the spot-the-security-guard in pictures of the Google Data Center. The Machine Learning demos were quite fun too.

I can highly recommend the workshop, for anyone who wishes to have an idea about what Google can do to help your application become great. The sheets alone give a good impression.

References

[1] Martin Fowler - MonolithFirst

https://martinfowler.com/bliki/MonolithFirst.html

[2] Cloud OnBoard - Learn how to Build What's Next with Google

https://cloudplatformonline.com/2017-onboard-benelux.html

[3] Google - Free Trial

https://cloud.google.com/free/

Google Developer Group Netherlands

gdgnetherlands.org

Google QwikLabs

google.qwiklabs.com

Measure your latency to GCP regions

http://www.gcping.com/

Coursera - Google Cloud Platform

https://www.coursera.org/googlecloud

Google Certified

https://cloud.google.com/certification/

Workshop Continuous Delivery with Docker

Last Monday, on the 14th of November, I got a chance to attend a little workshop on Docker organised by OV Software. It was at the NH Hotel near the facilities of OV Software in The Hague. It is a little far from my place of residence, but I could tag along with a colleague of mine, which helped.

They did a great job of preparing the workshop properly. A VirtualBox image was provided (https://goo.gl/XUM898). The downloaded file can be "imported" (File -> Import Appliance) into the VirtualBox platform (version minimum is 5.0).

Unfortunately, my labtop is not one of the greatest, and the amount of memory required is too much for it. I had to borrow a labtop from work. If this keeps up, I'll have to buy something better for myself one of these days.

Also, unfortunately, I was unable to attend a previous workshop of which this workshop is a followup, so I did miss some background on Docker, and this was my first foray into this new area. Thankfully, a lot can be found on the internet^{1 2}, and I already had plenty of experience with Linux, Java, Maven, and just software development in general.

I had small problems getting it to run, for example:

• I had Vx extentions turned off in the BIOS of my labtop. I needed to turn them on.
• PAE - Physical Address Extentions was turned off. It should be on.
• Somehow, when importing the image, I had turned the machine to 32bits. It's a Linux 64bits image.
• It seems it sometimes helps to turn off your USB in the Virtual Machine settings.

The workshop was provided by an employee of OV Software, Jeroen Peeters, who is an excellent and calm speaker.

One of the first things where I went wrong is thinking that Docker is a kind of Virtualisation. It is not. Docker runs within the same Operating System and the containers also all run in the same Operating System. The images, however, can contain everything the container needs to properly run the image. So, they are simply processes running under the same kernel with the same kernel libraries at their disposal, but it is true that these processes are a little better isolated from the rest of the system.

There are plenty of alternatives to Docker, but Docker seems to be the main menu these days:

• LXC
• Rocket, tooling for developers and systemmanagement
• Lmctfy
• OpenVZ
• Canonical LXD, a Ubuntu thing but more isoltation, so more Virtualisation as it also runs system images

To be more specific, the workshop was regarding Continuous Delivery^{3 4} with Docker.

During the workshop we set up the following Docker containers inside Docker running on Linux Lite 3.0 LTS (Ubuntu based):

• a container of GitLab that held a git repository of a Java Program
• a container of Jenkins
• a maven container (for inside Jenkins)
• a tomcat container
• a load balancer container (Haproxy)
• a testx container⁶

So a webhook in the GitLab repo made sure that jenkins was notified if the source code had changed. Jenkins would therefore start a build pipeline, that would build the application and deploy it on Tomcat and would start testing it using TestX⁵ (a frontend tester tool sauce over protractor and Selenium).

The github repository containing the Java application can be found at https://github.com/jeroenpeeters/docker-workshop-java-todolist/. It is a simple TODO list registration tool.

All this is running in one virtual server. For clustering and the like, you could have a look at Kubernetes, the google solution for running Docker on several machines. Complexity is quite high. But there are also other solutions.

It looks very interesting if you are interested in using Continuous Delivery. I do know that System Management (where I work) is very apprehensive about it. I think they fear to lose control on what Developers can do. We are already working with it in some fashion, but it needs to be improved.

References

[1] Docker

https://www.docker.com/

[2] Docker - Installation on Fedora

https://docs.docker.com/engine/installation/linux/fedora/

[3] Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation

https://www.amazon.com/dp/0321601912?tag=contindelive-20

[4] MartinFowler.com - ContinuousDelivery

http://martinfowler.com/bliki/ContinuousDelivery.html

[5] TestX

https://github.com/testxio/testx

[6] TestX Docker Image

https://hub.docker.com/r/testx/protractor/

Nginx - Docker for Java Developers

https://www.nginx.com/resources/library/docker-for-java-developers/

The Docker Book - Containerization is the new virtualization

https://www.dockerbook.com/

Jenkins

http://www.jenkins.io