Sunday 30 March 2014

Google App Engine - First try

Introduction

I own an Internet-enabled digital scale for weighing myself and measuring my impedance.

Architecture

Functional Requirements (FR):
  • I wish to store a backup of my scale measurements, for my own purposes
  • however, scale measurements must continue to be reported to the official website
I have several Non-Functional Requirements (NFR):
  1. needs to use Java (because that's what I do)
  2. the scale receiver uses a simple unencrypted proprietary protocol
  3. the program receiving data from the scale receiver can handle raw sockets (related to previous point)
  4. must scale easily
  5. must not cost any money (until a certain threshold of users is reached, obviously)
  6. must provide decent monitoring
  7. needs to have an extremely low amount of maintenance
  8. must be able to easily create/upgrade/extend the part that uses those measurements for stuff, without disrupting the core functionality of receiving measurements
  9. needs to have the Cool Tech Factor™
These dictate which kind of architecture to use:
  1. Google App Engine (NFR 1, 4, 5, 6, 7, 8 but fails 2 and 3, definitely allows for NFR 9)
  2. a simple socket server, receives measurements, sends them on to Google App Engine and the original website (NFR 2, 3, 5, 7, might fail 4 and 8, but perhaps not if we keep the amount of actual work done by the simple socket server to a minimum, definitely fails NFR 9)

Implementation

The "Daemon3" sends the results to the Google App Engine2, using a Java library1 to make a HTTP POST request.

Example of Use

In the upper-right corner of my blog, my current weight is visible. It is a first example of what is possible.

I am planning on providing, on my blog, a complete graph of my weight over time. Stay tuned.

For the "Daemon" in the picture, that is the socketproxy server I made first. See [3].

References

[1] Apache HTTP Components
https://hc.apache.org/
[2] Google App Engine
https://developers.google.com/appengine
[3] SocketProxy Server
http://randomthoughtsonjavaprogramming.blogspot.nl/2014/03/socketproxyserver.html

Tuesday 25 March 2014

Java 8 - Launch!

I have just witnessed the Live Webcast1 of the Oracle Java 8 Launch. It seems that Java 8 is officially2 launched.

I hear Java 8 was already released in the wild on the 18th of March, but this seems to be the Live Webcast of the Launch. Introducing Java 8 was Mark Reinhold3, the Chief Architect of the Java Platform Group at Oracle.

He was joined by some well known Software Developers:
  • Brian Goetz, core Java Language Architect at Oracle.
  • Robert Vandette, Technical Lead for Mobile Edition, does Compact Profiles.
  • Roger Riggs, new Date and Time API. Spec lead of ME 8.
  • Richard Bair, Java Client Architect, JavaFX 8.
  • John Rose, JVM Architect.
Java 8 Powers the Internet Of Things
Some notes in the promotional blurb at the beginning of the webcast:
  • Nine Million Java Developers
  • #1 Development Platform
  • Increased Developer Productivity
  • Revolutionary Platform Release
Some non-exhaustive list of things available in the new release for Java 8:
  • Lambda Expressions
  • Streams
  • Type Annotations
  • Date and Time API
  • Compact Profiles
  • Nashorn Javascript Engine
Java 8 is a Community Technology
There are several Java 8 sessions available to watch at [1].

References

[1] Java 8 Launch Webcast
http://www.oracle.com/events/us/en/java8/index.html
[2] Java 8 Launch Webcast - Overview
http://eventreg.oracle.com/profile/web/index.cfm?PKWebId=0x637279c68
[3] Mark Reinhold’s Blog - JDK 8: General Availability
http://mreinhold.org/blog/jdk8-ga
Everything about Java 8
http://www.techempower.com/blog/2013/03/26/everything-about-java-8/
A deeper look into the Java 8 Date and Time API
http://www.mscharhag.com/2014/02/java-8-datetime-api.html

Monday 24 March 2014

Java 8 Launch Webcast

Tuesday 25th of March 2014, at about 18:00 (Central European Time) Java SE 8 will be launched worldwide.

That's tomorrow.

Just so you know.

I shall attempt to use it in one of my pet projects. In this case that will be the YPPO (YourPersonalPhotographOrganiser).

Tuesday 18 March 2014

Netbeans : Oracle's IDE for the Java Platform

Transfer Solutions, a consultancy, education and managed services company in Leerdam provides a Transfer Café every month. Since last year it has been open to non-Transfer Solutions personnel. I decided to go Thursday, 13th of March 2014, 18:15 as I was interested in the subject matter discussed that evening.

There to tell us all about NetBeans was the Principal Product Manager, Geertjan Wielenga1, of Oracle.

One of the reasons for the existence of NetBeans, is that Oracle would look silly to release a brand new version of Java or JEE, and not have anything available for Developers to play with it. We as developers would have to wait until Eclipse, JDeveloper or IntelliJ catches up. So it just has an excellent upside for them.

One of the advantages of Java and JEE is that they are standards. This means there are definitions for what file does what and goes where and what format it can have. This makes it ideal for IDE's to automatically generate entire file structures based on user choices.

One of the primary requirements for NetBeans is to provide an extremely (seriously!) low threshold to start designing Java and/or JEE software.

Some of the items that came up during the talk:
  • NetBeans makes it very easy to start learning and programming for a starting Java developer.
  • NetBeans has everything! And if it doesn't have it, it is likely that you won't need it.
  • NetBeans has good integrated Maven support
  • Tutorials can provide you everything from a simple HelloWorld application to an entire Web Application Online Shopping cart on an Application Server
  • NetBeans Website apparently has a weekly Newsletter.
  • NetBeans can show you the Graph of your Maven Dependencies, and you can change the graph, and therefore mutate your dependencies, and export it to image. How cool is that?
  • NetBeans supports The Cloud, for example Amazon Beanstalk out of the box.
  • You can easily import your Eclipse Coding Guideline Settings (formatting and the like) into NetBeans.
  • There's a hook that allows you to automatically do stuff to a file right before you try and save it. For example "Organize imports". (Tools-Options-Editor-On Save)
  • Working with many different projects, it is difficult (especially when using multiple branches) to see to which project the file you are editing belongs. Therefore you can have coloured tabs! (Tools-Options-Miscellaneous-Windows)
  • If you are like some people, that have many files open in the editor, it gets difficult to keep track of them all. There are many solutions. One is the possiblity to indicate that you wish to see multiple lines of tabs (instead of scrolling). (Tools-Options-Miscellaneous-Windows)
  • In order to change the font-size of the editor (for example during presentations), try Alt-MouseScroll.
  • To easily change a lot in a file in the editor, try "Blockselection".
  • There is a nice GUI designed for Swing. Unfortunately, if you wish to use a GUI designer to create JavaFX user interfaces, you are forced to use SeamDesigner, an external tool.
  • NetBeans has integration possibilities for PhoneGap
  • NetBeans can easily be used for developing HTML5, CSS and JAVASCRIPT by means of plugins to the Chrome browser. Updates to your web files will be immediately visible in the Chrome browser, without even saving the file. Also, selecting an element in the Chrome browser, will immediately show you where in which file the item is put.
  • NetBeans has support for several Javascript frameworks. Especially AngularJS seems to be very easy to use.
  • Bookmarks is a very nice feature allowing you to bookmark lines of code to jump back to when you need it.
  • NetBeans is well integrated with Bug reporting tools, like Bugzilla and JIRA.

My Questions


Software integration, how does it work?
NetBeans provides, in the Services Tab, several hooks into for example Jenkins. Interesting tidbit is that the only difference between Jenkins and Hudson plugin is the Name.
How well works NetBeans with Android Development?
Well, there's a plugin/project that is being actively developed by some people called NBAndroid5. Currently there is nothing native set up in NetBeans and there won't be for the foreseeable future. The NBAndroid project could use some funding, though.
JavaDoc, why is it in a separate tab? Why not similar to Eclipse, where it shows up as a tooltip under the mouse cursor?
Interesting that you should mention it. It was a much requested feature by (former?) Eclipse users, and has been implemented in NetBeans 8.0
What is your experience with Plugins? Compared to, for example, Eclipse, where some are able to destabilize the IDE?
Well, since a lot of what you need is already available in NetBeans without the need for Plugins, the plugins landscape is not as vast as Eclipse. With that in mind, the core functionality of NetBeans is already very big, and very very stable.
Do you eat your own dogfood?
Well, of course! We develop NetBeans inside NetBeans.
How do you keep a big beast like NetBeans contained?
NetBeans is actually designed to be very modular with loose coupling, so there are teams working on the different modules of NetBeans. It works out very well. For more information on how we as developers can make use of this modularity for our own applications, see the chapter on NetBeans Platform below.
What does NetBeans use? Maven or Ant?
Ant is the default. If you choose "Create Java Application", you will get an Ant-build java project. Select "Maven Projects" if Maven is what you need. If you wish to port your application that was created using Ant to Maven, you have to do it by hand. There are no convenient tools for doing it for you.

NetBeans as a Platform

NetBeans as a Platform (Let's call it NAAP, I've not heard the term before anywhere, but it's going to have to start somewhere.) is a way of leveraging NetBeans to quickly build a Rich Desktop Application using Modular components, of which NetBeans itself is composed.

You can build your own Rich Desktop Application, in the same way that NetBeans does it. And let's be honest here, NetBeans is doing a fine job.

The advantages are legion:
  • can accommodate many users
  • can support very large applications
  • can create applications for 'the long haul', need to be stable and extendable for many years
  • can accommodate many applications, large organizations with many users usually do not have one single application, but often over a hundred.
  • modularity is key here, in order to keep maintenance manageable and development in parallel between different teams
  • loose coupling, between modules, is also key
  • can provide a consistent UI over many applications, while you can focus on the Business Logic. No more applications that are notorious for their flawless business logic and their crummy UI/Layout/Mainframe Forms, but flawless looking clients.

For screenshots see [2].

References

[1] GeertJan's Blog
https://blogs.oracle.com/geertjan/
[2] NetBeans as a Platform
http://platform.netbeans.org
[3] NetBeans Platform for Beginners
http://www.leanpub.com/nbp4beginners
[4] Transfer Solutions Presents: Transfer Café: NetBeans: Oracle's IDE voor het
Java Platform
http://eventreg.oracle.com/profile/web/index.cfm?PKWebID=0x514725fdc&source=WWPN13024859MPP133&goback=.gde_2120_member_5836764236474249217
[5] NBAndroid
http://www.nbandroid.org/
Adam Bien - Nothing compares... to NetBeans
http://www.adam-bien.com/roller/abien/entry/nothing_compares_8230_to_netbeans


Tuesday 11 March 2014

Simple Logging - Extended

This is a followup of the Simple Logging blogpost.

It is possible to change the logging.properties file to use a FileHandler. This directs everything to a file, (obviously).

Check reference 3 for a good explanation of the logging.properties file.

Unfortunately the default values of the FileHandler in the default JSE logging.properties6 file leaves a lot to be desired.

property
default
comments
mine
limit5000050 kilobytes is quite small for a log file5000000
count1this means the total number of files written is 1, so if the log increases beyond 'limit', you lose the old messages as the file is re-initialized100
appendfalsemight want to set this to true, to prevent loss of log messages upon restart. Unfortunately, this means you'll have *two* XML files within the log and they need to be separated afterwards.true

The following Handlers seem to be already available in Java4, if you're looking for something different.

I very much like the XMLFormatter. It provides even more information than general logging, and it can be easily post-processed.

JLogViewer


JLogViewer1 can view the XML logs written to file. I am sure there are other programs that perform the same function, but this is the one I happened to come across.

java -cp jlogviewer_1_0_0d.jar gianluca.utility.LogViewer

Fixing the logs

Unfortunately for me, the logs contain odd characters (because I am writing received data from sockets into the logs) that are not properly handled by jlogviewer.

I was required to remove the special characters from my logs. Found a blog post2 on how to remove them.
cat java0.log.0 | tr -d "\013" | tr -d "\001" | tr -d "\000" | tr -d "\017" > sanitized_java.log

Update 17-03-2014: updated the item on appending to existing logs.

References

[1] JLogViewer
http://jlogviewer.sourceforge.net/
[2] Removing special characters in Linux
http://deepupc.wordpress.com/2009/10/28/removing-special-charactersmca-in-vi/
[3] java Logging - Configuration
http://tutorials.jenkov.com/java-logging/configuration.html
[4] javadoc java.util.logging.Handler
http://docs.oracle.com/javase/7/docs/api/java/util/logging/Handler.html
[5] javadoc java.util.logging.LogManager
http://docs.oracle.com/javase/7/docs/api/java/util/logging/LogManager.html
[6] default logging.properties provided by JRE 7
https://gist.github.com/maartenl/9346237

Tuesday 4 March 2014

SocketProxyServer

I recently had a need to check out the communication between a client and a server, as well as act upon the contents. Now, I could have used a Network Sniffer to find that out, but it's hard to have a hook with a Network Sniffer and a Java program. Besides, the level of detail is a bit too high. I was only interested in what data is actually going over the line.

With this in mind, I set about to write my "socket proxy server".

The main reason for this "socket proxy server" is because I both do not control the client and the server, yet I wish to "be involved". One saving grace is that I can change the server IP address on the client.

I tested my socket proxy server by starting it as a proxy to a simple website.

java -cp socketproxy.jar com.tools.socketproxy.Main 8080 www.karchan.org 80

So, now I could connect to http://localhost:8080 and I would see the website provided.

Here is the communication as the socket proxy server saw it. First the messages as sent by the client.
received from client:{GET / HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: has_js=1
Connection: keep-alive
If-Modified-Since: Wed, 26 Feb 2014 16:26:34 +0000
If-None-Match: "1393431994"

}
Consequently the reply from the server.
received from server:{HTTP/1.1 200 OK
Date: Wed, 26 Feb 2014 16:29:04 GMT
Server: Apache/2.2.15 (CentOS)
X-Powered-By: PHP/5.3.3
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified: Wed, 26 Feb 2014 16:29:04 +0000
Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0
ETag: "1393432144"
Link: </node/1>; rel="canonical",</node/1>; rel="shortlink"
X-Generator: Drupal 7 (http://drupal.org)
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8

26a5
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN"
  "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" version="XHTML+RDFa 1.0" dir="ltr"
  xmlns:content="http://purl.org/rss/1.0/modules/content/"
  xmlns:dc="http://purl.org/dc/terms/"
  xmlns:foaf="http://xmlns.com/foaf/0.1/"
...


The UML diagram above, shows the main Thread classes and how they relate to each other.

Dealing with multi-threaded programming is always hard and dangerous. In our case, the threads are provided by a fixed thread pool in the concurrency package of Java. I also had a List of Messages, so the list was created using Collections.synchronizedList(List list). What helps is that two connections cannot interfere with each other, making threading easier.

You can find the source code (Java 7) and javadoc in the references below.

Currently the functionality is limited, however it should be easy to extend.

For example, the current listener will only be called after the conversation has finished. This makes the current implementation unsuitable for directly interfering in the communication.

I required this socket proxy server for one of my private hobby projects. This will be the subject of one of my next blog posts. See [1].

References

Github - socketproxy repository
https://github.com/maartenl/socketproxy
Github - socketproxy javadoc
http://maartenl.github.io/socketproxy/javadoc/
[1] Google App Engine - First Try
http://randomthoughtsonjavaprogramming.blogspot.nl/2014/03/google-app-engine-first-try.html