Thursday, 12 January 2017

Base 64 Encoding and URLs

I recently had some issues with base64 encoding of images and documents, prior to sending them over HTTP to the frontend of my app, and there decoding it again.

The issues were manifold.

Let me try and indicate the problems I encountered.

In the backend I use the BaseEncoding class provided in the Core of Google Guava. ( The method "base64()"speaks for itself.

At the frontend, using Javascript, I used the "atob()" method1 to get the whole base64 encoded string back into it's original shape.

Problem 1 - String contains an invalid character

The "atob()" method threw an error, when attempting to decode String. After some research, involving comparing the string that is sent by the Backend, with the string received by the Frontend, I did notice two differences.

Apparently, the backend is sending a url-safe encoded string2, despite me not having specified that this is
what I want.

Well, the differences aren't major and a simple solution does the trick:
atob(contents.replace(/-/g, "+").replace(/_/g, "/"));
And voila, atob() no longer complains about invalid characters.

Problem 2 - Decoded document does not match original document

My app stored the received document, after decoding, into a file locally, so it can be opened. My App is created using Cordova (and Ionic and some other stuff) and I use the Cordova File Plugin3 to write the file.

The PDF document that I was using as a test, seemed to be transferred just fine, but upon opening it on the Tablet, an empty PDF document was shown.

There were vast differences between the original document and the decoded document. The differences seem to focus on the decidedly "weird" characters. The alphabet seemed just fine.

Being at a loss for the moment, I decided to use a library named js-base644. That didn't help in the slightest. Using runkit5 to test it, I found that it actually decodes base64 encoding into UTF-8 badly.

There were a number of bugs reported with it in GitHub.

Problem 3 - Cordova File Plugin

After switching back to the method "atob()", and comparing the output of "atob()" with the original document, I found them to be identical.

However, the file stored on the Tablet was still suffering from the exact same symptoms. Clearly something was going wrong with the Cordova File Plugin.

After looking at the documentation3, I found that the File Plugin will also output UTF-8, similar to the js-base64 library.

In the end, I found out that it only outputs UTF-8, if I write a string in a Blob to the File. If I change what I write into a JavaScript ArrayBuffer in a Blob, things work as they should.

And I finally got a nice PDF in the standard PDF Viewer of the Tablet.


[1] MDN - Base64 encoding and decoding
[2] RFC4648 The Base16, Base32, and Base64 Data Encodings - Section 5
[3] File - Apache Cordova - cordova-plugin-file
[4] js-base64 - Yet another Base64 transcoder in pure JS
[5] Runkit
StackOverflow - Using Javascript's atob to decode base64 doesn't properly decode utf-8 strings
C#411 - Convert Binary to Base64 String

1 comment: