Tuesday, 22 October 2024

Git: Adding your own commands

So I do have a habit of adding my own commands, simply as an alias, simply in the startup scripts of my *nix account and it works fine.

But apparently it also can be done inside git.

So in the example in [1], the new command is "git eradicate".

It was added to the ~/.git-config file as:

eradicate = "!git reset --hard; git clean -fdx"

"-fdx" will also remove files that are mentioned in the .gitignore file.

As there are settings in there, this is perhaps not what you want.

According to the manual in [2] regarding git clean, the option "-x" has the following interesting effect:

"Don’t use the standard ignore rules (see gitignore[5]), but still use the ignore rules given with -e options from the command line. This allows removing all untracked files, including build products. This can be used (possibly in conjunction with git restore or git reset) to create a pristine working directory to test a clean build."

In that case, perhaps the one below is safer:

eradicate = "!git reset --hard; git clean -fd"

References

[1] Medium.com - SE Radio, Git Eradicate, Progress On JMS
https://medium.com/nipafx-news/jpms-support-for-module-versions-a-research-log-2c96f5d0c1ee
[2] Git - --distributed-is-the-new-centralized - Git Clean
https://git-scm.com/docs/git-clean
explainshell.com
https://explainshell.com/explain

Monday, 14 October 2024

Devoxx 2024 - Writeup

So, I went to Devoxx 2024 and I thought it would be good idea to write up what I've witnessed, and what I've missed and thought was interesting.

What I would have liked to see

And since these things are available on the Internet, I'll see about viewing them afterwards.

Notes

The Security Manager will be removed, which makes sense. It's one of the last things left over from the "Java Apps running in your Browser" - days. Nowadays, all the security takes place in dockers and containers and operating systems etc.

JEP stands for Java Enhancement Proposal. But these can be very different. Apparently there are "Process" JEPs and "Informational" JEPS (for example JEP 14).

StringTemplates was removed from the JDK for now, next iteration of the design in the works soonish.

With the new Memory API, there's a focus on making it secure, because a lot of security breaches and hackers make use of badly written code that messes with memory. The white house even published something about it. See the references. In the future there might even be a Draft JEP for it "Integrity by Default".

In the early days, arithmetic and memory fetch had the same cost. Nowadays the CPUs are soo fast, arithmetic has a much faster operation speed, than a memory fetch. This needs to be taken into account when designing language features.

Quotes

Some relevant quotes that I picked up during talks, always fun:

“We do these things not because they are easy, but because we thought they would be easy.”
“I apologize for writing you a long letter. I did not have time to write a short one.”
- Blaise Pascal
“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.”
- Antoine de Saint-Exupéry

References

Sheets - Java 23 Better Language, Better APIs, Better Runtime
https://slides.nipafx.dev/java-x/#/
Sheets - Memory API patterns, use cases and performance
https://speakerdeck.com/josepaumard/memory-api-patterns-use-cases-and-performance
Devoxx Google Cloud
https://cloud.google.com/developers/devoxxbelgium
WebForJ
https://documentation.webforj.com
Martin Fowler - Anemic Domain Model
https://martinfowler.com/bliki/AnemicDomainModel.html
Github - DevoxxGenie
https://github.com/devoxx/DevoxxGenieIDEAPlugin
IntellIJ Plugins - DevoxxGenie
https://plugins.jetbrains.com/plugin/24169-devoxxgenie
Amsterdam Voxxeddays
https://amsterdam.voxxeddays.com/
Google Notebook
https://notebooklm.google.com/?pli=1
Oracle Cloud
https://go.oracle.com/LP=144680
OpenJDK - JEP 14: The Tip & Tail Model of Library Development
https://openjdk.org/jeps/14
Crafting AI Prompts Framework - Adversarial Prompting
https://craftingaiprompts.org/documentation/adversarial-prompting#adversarial-prompting
Project Valhalla
https://openjdk.org/projects/valhalla/
The White House - Press Release: Future Software Should Be Memory Safe
https://www.whitehouse.gov/oncd/briefing-room/2024/02/26/press-release-technical-report/
Quarkus In Action
https://developers.redhat.com/e-books/quarkus-action
The best Java 22 feature: concurrent gathering
https://softwaregarden.dev/en/posts/new-java/gatherers/concurrent/
GitHub - Modern Java In Action
https://github.com/nipafx/modern-java-demo

Sunday, 6 October 2024

Devoxx 2024

Hello, there!

I am going to Devoxx 2024, in Antwerpen for the entire week (starting coming Monday, 6th October 2024). It's been quite a while since I went to a conference (two years?) and the last time I went to Devoxx is in 2019 (which was five years ago).

I'll try and write some blogposts on it.

Sunday, 29 September 2024

My Current MDADM Setup

Just some notes on my current MDADM setup, so I do not forget.

So my current hard drives are:

# lsscsi
[0:0:0:0]    disk    ATA      Samsung SSD 860  1B6Q  /dev/sda 
[4:0:0:0]    disk    ATA      WDC WD4003FZEX-0 1A01  /dev/sdb 
[5:0:0:0]    disk    ATA      WDC WD5000AAKS-0 3B01  /dev/sdc 
[5:0:1:0]    disk    ATA      ST2000DM001-1CH1 CC29  /dev/sdd 
[10:0:0:0]   disk    WD       Ext HDD 1021     2021  /dev/sde 
[11:0:0:0]   disk    WD       Ext HDD 1021     2021  /dev/sdf 

The first one is my primary and Operating Systems and all that. Seems obvious.

The second one contains a large partition for my /home on sdb1.

Partition sdc1 of the sdc drive is a simple 500Gb partition containing some odds and ends. Not big enough for the RAID, not fast enough for the operating system.

Starting my raid setup I can do with:

# mdadm --assemble /dev/md127 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdb2 mdadm: /dev/md127 has been started with 4 drives.

Just so you know, it's a simple mirror setup, so I am using 4 drives to mirror a 2 TB partition.

And then I simple mount the damn thing with:

# mount /dev/md127 /mnt/raid/

Then I can watch the status with detail:

mdadm --detail /dev/md127

Tuesday, 10 September 2024

ORM Hate

I came upon an article1 of Martin Fowler, regarding ORM, and how a lot of people tend to dislike it?

It makes things a lot clearer for me.

Also because I see now that the CQRS way of working is actually used in our company, where we use JPA/QueryDSL for updating, and SQL/QueryDSL for reading.

References

[1] MartinFowler.com - Orm Hate
https://martinfowler.com/bliki/OrmHate.html

Friday, 16 August 2024

On Database Constraints in Oracle

So there was something confusing about a check constraint in one of my tables.

I've proven that the check constraint works as it should, but I failed to understand why.

Actually there were two constraints that sort of combine.

Let's say I have a table called "customers". And that there are two constraints at play:

  • constraint check01 check (bankaccountnr is not null or creditcardnr is not null or paypalemail is not null)
  • constraint check02 check (bankaccountnr > 0 and creditcardnr > 0 and length(paypalemail) > 0)

The first constraint is obvious, at least one of the fields must be filled in, in order to process orders.

It's the second constraint that is confusing.

I thought that, because of the and, the check02 constraint should fire if I try to enter only a bankaccountnr (as the other two conditions evaluate to false).

And I thought I verified that by using the query:

select * from customers where bankaccountnr > 0 and creditcardnr > 0 and length(paypalemail) > 0;

As I suspected it provided me ONLY the records that had ALL fields entered properly.

But apparently, in (check) constraints, if the condition is "unknown", for example in the case of NULL-values, it is accepted.

According to the following quote, found in [1] which I found via [2]:

A check constraint on a column or set of columns requires that a specified condition be true or unknown for every row. If DML results in the condition of the constraint evaluating to false, then the SQL statement is rolled back.

References

[1] Oracle - Data Integrity
https://docs.oracle.com/database/121/CNCPT/datainte.htm#CNCPT1660
[2] StackOverflow - Why is this check constraint not working when it checks length?
https://stackoverflow.com/questions/66031098/why-is-this-check-constraint-not-working-when-it-checks-length

Saturday, 27 July 2024

jakarta.json.bind.JsonbException: Cannot create instance of a record: class SomeClass, Multiple constructors found.

So I've been trying to use Java Records as data holders for JSONB serialisation and deserialisation. And it works well, until I encountered the following:

jakarta.json.bind.JsonbException: Cannot create instance of a record: class SomeClass, Multiple constructors found.

It turns out, I can put in an annotation @JsonbCreator on it, as explained in reference [1].

It just takes a bit of work, as I cannot put the annotation on top of a record class, it needs to be put on a constructor. The way this works, is to create a compact constructor in the record.

The compact constructor is usually used to put in some sort of validation in the record class upon instantiation, but here we need it for the annotation @JsonbCreate.

Brian Goetz in the comments of reference [2] indicates that this is the proper way to do it.

Unfortunately, it causes one of those "empty constructors" messages of SonarLint.

Warning:(31, 10) Remove this redundant constructor which is the same as a default one.

I assume SonarLint will fix this eventually.

For completeness, my code:

public record AdminItem(Integer id,
                        String belongsto,
                        Long room,
                        String shopkeeper,
                        String owner,
                        LocalDateTime creation)
{

  @JsonbCreator
  public AdminItem
  {
    // empty constructor, because I need to put the annotation @JsonbCreator somewhere.
  }

  public AdminItem(Item item)
  {
    this(item.getId(), 
        item.getBelongsTo() == null ? null : item.getBelongsTo().getName(),
        item.getRoom() == null ? null : item.getRoom().getId(),
        null,
        item.getOwner() == null ? null : item.getOwner().getName(),
        item.getCreation());
  }

}

References

[1] Carlos Chacin - 💾 Java 14 Records 🐞 with JakartaEE JSON-B
https://carloschac.in/2020/04/20/java-records-jsonb/
[2] StackOverflow - Constructor annotation on java records
https://stackoverflow.com/questions/67168624/constructor-annotation-on-java-records

Thursday, 18 July 2024

Angular and Updates

Recently was interested if and when Angular upgrades to a new release, as I found myself skipping release 17, and the update process can be a tiny bit troublesome if you skip a major version.

Angular has Semantic Versioning, and releases a new major version every 6 months. Angular 19 will possibly be out and about end of November 2024.

Luckily, from the references below, there's some things that I can hold on to:

  • a major version may require a bit of work by the developer.
  • a minor version can be installed without problems immediately. No developer assistance is expected during update. However, you can optionally decide to use new features in your code or change old features to use new stuff.
  • a fix/patch version can be installed without problems immediately. No developer assistance is expected during update.

And GitHub tends to complain (a lot!) if my Angular projects are a tiny bit old, and there're Security concerns with the javascript dependencies Angular uses.

With that in mind, the following quote on the website seems very good to know:

"We only make npm dependency updates that require changes to your applications in a major release. In minor releases, we update peer dependencies by expanding the supported versions, but we do not require projects to update these dependencies until a future major version. This means that during minor Angular releases, npm dependency updates within Angular applications and libraries are optional."

References

Angular - Roadmap
https://angular.dev/roadmap
Angular - Releases
https://angular.dev/reference/releases
GitHub - Angular Public API Surface
https://github.com/angular/angular/blob/main/contributing-docs/public-api-surface.md
Angular Blog
https://blog.angular.dev/
Angular Dev
https://angular.dev/

Friday, 12 July 2024

Kotlin: The Spead Operator

Recently ran into a brick wall trying to pass a varargs parameter to another function that also has a varargs parameter.

A colleague mentioned the "spread" operator to me and it took me a little while to find information about it.

An example

package org.mrbear.kotlin

enum class ErrorCode(val description: String) {
    OBJECT_NOT_FOUND("Object %s not found."), NO_DEFAULT_PROVIDED("No default provided for parameter %s."), MALFORMED_URL(
        "Malformed url (%s)"
    )
}

abstract class MyException : Exception {
    constructor(errorCode: ErrorCode, cause: Throwable, vararg params: Any) : super(
        String.format(
            errorCode.description,
            *params
        ), cause
    )

    constructor(errorCode: ErrorCode, vararg params: Any) : super(String.format(errorCode.description, *params))
}

class ObjectNotFoundException(vararg params: Any) : MyException(ErrorCode.OBJECT_NOT_FOUND, *params)

Now to throw it in a test.

class ExceptionTest {

    @Test(expectedExceptions = [ObjectNotFoundException::class], expectedExceptionsMessageRegExp = "Object User mrbear not found.")
    fun testException() {
        throw ObjectNotFoundException("User mrbear")
    }
}

References

Kotlin - Variable number of arguments (varargs)
https://kotlinlang.org/docs/functions.html#variable-number-of-arguments-varargs
Kotlin - Java varargs
https://kotlinlang.org/docs/java-interop.html#java-varargs
Baeldung - Convert Kotlin Array to Varargs
https://www.baeldung.com/kotlin/array-to-varargs
Baeldung - Varargs in Java
https://www.baeldung.com/java-varargs

Sunday, 7 July 2024

Upgrading to Angular 18

Upgrading Javascript things is always a chore.

That's why these blogposts keep appearing.

% ng version

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/


Angular CLI: 16.2.5
Node: 18.16.1
Package Manager: npm 9.7.2
OS: darwin x64

Angular: 16.2.8
... animations, common, compiler, compiler-cli, core, forms
... localize, platform-browser, platform-browser-dynamic, router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1602.5
@angular-devkit/build-angular   16.2.5
@angular-devkit/core            16.2.5
@angular-devkit/schematics      16.2.5
@angular/cli                    16.2.5
@schematics/angular             16.2.5
rxjs                            7.5.7
typescript                      4.9.5
zone.js                         0.13.1

So I've used the tool in [2] to check my dependency requirements but there's also [1] if you're interested.

Upgrading node.js

cd ~
rm nodejs
tar zxvf ~/Downloads/node-v20.15.0-darwin-arm64.tar.gz
ln -s node-v20.15.0-darwin-arm64 nodejs

Upgrading typescript

Upgrading typescript using "npm install -g typescript@latest".

Upgrading zone.js

I'm hoping Angular does this for me.

Upgrading Angular

For my "personal" project.

npm install -g @angular/cli@18

ng update @angular/core@17 @angular/cli@17

ng update @angular/core@18 @angular/cli@18 @ng-bootstrap/ng-bootstrap@17 ngx-cookie-service@18

For my "admin" project, the thing below only worked after upgrading cdk and material to 17.

ng update @angular/core@18 @angular/cli@18 @ng-bootstrap/ng-bootstrap@17 ngx-cookie-service@18 @angular/cdk@18 @angular/material@18

Upgrading dependencies

ng update bootstrap bootswatch material-icons

My dependencies for "personal" project

@kolkov/angular-editor
the editor I use in my webapps
@ng-bootstrap/ng-bootstrap
integration between angular and bootstrap
@popperjs/core
this is used for tooltips by ng-bootstrap
bootstrap
bootstrap, is awesome
bootswatch
free themes for bootstrap, I like/use "cyborg" as a darkmode theme.
material-icons
latest icon fonts and CSS for self-hosting material design icons.
ngx-cookie-service
Angular service to read, set and delete browser cookies.
ng update @angular/cdk@16 @angular/material@16
ng update @angular/cdk@17 @angular/material@17

My dependencies for "admin" project

@angular/material
for creating of a virtual scroller
@angular/cdk
for creation of a virtual scroller
@ng-bootstrap/ng-bootstrap
integration between angular and bootstrap
@popperjs/core
this is used for tooltips by ng-bootstrap
bootstrap
bootstrap, is awesome
bootswatch
free themes for bootstrap, I like/use "cyborg" as a darkmode theme.
material-icons
latest icon fonts and CSS for self-hosting material design icons.
ngx-cookie-service
Angular service to read, set and delete browser cookies.

Things I noticed

Migrate application projects to the new build system.
You can read more about this, including known issues and limitations, here: https://angular.dev/tools/cli/build-system-migration The output location of the browser build has been updated from "../webapp" to "../webapp/browser". You might need to adjust your deployment pipeline or, as an alternative, set outputPath.browser to "" in order to maintain the previous functionality.
Replace deprecated HTTP related modules with provider functions.
HttpClientModule replaced with provideHttpClient, withInterceptorsFromDi

Error messages when upgrading

Package "@ng-bootstrap/ng-bootstrap" has an incompatible peer dependency to "@angular/common" (requires "^16.0.0" (extended), would install "18.0.6").
[WARNING] Polyfill for "@angular/localize/init" was added automatically. [plugin angular-polyfills]

  In the future, this functionality will be removed. Please add this polyfill in the "polyfills" section of your "angular.json" instead.

I found the answer to this one in [3] on StackOverflow.

[WARNING] Unable to locate stylesheet: /assets/css/bootstrap.min.css
[WARNING] Unable to locate stylesheet: /assets/css/karchan.css

Replaced this with a ThemeService (which is nice) which I found at [4].

And removed the hardcoded stylesheets in index.html. These are now added by the ThemeService.

Final result

% ng version

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/


Angular CLI: 18.0.7
Node: 20.15.0
Package Manager: npm 10.8.1
OS: darwin arm64

Angular: 18.0.6
... animations, cdk, common, compiler, compiler-cli, core, forms
... localize, material, platform-browser
... platform-browser-dynamic, router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1800.7
@angular-devkit/build-angular   18.0.7
@angular-devkit/core            18.0.7
@angular-devkit/schematics      18.0.7
@angular/cli                    18.0.7
@schematics/angular             18.0.7
rxjs                            7.5.7
typescript                      5.4.5
zone.js                         0.14.7

Small note

I am using the assets to store a normal bootstrap (bootstrap.min.css) and a darkmode bootstrap (bootstrap.darkmode.min.css).

These come from respectively node_modules/bootstrap/dist/css/bootstrap.min.css and node_modules/bootswatch/dist/cyborg/bootstrap.min.css.

References

[1] TechiDiaries - Upgrade to Angular 18
https://www.techiediaries.com/upgrade-angular-to-v18/
[2] Official Angular Updater Tool on the Web
https://update.angular.io/
[3] StackOverflow - Angular 18 Polyfills warning
https://stackoverflow.com/questions/78636168/angular-18-polyfills-warning
[4] Medium - Theme Switcher in Angular: From Dark to Light and Back Again
https://medium.com/@davdifr/theme-switcher-in-angular-from-dark-to-light-and-back-again-f42fc3f9fab0

Monday, 3 June 2024

Kotlin Scope Functions

Scope function examples below:

  • let
  • with
  • run
  • apply
  • also
  • takeIf and takeUnless
  • Using them all at the same time!!!

I really hope I'm not the only one that gets confused about the different Scope functions1 in Kotlin and what they mean and when to use what.

The reference in [1] is excellent, but if I have to look up documentation on what certain methods do, the methods are not very well named.

So, in short, here's some examples, actually gleaned from the documentation and given my own spin on it with things that actually make sense.

Let

    /**
     * Useful if you do not wish to assign the result to an intermediate variable.
     * Useful with ? in case the result is NULL.
     */
    fun getDescription(address: Address?): String? =
        address?.let { address.getDescription() }

With

    /**
     * "with this object, do the following."
     * We're not interested in the result.
     */
    @Test
    fun withTest() {
        with(addressInEngland) {
            assertThat(street).isEqualTo("Morsestreet")
            assertThat(city).isEqualTo("London")
        }
    }

    /**
     * "with a helper object, do the following."
     */
    @Test
    fun withHelperTest() {
        val description = with(addressHelper) {
            computeAddress(addressInEngland)
        }
        assertThat(description).isEqualTo("12 Morsestreet London")
    }

Run

    /**
     * "run the code block with the object and compute the result."
     * Nice if you need to use it in an expression.
     */
    @Test
    fun runAsExtentionFunctionTest() {
        val didItWork: Boolean = database.run {
            val address = retrieveAddressFromDatabase()
            addressInAmerica.pobox = "43000"
            updateInDatabase(addressInAmerica)
        }
    }

    /**
     * "run the code block and compute the result."
     * Does not have a "this" or "it". Nice if you need to use it in an expression.
     */
    @Test
    fun runAsNonExtentionFunctionTest() {
        val didItWork = run {
            val address = database.retrieveAddressFromDatabase()
            with(shippingService) {
                sendItemToAddress(address)
            }
            log("Item sent")
            success
        }
    }

Apply

    /**
     * "apply the following assignments to the object."
     * The most common use case is object configuration, as below.
     */
    @Test
    fun applyTest() {
        val sameObject = Address().apply {
            housenumber = 12L
            street = "Morsestreet"
            city = "London"
        }
    }

Also

/**
     * "and also do the following with the object."
     * A common use case is to also assign the object to a property/variable.
     */
    private fun createHomeAddress() =
        Address().apply {
            housenumber = 12L
            street = "Morsestreet"
            city = "London"
        }
            .also { homeAddress = it }

TakeIf and TakeUnless

    /**
     * "return/use the value if this condition is true"
     * The opposite is takeUnless.
     */
    @Test
    fun takeIfTest() {
        assertThat(addressInEngland.takeIf { it.state != null }).isNull()
        assertThat(addressInAmerica.takeIf { it.state != null }).isEqualTo(addressInAmerica)
    }

And now for the big one!!!

    /**
     * Let's try all of them at the same time!
     */
    @Test
    fun allScopesTest() {
        val mailingSentForNewAddress = Address()
            .apply {
                housenumber = 12L
                street = "N. High Street"
                city = "Columbus"
                state = "Ohio"
                country = "United States of America"
            }
            .also {
                homeAddress = it
                with(database) {
                    updateInDatabase(it)
                }
            }
            .takeUnless { mailingAlreadySent(it) }
            ?.run {
                sendMailing()
                log("Mailing sent to ${getDescription()}.")
                success
            } ?: false

        val mailingSentForAddress =
            with(addressInEngland) {
                takeUnless { mailingAlreadySent(this) }
                    ?.run {
                        sendMailing()
                        log("Mailing sent to ${getDescription()}.")
                        success
                    } ?: false
            }
    }

Here I have tried to make use of the Scope functions in such a way that the different operations make sense.

P.S. I take offence on using "it" as the automatic name for the argument of the lambda. It sounds too much like the old Java "int i" in for loops. In other words, "it" has no meaning and the meaning depends entirely on context.

There's too much "it" and too much "this" and the context switching when using several Scope functions makes my head hurt.

References

[1] Kotlinlang.org - Scope Functions
https://kotlinlang.org/docs/scope-functions.html#functions

Friday, 10 May 2024

Caching REST Resources In Jakarta REST

I found the following blog post on caching interesting. See reference [1]

I like caching, but I understand there's a very real danger of seeing outdated results. The Blogpost highlights this quite good.

HTTP Caching has several great options, apparently:

  • Expired Header (HTTP 1.0) - ResponseBuilder.expires
  • Cache-Control Header (HTTP 1.1) - CacheControl class for more control
  • ETags - EntityTag class, indicates a kind of hashcode to verify if a Resource has changed, without actually accessing the resource

Nice.

References

[1] Caching REST Resources In Jakarta REST (formerly JAX-RS)
https://blog.payara.fish/caching-rest-resources-in-jakarta-rest-formerly-jax-rs?hs_preview=dxtFzcIy-164496731149

Thursday, 2 May 2024

Stream.sorted() - For unordered streams, no stability guarantees are made.

So I was writing code, and I had to sort a stream, because originally I had a Set, and everyone knows Sets are unordered by default.

But my eye fell on the javadoc on .sorted(). To be more specific:

For ordered streams, the sort is stable. For unordered streams, no stability guarantees are made.

And the sentence kind of made me nervous.

Example

An example of an unstable sort is visible in the unit test below. The unit test will sometimes pass and sometimes fail.

The reason for this is that Ken Thompson and Simon Thompson are equivalent in the sorting and, as it is an unsorted stream to begin with, its order can change between program runs.

Something to keep in mind.

References

Oracle Javadoc - Stream (Java SE 21 & JDK 21)
https://docs.oracle.com/en%2Fjava%2Fjavase%2F21%2Fdocs%2Fapi%2F%2F/java.base/java/util/stream/Stream.html#sorted()

Friday, 23 February 2024

Migrating Oracle to PostgreSQL

So at work we're currently trying to see if we can move away from Oracle and onto PostgreSQL.

I expect this to be a long process.

This blog post is here to document some of the issues we ran into and also remember references I found.

Migrating schema and data

There are several tools that can migrate the schema and the data from Oracle to Postgres.

There are some differences in what both database support. See reference [1].

Make changes to SQL Scripts

References

[1] Hackermoon - How to Migrate from Oracle to PostgreSQL
https://hackernoon.com/how-to-migrate-from-oracle-to-postgresql
Postgres Wiki - Oracle to Postgres Conversion
https://wiki.postgresql.org/wiki/Oracle_to_Postgres_Conversion
PostgreSQL Manuals
https://www.postgresql.org/docs/
Ora2Pg - Moves Oracle and MySQL database to PostgreSQL
https://ora2pg.darold.net/
Life in USA - install ora2pg on mac osx
https://vcfvct.wordpress.com/2016/10/18/install-ora2pg-on-mac-osx/

Friday, 12 January 2024

SQL: Concat operator and Spaces

So I had a pressing need to query my database and concatenate several values from a row together (using either concat or ||), but with spaces in between.

But I don't want two spaces if the value in between is empty.

It is surprisingly difficult to do in SQL, but there are some options available.

First the data:

TITLEFIRSTNAMEMIDDLENAMEFAMILYNAME
IchabodCrane
Dr.ThomasLancaster
PhilipMartinBrown

0. No solution

select TITLE || ' ' || FIRSTNAME || ' ' || MIDDLENAME || ' ' || FAMILYNAME from PERSON;

So, if you use the SQL statement above, you get the problem, extraneous spaces. For example "[ ]Ichabod[ ][ ]Crane". We wish to eliminate those.

1. COALESCE

select coalesce(TITLE || ' ','') || coalesce(FIRSTNAME || ' ','') || coalesce(MIDDLENAME || ' ','') || FAMILYNAME from PERSON;

This is a good try, but doesn't work, as the first expression of the coalesce is never NULL, because of the concat used with the space (|| ' ') used.

1. DECODE

select decode(TITLE, null, '', TITLE, TITLE || ' ') ||
       decode(FIRSTNAME, null, '', FIRSTNAME, FIRSTNAME || ' ') ||
       decode(MIDDLENAME, null, '', MIDDLENAME, MIDDLENAME || ' ') ||
       FAMILYNAME
       from PERSON;

This works but is a lot of code to do something very simple.

2. CASE

select (case
           when TITLE is null then ''
           else TITLE || ' ' end) ||
       (case
           when FIRSTNAME is null then ''
           else FIRSTNAME || ' ' end) ||
       (case
           when MIDDLENAME is null then ''
           else MIDDLENAME || ' ' end) ||
       FAMILYNAME
from PERSON;

This works but is a lot of code. Luckily the case statement, when indented properly, is quite readable and not at all bad.

3. NVL2

select nvl2(TITLE, TITLE || ' ', '') ||
       nvl2(FIRSTNAME, FIRSTNAME || ' ', '') ||
       nvl2(MIDDLENAME, MIDDLENAME || ' ', '') ||
       FAMILYNAME from PERSON;

This works but is hard to read. Requires knowledge of nvl2.

4. LTRIM

select ltrim(TITLE || ' ') ||
       ltrim(FIRSTNAME || ' ') ||
       ltrim(MIDDLENAME || ' ') ||
       FAMILYNAME from PERSON;

As far as I can tell, this is quite appropriate. The ltrim removes spaces at the beginning of the string. If the string contains only ' ', all ' ' are removed and we're home free.

It is short, but requires some knowledge of what ltrim does exactly.

Falls in the category of 'doing something clever'.

5. REGEXP_REPLACE

select regexp_replace(TITLE
        || ' ' || FIRSTNAME
        || ' ' || MIDDLENAME
        || ' ' || FAMILYNAME
        , ' +', ' ')
from person;

A colleague of mine came up with this little chestnut.

It's nice if you enjoy regular expressions. I do not.

Still, it's nice to be able to just concat everything together, and have the whole thing sorted out with one simple regular expression.

Addendum

The function wm_concat has been removed from Oracle PL/SQL and should not be used.

Which is a shame, as it does exactly what we want.

References

American Culture - Naming
https://culturalatlas.sbs.com.au/american-culture/american-culture-naming
Lalit Kumar B - Why not use WM_CONCAT function in Oracle?
https://lalitkumarb.wordpress.com/2015/04/29/why-not-use-wm_concat-function-in-oracle/

Updates: added regexp_replace solution