Kotlin: From List To Map

So I've been trying to make a Map from a List in Kotlin, and as I have not much experience with Kotlin and not much experience with Maps, I turn to the Internet.

Funnily enough, after getting it working, my IDE keeps giving me subtle hints that it can be shorter and more concise.

I thought it would be nice to put the steps here.

val primaryKeys = listOf(12L, 15L, 16L, 22L, 204L)
val firstTry = primaryKeys.map { it to getUser(it) }

In the example above it's not a Map yet, but it's a good first step.

As you can tell, it transforms your list into another List of type List<Pair<Long, User>>.

Kotlin has a toMap() method that does what we want.

    @Test
    fun testListToMapConversion() {
        val primaryKeys = listOf(12L, 15L, 16L, 22L, 204L)
        val map = primaryKeys.map { it to getUser(it) }.toMap()
        assertThat(map).isEqualTo(
            mapOf(
                12L to User(12L, "Bob"),
                15L to User(15L, "Jimmy"),
                16L to User(16L, "Jack"),
                22L to User(22L, "Henry"),
                204L to User(204L, "William")
            )
        )
    }

Now my IDE tells me this can be shortened.

    @Test
    fun testListToMapConversion() {
        val primaryKeys = listOf(12L, 15L, 16L, 22L, 204L)
        val map = primaryKeys.associate { it to getUser(it) }
        assertThat(map).isEqualTo(
            mapOf(
                12L to User(12L, "Bob"),
                15L to User(15L, "Jimmy"),
                16L to User(16L, "Jack"),
                22L to User(22L, "Henry"),
                204L to User(204L, "William")
            )
        )
    }

Now my IDE tells me this can be shortened AGAIN!

    @Test
    fun testListToMapConversion() {
        val primaryKeys = listOf(12L, 15L, 16L, 22L, 204L)
        val map = primaryKeys.associateWith { getUser(it) }
        assertThat(map).isEqualTo(
            mapOf(
                12L to User(12L, "Bob"),
                15L to User(15L, "Jimmy"),
                16L to User(16L, "Jack"),
                22L to User(22L, "Henry"),
                204L to User(204L, "William")
            )
        )
    }

Nice!

References

Syntax Highlighter

https://highlight.hohli.com/

Java - Behaviour of Equals in Collections

I just thought I'd write some things down that I already know, but sometimes it's nice to see this proven.

It's a very very beginner Java subject, but people won't fault me for blogging about it.

It might be of some use to somebody.

Combining two collections using streams

So, I have two collections, and I wish to combine both lists somehow.

Requirements are thusly:

• I have a collection newPersons
• I have a collection oldPersons
• I want a collection containing all the oldPersons but replaced (*some of) the oldPersons with the newPersons (based on id).

public record Person(Long id, String name, String surname) {
}

Using the record class above.

Solution 1.

Create a new list based on oldPersons, replace items in this new list with equivalent items in newPersons.

This solution leaves much to be desired. It's confusing and error prone.

I was looking for something better.

  public List<Person> mergeMetadata(List<Person> newPersons,
      List<Person> oldPersons) {
    var result = new ArrayList<>(oldPersons);
    newPersons.forEach(newPerson -> {
      result.stream()
          .filter(person -> person.id().equals(newPerson.id()))
          .findFirst()
          .ifPresent(result::remove);
      result.add(newPerson);
    });
    return result;
  }

Solution 2.

Sometimes getting back to our roots using for-loops can help readability.

We could try it the other way around, see if that helps.

This time we create a new list based on newPersons and add an oldPerson if it's not already in the list.

This seems a little more clear.

  public List<Person> mergeMetadata(List<Person> newPersons,
      List<Person> oldPersons) {
    var result = new ArrayList<>(newPersons);
    for (Person oldPerson : oldPersons) {
      if (result.stream().noneMatch(x -> x.id().equals(oldPerson.id()))) {
        result.add(oldPerson);
      }
    }
    return result;
  }

Solution 3.

Merge two collections into one list, by using a map.

  public List<Person> mergeMetadata(List<Person> newPersons,
      List<Person> oldPersons) {
    Map<Long, Person> result = Stream
        .concat(newPersons.stream(), oldPersons.stream())
        .collect(Collectors.toMap(Person::id, Function.identity(), (l, r) -> l));
    return new ArrayList<>(result.values());
  }

Although this solution seems to be the shortest (in code), using a Map function can be a bit daunting (it was for me) because of inherent complexity in the method call for creating it.

Still, perhaps it's just me and my inexperience with combining maps and streams.

I don't know if there's an even better way. Will keep an eye out.

Finding the index of an element in a List using Streams

So, I recently had a need to find out the index of an element in a List.

I could take the traditional approach, and use a for loop.

But I was wondering if I could use Streams, and if I could, would that be "better" (for some vague notion of "Better").

Any comments on what you would prefer, and why?

References

Baeldung - How to Iterate Over a Stream With Indices

https://www.baeldung.com/java-stream-indices

Why aren't Java Collections remove methods generic?

I recently came across a small problem. The program ran fine, but my IDE had it underlined as being highly dodgy.

It is the same for Collection.contains and Map.get

Just see the references why this is.

References

StackOverflow - Why aren't Java Collections remove methods generic?

https://stackoverflow.com/questions/104799/why-arent-java-collections-remove-methods-generic

Youtube - Advanced Topics in Programming Languages: Java Puzzlers,...

https://www.youtube.com/watch?v=wDN_EYUvUq0

Converting an Optional to a List

Just a quick small snippet on how to convert a Optional to a List.

Hope it helps someone, who's new at Optionals.

Using a composite key in a map

Once again I stumbled upon old code, that could use some refactoring.

It took me and my colleague a few seconds to process what the software designer had in mind when he did this.

As it was up to me to add another field to the construction, making it even more terrible than it was, it was time for a refactoring.

Refactored solution

We quite quickly came up with using a Composite Key, which of course is the default way to deal with this sort of thing.

Added benefit

There was some trickery going on to properly deal with NULL keys (which are sometimes not allowed in a Map implementation¹).

This disadvantage now disappears with this nice composite key.

Test setup

I tested it with a list of cars, and I do not wish to withhold the test setup:

Fuzzy matching

Of course it is quite easy to just match on some of the criteria. In this case, we could use a Map containing Lists, where the composite key used has empty fields, and would cause a list of cars to be returned and a full composite key (all search criteria) would just yield a list containing one specific car.

If we wish to use fuzzy matching, we are in a bit of a bind.

I'm pretty sure there are always other (perhaps even better) ways of doing this. If you know of some, let me know.

I'm always interested.

References

[1] Oracle JavaDoc - Map Implementation

https://docs.oracle.com/javase/8/docs/api/java/util/Map.html

Different Types Of Cars List

https://auto.ndtv.com/news/types-of-cars-1450327

Turning a Stream into an Iterable

Recently I wanted to know how to change a stream into an Iterable. The method in the API that I need to call expects an Iterable.

I found the following solution in [1]:

public class RealEstateEvaluator
{
  public void evaluate(Session session, Iterable<House> iterable)
  {
    // do stuff
  }
}

public void realEstateEvaluation()
{
  Stream<House> housenumbersStream = 
    housenumbers.stream()
      .map(realEstateService::getHouse);   

  new RealEstateEvaluator().evaluate(getSession(), housenumbersStream::iterator);
}

However, [1] did mention that the result was unreadable. It is confusing that a method reference that provides an Iterator, perhaps accidentally, fulfills the contract for Iterable².

That's right. Iterable is an interface³, which has three methods, two of which have default implementations. Which means it has one method (iterator()), which means it can be used as a Lambda expression.

I prefer:

public void evaluate()
{
  List<House> houses = 
    housenumbers.stream()
      .map(realEstateService::getHouse)
      .collect(Collectors.toList();   

  new RealEstateEvaluator().evaluate(getSession(), houses);
}

It is a lot more readable. And I am a big believer in the Principle of Least Surprise.

All Collections are Iterables, by the way.

Update 2018-02-23: after reading and evaluating the comment below, I still think using the method reference makes it harder to read, but is much safer to use. Perhaps getting used to using stream::iterator is just a question of time.

References

[1] LambdaFAQ - How do I turn a stream into an iterable

http://www.lambdafaq.org/how-do-i-turn-a-stream-into-an-iterable/

[2] StackOverflow - Why does stream not implement iterable

https://stackoverflow.com/questions/20129762/why-does-streamt-not-implement-iterablet

[3] Oracle JavaDoc - Iterable

https://docs.oracle.com/javase/8/docs/api/java/lang/Iterable.html?is-external=true

AdditionalCriteria

Small followup of From Hibernate to Eclipselink¹ post.

I am not entirely satisfied about the AdditionalCriteria⁴ thingy. I find it a chore to have to set a parameter on the EntityManager all the time to enable/disable it.

Biggest issue for me is that parameters set on the EntityManager are required. If they are omitted, an exception is thrown when querying.

Current solution in my software:
Turn the AdditionalCriteria on or off by means of a parameter that needs to be set on the EntityManager.

Looks like this:
Setting the parameter activePersonFilter can be done on the EntityManager as follows:

@PersistenceContext(properties =
{
  @PersistenceProperty(name = "activePersonFilter", value = "0"),
  @PersistenceProperty(name = "sundaydateFilter", value = "")
})
private EntityManager em;

Or

entityManager.setProperty("activePersonFilter", 0);

Other solutions

There are some other solutions.

1. You can remove the additionalCriteria (set it to "") in a subclass, and use the subclass specifically. See [2].
2. You can customize any mapping in EclipseLink and add the requirements/conditions that you need. See [3].
3. I could just decide to create a view on the offending database table. Then create two entities. Sounds very similar to the first option.
4. I could solve the problem in software. Just have EclipseLink not filter anything. (Which is silly, I don't wish for my ORM to get the 1000 persons in the room from the database, if there are say only three persons active.)
5. I could remove the collection entirely, and retrieve the required Persons using a NamedQuery. (Which is bogus. I like the ORM to deal with this for me, instead of having to do it myself. It's what the ORM is for.)

Customizing a Mapping

I have recently decided to try to customize the mapping specifically in Entities that have collections containing instances of Person class. That way I have more control. See reference [3] on how this works.

It requires a @Customizer annotation.

For instance, in a Room I only wish to see the active persons.

This requires me to define the PersonsFilterForRoom as follows.

"persons"

the name of the field that contains the collection

"room"

the name of the field in the Entity of the collection

"id"

the name of the field in the Room entity that identifies it

It works pretty good.

Note

I also noticed that this way I could have two (Lazy! That's the important bit!) Collections in the same Entity at the same time referring to the same Person. One will contain all Persons and one will contain only the Active Persons.

This is ideal, for instance for Guilds.

Like so:
This way the customizer PersonsFilterForGuild is designed to only work on the activeMembers collection.

I like it!

References

[1] From Hibernate to EclipseLink

http://randomthoughtsonjavaprogramming.blogspot.nl/2014/07/from-hibernate-to-eclipselink.html

[2] StackOverflow - Disable additional criteria only in some entity relations

http://stackoverflow.com/questions/37419406/disable-additional-criteria-only-in-some-entity-relations

[3] Mapping Selection Criteria

https://wiki.eclipse.org/EclipseLink/Examples/JPA/MappingSelectionCriteria

[4] JPA Extention in EclipseLink - @AdditionalCriteria

https://www.eclipse.org/eclipselink/documentation/2.6/jpa/extensions/annotations_ref.htm#additionalcriteria

Customizing EclipseLink JPA/ORM Relationship Joins

http://onpersistence.blogspot.nl/2008/01/customizing-eclipselink-jpaorm.html

Scala Collections

Great, I have just been overthinking myself.

I have the following setup:

val nums = List(('a', 2), ('b', 2))

I wanted to convert this List of Tupels into a Map of the kind Map( a -> 2, b -> 2). So, off I went...

val mapping1 = nums groupBy (_._1)    // mapping1  : Map[Char,List[(Char, Int)]] = 
                                      //    Map(b -> List((b,2)), a -> List((a,2)))
val mapping2 = mapping1.map(x => (x._1, x._2.head._2))
                                      // mapping2  : Map[Char,Int] = Map(b -> 2, a -> 2)

Well, this is hardly convenient.

If you have a Tupel in Scala, like val t = (5, 2), you can access both values using the syntax t._1 and t._2.

Here are the steps:

• groupBy makes a Map, where the key is the first Value in the tupel, with the original tupel assigned as the value.
• then the mapping, it takes each item from the map and transforms them to something else. In our case, it transforms (a, List((a, x))) into (a, x) by:
  
  • leaves the first value of the tupel unchanged, i.e. a
  • takes the head of the List((a, x)), i.e. (a, x)
  • grab the second value from the tupel (a, x), i.e. x

Well, to make a long story short, it seems I could have simply sufficed with:

nums.toMap     //> res6: scala.collection.immutable.Map[Char,Int] = Map(a -> 2, b -> 2)

*sighs*

Examples of a HashSet

Introduction

Some Real Life Examples where we use a Hash and Buckets.

Let me first shortly introduce how a hash works by means of the first example, an AddressBook, and as a followup how it is implemented in Java's HashSet.

Addressbook

A HashSet is a specific implementation of a Set. It uses the Hash of the stored objects to determine the position in an array of what are called Buckets. Every bucket in the array is, if filled, filled with another collection. So a HashSet is actually a collection of collections.

Let's see some examples of hashes people use when finding addresses in an AddressBook:

perfect hash: returns number in a numeral system with radix 26

in this case each name in the address book has a unique hashcode. Every list contains exactly one entry. If you use this as an address book, each page contains at most one address. You are going to need a huge amount of pages, if you have a lot of friends and relatives. With this behaviour it would make more sense to store addresses via a key-value store (for example an implementation of Map).

average hash: return first letter of the last name, capitalised

in this case, the behaviour is the same as your average normal address book. Each list contains all address of people whos names start with the same letter.

bad hash: return 'A'

in this case, the behaviour is terrible! You could compare it to an address book, on which all addresses are written on the first page. This way, the hashset will behave like a normal linked list.

Let's assume the hash used is the middle one. The Hashcode of entries in an AddressBook takes the first letter of the last name (capitalised).

Java's HashSet

We assume a class Address, containing the hashCode() method that returns the first letter of the last name.


The HashSet<Address> is implemented by using a HashMap<Address, Object> internally.

Now, the HashMap has an underlying data structure consisting of a Entry<Address, Object>[] table. This is the array that is going to contain all the first elements of each linked list. In other words, this is the Array containing the buckets.

The index of a bucket in the array is almost equivalent to the hashCode of the Address instance.

When I say almost, I mean that some additional conversions take place on the original hashCode, for instance:

• In the case of using Strings Sun (Oracle?) uses a special hashing function (sometimes)
• Some bit twiddling takes place to prevent collisions due to hashCodes that do not use lower bits, and therefore would cause the start of the bucket array to never be used.
• The result is "binary and"ed, ofcourse, with the table length (which is always a power of two) as a quick sort of modulo function to prevent ArrayIndexOutOfBounds problems.

For this reason, the buckets are not at the positions in the bucket array as you would expect, given the answers of the hashCode() function.

Every bucket contains the first Entry (Entry<Address, Object>) of a linked list.

A new entry is added to the top of the Linked list.

Where each Entry has a pointer to the next Entry in the bucket, or null if it is the only or last entry, effectively creating/ending the linked list.

So the contents should look like follows:

Supermarket

The Hashcode used in a supermarket is infinitely worse than my Addressbook. To start a search, accost an employee of said supermarket and inquire after the location of an article, say washing detergent. You'll receive the hashcode "aisle 14" or some such. When at the location of said hashbucket, you find all articles jumbled together. Your senses assailed by every colour swirl, bubble and geometric shape imaginable with letters and numbers in wild and disturbing images.

If you are lucky, the hashcode is maginally improved with additional information like "on the left", or "at the very end".

Conclusion

Apparently, in real life, there are HashSets in use as well. We can only come to the conclusion that any logical data structure that we use in real life without thinking, has it's corresponding implementation in the Software Realm.

I hope you've enjoyed this journey of discovery into the Java source code, as much as I did.

References

Why always override hashcode() if overriding equals()?

http://www.xyzws.com/javafaq/why-always-override-hashcode-if-overriding-equals/20

Java 6 HashMap JavaDoc

http://docs.oracle.com/javase/6/docs/api/java/util/HashMap.html

Java 6 HashSet JavaDoc

http://docs.oracle.com/javase/6/docs/api/java/util/HashSet.html

Java Generics and Collections

Maurice Naftalin & Philip Wadler

Vector Deprecated?

Someone asked me to explain a comment of mine where I mentioned that the Vector class isn't used (much) any more.

Back in the old days, when I was programming and I needed a collection (there were no generics in sight yet), I'd use the Vector class as a convenient way of having a list of objects to maintain. Vector, at the time, could be filled with instances of supertype Object and I got stuck with a lot of casting down to the appropriate type I wanted.

The Java class Vector has been a part of the JDK since version 1.0.

In J2SE 1.2 (JDK 1.2) the Collections framework was introduced.

The following javadoc comments can be found on the Vector class.

As of the Java 2 platform v1.2, this class was retrofitted to implement the List interface, making it a member of the Java Collections Framework. Unlike the new collection implementations, Vector is synchronized. If a thread-safe implementation is not needed, it is recommended to use ArrayList in place of Vector.^[1]

Also the following text is displayed in Netbeans if you do try to use the Vector class.^[4]

"This inspection reports any uses of java.util.Vector or java.util.Hashtable. While still supported, these classes were made obsolete by the JDK1.2 collection classes, and should probably not be used in new development."

Basically, what I'm saying is, that Vector class was implemented before the Collections framework. As such it does not follow the Collections framework naming scheme of List,Set and Map. It also has a number of methods, for backwards compatibility, that are not to be found in any of the other collection classes.

If you do need synchronization, the Vector is indeed synchronized, yet it is synchronized on each and every operation. This is fine, if you do not need to execute a lot of operations as an atomic transaction. But, for instance, if you need to iterate over all the items in the Vector, it is always better to get a lock before the iteration and release it after the iteration.

So, in short:

• use ArrayList, if you do not care for the synchronization aspect of Vector
• use Collections.synchronizedList^[2] if you need an array that is synchronized
• use CopyOnWriteArrayList if you are dealing with multiple concurrent read operations and only a few write operations ^*)
• use a native array, if you do not care for synchronization and performance is an issue.

*) be careful, as a write/add on a CopyOnWriteArrayList (as the name says) will create an entirely new Array populated with the old items and the new/added item. For large arrays this might be slow.

In the javadoc of synchronizedList there is a good example of iterating over it. I shall just write it down here for completeness.

List list = Collections.synchronizedList(new ArrayList());
      ...
synchronized (list) {
    Iterator i = list.iterator(); // Must be in synchronized block
    while (i.hasNext())
        foo(i.next());
}

As always in these things, the requirements you have, determine what kind of Collection you need to use.

Updates

23/10/2012: Updated because of the comment below on CopyOnWriteArrayList.

28/01/2013: Update: the same goes for HashTable (old) and HashMap (new).

References

[1] Vector

http://docs.oracle.com/javase/7/docs/api/java/util/Vector.html

[2] public   static  <T> List<T> synchronizedList(List<T> list)

http://docs.oracle.com/javase/7/docs/api/java/util/Collections.html#synchronizedList%28java.util.List%29

[3] Why is Java Vector class considered obsolete or deprecated?

http://stackoverflow.com/questions/1386275/why-is-java-vector-class-considered-obsolete-or-deprecated

[4] Is Vector an obsolete collection class?

http://www.coderanch.com/t/515352/java-developer-SCJD/certification/Vector-obsolete-collection-class

[5] Vector or ArrayList -- which is better?

http://www.javaworld.com/javaqa/2001-06/03-qa-0622-vector.html

Collections Empty List Factory Method

You can create an empty list like so:

List<String> s = Collections.emptyList();

However, the following doesn't work:

setStrings(Collections.emptyList());

Because the compiler cannot determine what the type should be at compile time. In the first example type inference comes to the rescue. In the second example, not soo much.

What, however, does work is the following syntax I came across:

setStrings(Collections.<String> emptyList());

It's a little odd at first, but looking at the source code of emptyList() reveals:

public static final <T> List<T> emptyList() {
    return (List<T>) EMPTY_LIST;
}

The <T> in front of List<T> indicates the type that is supposed to be inferred.

Please do not use EMPTY_LIST directly, if possible. It doesn't use generics.

References

[1] Java: Collections.emptyList() returns a List<Object>?

http://stackoverflow.com/questions/306713/java-collections-emptylist-returns-a-listobject