Zerubbabel vs the JVM

Systemutveckling Teknologi JVM Testing Functional Programming

Uzi Landsmann

Systemutvecklare

Many languages live in the JVM landscape. But how do we know which one to choose? In this article, we will look into a specific aspect of some JVM languages using quite a limited looking glass: comparing several functional operations made in each language to achieve the same goal, namely analyzing the bible for some words. Hopefully, the solutions will allow us to see some differences between the JVM languages and perhaps even introduce some of the features in those languages that you might not be familiar with.

 

The project

We’ve created a simple specification with some requirements and a testing template to be implemented in different languages:


https://github.com/WebstepSweden/Zerubbabel Länk till annan webbplats, öppnas i nytt fönster.

 

This is an attempt to solve a problem in different languages using a functional programming paradigm, in order to compare the different implementations.

The Problem: how many times does the word God appear in the Bible? And how about Jesus? Is there more love than hate in the bible? How about sheep vs goats? And most importantly, is there really a guy named Zerubbabel in the Bible? In this project, we intend to analyze the bible text and create a dictionary with words as keys and the number of appearances as values, in order to draw some conclusions about the questions above.


The subprojects: different implementations are to be added in folders with the specific language implementation inside them.


The text: a text file called bible.txt resides in the root folder. When implementing a subproject, the file should be copied to it and used by the implementation.


The words: all the words in the text file should be read into a list or an array, and transformed into lowercase.


The filters:

  • empty words are to be removed using the following regex: ^\w+
  • the following boring words should be removed: the, and, of, to, that, for, in, i, his, a, with, it, be, is, not, they, thou

The dictionary: all the words in the text should be grouped using the distinct words as keys and the number of places in which they appear as values. This dictionary is the return value of a function.


The tests: in order to test the implementation, the following tests need to be created:



We have then attempted to implement the solution in multiple JVM languages, described below.

 

Implementations

In this section, we discuss what the implementations look like in the different languages, what we got stuck on, and what we liked while writing the code. We summarize with a list of pros and cons.

 

Java

First of all, we’ve implemented a solution using the most popular language in the JVM, Java. We’re using the nio library Länk till annan webbplats.to comfortably fetch the text file, then use the stream library to create fluent code. We use multiple static imports to make the code even more readable (where it makes sense: statically importing List.of(), for example, would not help because of(1,2,3) does not make the code any clearer).

pros: fluent code, nio makes it easy to read strings from a file

cons: every lambda call requires a named variable.

 

package zer.ubba.bel;

import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;

import static java.lang.ClassLoader.getSystemResource;
import static java.util.Arrays.stream;
import static java.util.function.Function.identity;
import static java.util.stream.Collectors.counting;
import static java.util.stream.Collectors.groupingBy;

public class JavaBible {
    private final List<String> boringWords = List.of(("the,and,of,to,that,for,in,i,his,a,with,it,be,is,not,they,thou")
            .split(","));

    Map<String, Long> readTheBible() throws Exception {
        final var path = Paths.get(getSystemResource("bible.txt").toURI());

        return Files.lines(path)
                .flatMap(line -> stream(line.split(" ")))
                .filter(Pattern.compile("^\\w+").asMatchPredicate())
                .map(String::toLowerCase)
                .filter(word -> !boringWords.contains(word))
                .collect(groupingBy(identity(), counting()));
    }
}

 

Kotlin

Next up was Kotlin. The solution is quite elegant, though the .groupingBy { it }.eachCount() part is far from obvious and like much other stuff, you need to google it/stackoverflow it to know it. We use the automatic it variable in the lambdas which makes the functions slightly more compact.

pros: compact code

cons: grouping/counting is non-trivial

 

package zer.ubba.bel

import java.io.File
import java.lang.ClassLoader.getSystemResource
import java.util.Locale

private val boringWords = ("the,and,of,to,that,for,in,i,his,a,with,it,be,is,not,they,thou")
    .split(",")

fun readTheBible(): Map<String, Int> {
    val file = File(getSystemResource("bible.txt").file)

    return file
        .readLines()
        .flatMap { it.split(" ") }
        .filter { """^\w+""".toRegex().matches(it) }
        .map { it.lowercase(Locale.getDefault()) }
        .filter { !boringWords.contains(it) }
        .groupingBy { it }.eachCount()
}

 

Scala

The Scala solution is quite similar to the Kotlin one, but here we use _ as the automatic variable in the lambdas. A quirk we found was that the filter() function returns an iterator, for which the groupBy() method is not defined, so we had to convert it into a list before grouping the words. The mapValues() method was quite a straightforward way to map the list of words into the size of the list.

pros: compact code, fromResource() function makes life easy

cons: need to convert to list before grouping

 

package zer.ubba.bel

import scala.io.Source.fromResource

class ScalaBible {
  private val boringWords = ("the,and,of,to,that,for,in,i,his,a,with,it,be,is,not,they,thou")
    .split(",")

  def readTheBible(): Map[String, Int] = {
    val source = fromResource("bible.txt")

    source
      .getLines
      .flatMap(_ split " ")
      .filter(_ matches """^\w+""")
      .map(_.toLowerCase)
      .filter(!boringWords.contains(_))
      .toList
      .groupBy(identity)
      .mapValues(_.size)
  }
}

 

Groovy

In Groovy, the standard lambda names have been replaced with other, less straightforward ones, like collect() instead of map() and findAll() instead of filter() — which takes some getting used to. Also, we’re missing the flatMap() function and had to use flatten() instead, which unfortunately returns a List<?> so we had to m̶a̶p̶ collect it back to a list of strings before continuing. An extra bonus was though the countBy() method that did exactly what we needed. Why don’t other languages have this feature?

pros: the countBy() method is great

cons: non-standard lambda method names, flatMap() is missing

 

package zer.ubba.bel

class GroovyBible {
    private final def boringWords = "the,and,of,to,that,for,in,i,his,a,with,it,be,is,not,they,thou"
            .split(",");

    def readTheBible() {
        def resource = ClassLoader.getSystemResource("bible.txt")

        return resource
                .readLines()
                .collect { it.split(" ") }
                .flatten()
                .collect { it.toString() }
                .findAll { it.matches("^\\w+") }
                .collect { it.toLowerCase() }
                .findAll { !boringWords.contains(it) }
                .countBy { it }
    }
}

 

Clojure

Clojure is…different. Also, it uses its own lambda names which takes some googling to find the correct one — so slurp mean read file and mapcat means flatMap. The most confusing part was trying to find a way to filter out the boring words. Apparently, contains is one of the most confusing functions in Clojure Länk till annan webbplats., and we had to reinvent the wheel a bit and fiddle with some and some? to find a solution (probably not the best one) to remove the boring words. Also, we’ve used multiple def variables to make the code more readable, but we’re guessing more competent Clojure developers would probably not need those and would write a much more compact and beautiful code.

 

pros: the frequencies method is great, (potentially) very compact code

 

cons: non-standard lambda method names, filtering boring words was non-trivial.

 

(ns zer.ubba.bel.clojure_bible
  (:require [clojure.java.io :as io])
  (:require [clojure.string :as str])
  (:gen-class))

(def data-file (io/resource "bible.txt"))

(def boringWords
  (str/split "the,and,of,to,that,for,in,i,his,a,with,it,be,is,not,they,thou" #","))

(defn readTheBible []
  (def text (slurp data-file))
  (def lines (str/split text, #"\n"))
  (def words (mapcat #(str/split % #" ") lines))
  (def noempty (filter #(re-matches #"^\w+" %) words))
  (def lower (map #(str/lower-case %) noempty))
  (defn isboring [word] (some? (some #(= word %) boringWords)))
  (def noboring (remove #(isboring %) lower))
  (frequencies noboring))

 

Tests

Though there are many test framework alternatives, we tried using the most typical one for each language, give an example of some tests and describe them below.

 

Java

The Java tests are written using JUnit 5 Länk till annan webbplats.. They are simple and straightforward, although the test method names are compressed and are a bit hard to read. Some people also consider the parameter order in the assertion methods confusing (which one is expected and which is the actual result?), and assertion frameworks like AssertJ Länk till annan webbplats. and Hamcrest Länk till annan webbplats. are used frequently instead of the JUnit ones or in addition to them.

 

pros: simple and easy

cons: test method names are somewhat hard to read, assertions are a bit confusing

 

package zer.ubba.bel;

import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;

import java.util.Map;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;

class JavaBibleTest {

    private static Map<String, Long> bibleDictionary;

    @BeforeAll
    static void setUpBeforeClass() throws Exception {
        bibleDictionary = new JavaBible().readTheBible();
    }

    @Test
    void theBibleShouldContainManyDifferentWords() {
        assertEquals(9582, bibleDictionary.size());
    }

    @Test
    void godShouldAppearALotOfTimes() {
        assertEquals(2302, bibleDictionary.get("god"));
    }

    // more tests...
}

 

The error message we receive when the test fails is quite clear:

org.opentest4j.AssertionFailedError:
Expected :2302
Actual :2303
<Click to see difference>
at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
at org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:635)
at zer.ubba.bel.JavaBibleTest.godShouldAppearALotOfTimes(JavaBibleTest.java:27)
...more stack lines...


Kotlin

We’re using Kotest Länk till annan webbplats. to test Kotlin. The test method names are given as strings and are easy to read. The infix Länk till annan webbplats. assertions, like shouldBe, make the tests very clear as well. The test class declaration, including the StringSpec thing, is a bit special with that combination of parentheses and curly brackets but is not too annoying.

pros: readable test method names and assertions


cons: strange class declaration

 

import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.comparables.shouldBeGreaterThan
import io.kotest.matchers.shouldBe
import zer.ubba.bel.readTheBible

class KotlinBibleTest : StringSpec({

    val bibleDictionary = readTheBible();

    "The Bible should contain many different words" {
        bibleDictionary.size shouldBe 9582
    }

    "God should appear a lot of times" {
        bibleDictionary["god"] shouldBe 2303
    }

    // more tests...
})

 

Even here, the error message we receive when the test fails is quite clear:

expected:<2302> but was:<2303>
Expected :2302
Actual :2303
<Click to see difference>
io.kotest.assertions.AssertionFailedError: expected:<2302> but was:<2303>
at KotlinBibleTest$1$2.invokeSuspend(KotlinBibleTest.kt:15)
at KotlinBibleTest$1$2.invoke(KotlinBibleTest.kt)
...more stack lines...

Scala

For Scala, we’re using the ScalaTest framework Länk till annan webbplats., though some people prefer using Specs2 Länk till annan webbplats.. It is straightforward and easy to write tests using it and has a lot of nice matchers like should be > and shouldNot contain — although we had to choose carefully when trying to find the correct matcher to import.

pros: readable test method names and assertions, rich matcher library with infix matchers

cons: finding the correct matcher to import can be a challenge

 

import org.scalatest.funsuite.AnyFunSuite
import zer.ubba.bel.ScalaBible

class ScalaBibleTest extends AnyFunSuite {

  private val bibleDictionary = new ScalaBible().readTheBible();

  test("The Bible should contain many different words") {
    bibleDictionary.size shouldBe 9582
  }

  test("God should appear a lot of times") {
    bibleDictionary("god") shouldBe 2303
  }

  // more tests...
}

Even Scalatest shows a clear error message when the test fails:

2303 was not equal to 2302
ScalaTestFailureLocation: ScalaBibleTest at (ScalaBibleTest.scala:16)
Expected :2302
Actual :2303
<Click to see difference>
org.scalatest.exceptions.TestFailedException: 2303 was not equal to 2302
at org.scalatest.matchers.MatchersHelper$.indicateFailure(MatchersHelper.scala:392)
at org.scalatest.matchers.should.Matchers$AnyShouldWrapper.shouldBe(Matchers.scala:7021)

 

Groovy

Spock Länk till annan webbplats. is probably the most popular testing framework for Groovy. The tests are written in a BDD manner using different blocks like given, when and then or, as in our case, simply expect. This gives the tests a nice structure and separates the different parts. We were a bit confused when it came to the dictionary instance that is used by all the tests, and it took some googling to realize that we need to use the @Shared annotation for it.

pros: well structured BDD-like tests using test blocks


cons: shared variables are a bit confusing at first

 

package zer.ubba.bel

import spock.lang.Shared
import spock.lang.Specification

class GroovyBibleTest extends Specification {
    @Shared
    def bibleDictionary = new GroovyBible().readTheBible()

    def "The Bible should contain many different words"() {
        expect:
        bibleDictionary.size() == 9582
    }

    def "God should appear a lot of times"() {
        expect:
        bibleDictionary["god"] == 2303
    }
    
    // more tests...
}

 

We find the error message incredibly clear and useful, pointing us to the exact location where the error occurs:

 

Condition not satisfied:
bibleDictionary[“god”] == 2302
|              |       |
|              2303    false
[old:264, testament:8, king:1793,....]

 

Clojure

clojure.test Länk till annan webbplats. is the default test framework used for testing Clojure, using the is macro Länk till annan webbplats. for assertions. We import it and our clojure_bible file using :require and can then write our tests in a quite readable, clojure manner.

pros: clear test names, the is macro is quite straightforward to use

cons: no matchers are used. Instead, we’re using operators like =, <, > etc.

 

(ns zer.ubba.bel.clojure_bible_test
  (:require [clojure.test :refer :all]
            [zer.ubba.bel.clojure_bible :refer :all]))

(deftest bible-test

  (def bibleDictionary (readTheBible))

  (testing "The Bible should contain many different words"
           (is (= (.size bibleDictionary) 9582)))

  (testing "God should appear a lot of times"
           (is (= (get bibleDictionary "god") 2303)))

  ; more tests...       
  

 

The error message we receive is rather clear as well:

FAIL in (bible-test) (clojure_bible_test.clj:13)
God should appear a lot of times
expected: (= (get bibleDictionary “god”) 2302)
actual: (not (= 2303 2302))

Final words

We know that there are more JVM languages but chose to compare only the high-profile languages Länk till annan webbplats.. This comparison was not in any way done in an academic way and is highly subjective. Also, as described above, we only wanted to highlight a small aspect of the languages — a few functional operations — but we hope we managed to show how a similar goal can be achieved in each of the different languages and to provide a glimpse into those and to the way we use different test frameworks to test them.

 

Finally, the project is open to adding more programming languages than the ones living in the JVM. If you wish to add an implementation in your favorite language, you’re more than welcome to add a pull request in the Zerubbabel project Länk till annan webbplats..

Fler inspirerande inlägg du inte får missa