Category: Java

From Java Builders to Kotlin DSLs

From Java Builders to Kotlin DSLs

From Java Builders to Kotlin DSLs

Introduction

DSLs – Domain Specific Languages – are an ever trending topic in Kotlin circles. They allow us to flex some of the most exciting language features while accomplishing more readable and maintainable solutions in our code.

Today I’d like to show you how to implement a certain kind of DSL – we’re going to be wrapping an existing Java Builder in Kotlin. No doubt you’ve come across the builder pattern in Java before, for example if you’re an Android developer, you must’ve used an AlertDialog.Builder, an OkHttpClient.Builder, or a Retrofit.Builder at some point. Wrapping a builder like this is a good exercise in just pure DSL design. All you have to worry about is designing the API you provide with your wrapper, since all the implementation for whatever your DSL provides its users is already done inside the builder!

Our example case

It just so happens that I’m the creator and maintainer of a library that does this very thing, and a small part of its implementation is what we’re going to be using as our example. The original library is the wonderful and hugely popular MaterialDrawer by Mike Penz, which allows you to create complex, good looking, and customized navigation drawers in your application, all via various Builder objects in Java, with no XML writing involved on your part. This is what my library, MaterialDrawerKt provides a convenient Kotlin DSL wrapper for.

The Builder API

Let’s take a look at the drawer we’re going to be creating in our example.

android_drawer

Here’s the code, using the builders in the original API:

DrawerBuilder()
        .withActivity(this)
        .withTranslucentStatusBar(false)
        .withDrawerLayout(R.layout.material_drawer_fits_not)
        .addDrawerItems(
                PrimaryDrawerItem()
                        .withName("Home")
                        .withDescription("Get started here!"),
                PrimaryDrawerItem()
                        .withName("Settings")
                        .withDescription("Tinker around")
        )
        .withOnDrawerItemClickListener { view, position, drawerItem ->
            when(position) {
                0 -> toast("Home clicked")
                1 -> toast("Settings clicked")
            }
            true
        }
        .withOnDrawerListener(object: Drawer.OnDrawerListener {
            override fun onDrawerSlide(drawerView: View?, slideOffset: Float) {
                // Empty
            }

            override fun onDrawerClosed(drawerView: View?) {
                toast("Drawer closed")
            }

            override fun onDrawerOpened(drawerView: View?) {
                toast("Drawer opened")
            }
        })
        .build()

In this code, we’ve…
– set the Activity we want the drawer to appear in,
– made some layout adjustments so that the drawer is below the ActionBar,
– created just two menu items with names and descriptions,
– set a listener where we can handle item selections by position, using a SAM conversion to implement a single method interface,
– added a listener to detect drawer movement, using an object expression, because the necessary interface has multiple methods.

We saw two different builders used here, namely, the DrawerBuilder and PrimaryDrawerItem classes. While their builder syntax actually looks pretty decent and readable, we’ll see that a DSL can do even better.

Note that you can check out the entire working demo project for this article on GitHub here. See the commit history to follow the article step by step as we build our DSL.

Creating instances

Let’s start small. We’ll want to create a drawer with a drawer {} call, let’s implement just that.

fun Activity.drawer(dummy: () -> Unit) {
    DrawerBuilder()
            .withActivity(this)
            .build()
}

We’ve defined our first function as an extension on Activity, so that it’s available when we’re in one, and it can access the Activity instance as this without us having to pass it in explicitly.

Now, we should add our PrimaryDrawerItem instances. This syntax would be nice for a start:

drawer {
    primaryItem()
    primaryItem()
}

To get this, we’ll need a primaryItem function that’s only available within the drawer block, and that somehow adds an item to the DrawerBuilder we’ve already created.

To be able to call methods on our DrawerBuilder instance before we build() it, we’ll introduce a new wrapper class that holds it:

class DrawerBuilderKt(activity: Activity) {
    val builder = DrawerBuilder().withActivity(activity)

    internal fun build() {
        builder.build()
    }
}

We’ll update our original drawer function to create an instance of this wrapper class. We’ll also modify its parameter – by making the setup function an extension on our own class, the client of the DSL will be placed in a new scope inside the lambda passed to the drawer function, where the methods of DrawerBuilderKt become available, as we’ll see in a moment.

fun Activity.drawer(setup: DrawerBuilderKt.() -> Unit) {
    val builder = DrawerBuilderKt(this)
    builder.setup()
    builder.build()
}

You might have spotted that we’ve marked our own build method internal – this is because the only call to it will be the one inside the drawer function. Hence, there’s no need to expose it to the clients of our library.

The visibility of builder is another story – we’ll have to keep this public so that our library stays extensible. We could make it internal for the purpose of us implementing wrappers around the built-in drawer items, but that would mean that nobody else could add their own custom drawer items to the DSL – something you could do with the original library. This is functionality we don’t want to strip from our clients.

Now we can finally add the primaryItem function we were planning earlier. To make this available inside the lambda passed to drawer, we could make it a member of the DrawerBuilderKt class. Modifying this class for every new drawer item type we add, however, seems like an odd thing to do. The various types of drawer items should in no way affect how the drawer itself works.

We can instead use the same mechanics as clients would use to create custom drawer items. We’ll get a neat, decoupled design by adding primaryItem as an extension function:

fun DrawerBuilderKt.primaryItem() {
    builder.addDrawerItems(PrimaryDrawerItem())
}

We can now add blank drawer items to our drawer!

Setting properties

Before we get to setting the name and description of our drawer items with the DSL, we have the properties of DrawerBuilder to take care of, as we’ve seen in the very first code snippet. This is the syntax we’ll create for these:

drawer {
    drawerLayout = R.layout.material_drawer_fits_not 
    translucentStatusBar = false
}

These, of course, will be properties on the DrawerBuilderKt class so that they’re available in the right scope.

Since the original builder doesn’t let us access the values we’ve set, what we’ll need to create are essentially write-only properties. These properties won’t have backing fields to store values, all they’ll do is forward the calls to the appropriate builder methods.

Unfortunately, Kotlin only has properties that can be both read and written (var) and read-only ones (val). We’ll solve this by using a var, and throwing an exception when someone tries to read these properties. We’ll also include Kotlin’s powerful @Deprecated annotation that lets us mark using the getters an error, so that clients are stopped from doing so at edit/compile time, rather than just getting the runtime exception:

class DrawerBuilderKt(activity: Activity) {
    ...

    var drawerLayout: Int
        @Deprecated(message = "Non readable property.", level = DeprecationLevel.ERROR)
        get() = throw UnsupportedOperationException("")
        set(value) {
            builder.withDrawerLayout(value)
        }
}

Alternatives for setting properties

Now, we can move on to customizing the drawer items themselves. The obvious solution here is to continue with the same syntax style as before:

drawer {
    primaryItem {
        name = "Home"
        description = "Get started here!"
    }
}

We know how to do this by creating a wrapper class around PrimaryDrawerItem, and then adding a couple non-readable properties, just like we did before. But let’s do something more interesting, and create this alternative syntax:

drawer {
    primaryItem(name = "Home", description = "Get started here!")
}

This is also pretty straightforward, we’re just calling a function that has two parameters, and using named parameters for readability. Let’s add these parameters to primaryItem then. We’ll also throw in default values so that they’re each optional:

fun DrawerBuilderKt.primaryItem(name: String = "", description: String = "") {
    val item = PrimaryDrawerItem()
            .withName(name)
            .withDescription(description)
    builder.addDrawerItems(item)
}

This, of course, isn’t a feasible method for adding dozens of properties to an item we’re constructing with our DSL, adding a wrapper around the PrimaryItemClass with separate write-only properties that can be set in a setup lambda is still the way to go for most things.

However, this is a nice way to lift some very basic or commonly set properties to a more prominent position in the code. Here’s what the DSL could look like with some more properties implemented:

primaryItem(name = "Games", description = "Ready, player one?") {
    iicon = FontAwesome.Icon.faw_gamepad
    identifier = 3
    selectable = false
}

Listeners

We know how to add properties to our DSL, now let’s see how we can go about listeners. We’ll start with the easy one, setting the OnDrawerItemClickListener to handle item clicks for a given position in the drawer. Here’s our goal:

drawer {
    onItemClick { position ->
        when (position) {
            0 -> toast("Home clicked")
            1 -> toast("Settings clicked")
        }
        true
    }
}

onItemClick will be a method in DrawerBuilderKt, and it will take a lambda parameter that can be called when the original listener fires:

class DrawerBuilderKt(activity: Activity) {
    ...

    fun onItemClick(handler: (view: View?, position: Int, drawerItem: IDrawerItem<*, *>) -> Boolean) {
        builder.withOnDrawerItemClickListener(handler)
    }
}

We’re making use of SAM conversion with the call to the withOnDrawerItemClickListener method of the builder here. The usual SAM conversion syntax would have us passing in a lambda that gets transformed to the OnDrawerItemClickListener interface, but instead, we’re going just a small step further, and we’re passing in the handler parameter which has the appropriate function type for a conversion.

We’ll simplify the above method a bit, by taking a lambda which only gets the position passed to it, as clients will usually only care about that parameter. We’re using SAM conversion again, this time with a regular lambda, because we want to ignore some parameters when calling our simpler handler parameter.

class DrawerBuilderKt(activity: Activity) {
    ...

    fun onItemClick(handler: (position: Int) -> Boolean) {
        builder.withOnDrawerItemClickListener { _, position, _ -> handler(position) }
    }
}

Of course, having only this version of the method would hide functionality of the original library, so in the real wrapper library, I’ve included both of these.

Complex listeners

Last but not least, let’s see what we can do about the OnDrawerListener in our example. This interface has three methods, so the previous, simple solution won’t work here. As always, let’s start with the syntax we want to achieve. Not that we’re only setting two of the three methods that the interface defines.

drawer {
    onClosed {
        toast("Drawer closed")
    }
    onOpened {
        toast("Drawer opened")
    }  
}

As you can see, it would be nice to be able to specify either one or multiple of the methods of the interface independently of each other. We know that we’ll want to define three methods to take the appropriate handler lambdas, very similarly to what we did before:

class DrawerBuilderKt(activity: Activity) {
    ...

    fun onOpened(handler: (drawerView: View) -> Unit) {
        // TODO implement
    }

    fun onClosed(handler: (drawerView: View) -> Unit) {
        // TODO implement
    }

    fun onSlide(handler: (drawerView: View, slideOffset: Float) -> Unit) {
        // TODO implement
    }
}

The question is how to pass these handlers to the builder we’re holding. We can’t make a withOnDrawerListener call in each of them and create an object that wraps just that one handler, as the object created there would always implement just one of the three methods.

What I came up with for this is an object property in our DrawerBuilderKt wrapper class that implements the OnDrawerListener interface, and delegates each of these calls to one of its properties.

class DrawerBuilderKt(activity: Activity) {
    ...

    private val onDrawerListener = object : Drawer.OnDrawerListener {
        var onSlide: ((View, Float) -> Unit)? = null

        override fun onDrawerSlide(drawerView: View, slideOffset: Float) {
            onSlide?.invoke(drawerView, slideOffset)
        }

        var onClosed: ((View) -> Unit)? = null

        override fun onDrawerClosed(drawerView: View) {
            onClosed?.invoke(drawerView)
        }

        var onOpened: ((View) -> Unit)? = null

        override fun onDrawerOpened(drawerView: View) {
            onOpened?.invoke(drawerView)
        }
    }
}

All our previous methods have to do then is to set these properties when they’re called:

class DrawerBuilderKt(activity: Activity) {
    ...

    fun onOpened(handler: (drawerView: View) -> Unit) {
        onDrawerListener.onOpened = handler
    }

    fun onClosed(handler: (drawerView: View) -> Unit) {
        onDrawerListener.onClosed = handler
    }

    fun onSlide(handler: (drawerView: View, slideOffset: Float) -> Unit) {
        onDrawerListener.onSlide = handler
    }
}

Of course, we’ll still have to pass this object to the builder at some point – we’ll do that after the setup lambda passed to drawer has already done its work, and it calls our DrawerBuidlerKt.build() method:

class DrawerBuilderKt(activity: Activity) {
    val builder = DrawerBuilder().withActivity(activity)

    internal fun build() {
        builder.withOnDrawerListener(onDrawerListener)
        builder.build()
    }

    ...
}

Result

As a reminder, here’s the builder code we started the article with:

DrawerBuilder()
        .withActivity(this)
        .withTranslucentStatusBar(false)
        .withDrawerLayout(R.layout.material_drawer_fits_not)
        .addDrawerItems(
                PrimaryDrawerItem()
                        .withName("Home")
                        .withDescription("Get started here!"),
                PrimaryDrawerItem()
                        .withName("Settings")
                        .withDescription("Tinker around")
        )
        .withOnDrawerItemClickListener { view, position, drawerItem ->
            when(position) {
                0 -> toast("Home clicked")
                1 -> toast("Settings clicked")
            }
            true
        }
        .withOnDrawerListener(object: Drawer.OnDrawerListener {
            override fun onDrawerSlide(drawerView: View?, slideOffset: Float) {
                // Empty
            }

            override fun onDrawerClosed(drawerView: View?) {
                toast("Drawer closed")
            }

            override fun onDrawerOpened(drawerView: View?) {
                toast("Drawer opened")
            }
        })
        .build()

And then here’s the final DSL we’ve put together:

drawer {
    drawerLayout = R.layout.material_drawer_fits_not
    translucentStatusBar = false

    primaryItem(name = "Home", description = "Get started here!")
    primaryItem(name = "Settings", description = "Tinker around")

    onItemClick { position ->
        when (position) {
            0 -> toast("Home clicked")
            1 -> toast("Settings clicked")
        }
        true
    }
    onClosed {
        toast("Drawer closed")
    }
    onOpened {
        toast("Drawer opened")
    }
}

I hope this serves as a good example of how much better looking and more expressive a DSL can be than builder style solutions. Another benefit is how much easier making changes in the DSL is, which of course becomes apparent when you’re actually making them (but worrying about placing commas in the right places, for example, is in the past).

Conclusion

Thank you for sticking with me for this entire journey. I hope you got a good feel for how to design your own DSLs (always write down your desired syntax first!), and how to structure your DSL implementations.

If you’re interested in more, I recommend checking out the source of the actual MaterialDrawerKt library, as it deals with many more advanced concepts that I couldn’t fit in this post. Among other things, it uses generics and inheritance heavily to deal with MaterialDrawer‘s complex hierarchy of built-in drawer item types, restricts function calls and property accesses to the appropriate levels of the DSL hierarchy, and much more. I might cover these in a future article.

Additionally, you can find another article covering the process of writing a DSL here, and another article about more general DSL design here.

But this is it for now, and with that, it’s now your time to go and start creating your own DSLs. Good luck!

Márton is a Kotlin enthusiast from Hungary, currently working as an Android developer at AutSoft, while still pursuing a master’s degree in computer engineering. Brags a lot about being ranked third in the top users list of the Kotlin tag on Stack Overflow.

Please follow and like this Blog 🙂
Kotlin Nullability Features

Kotlin Nullability Features

Null-Safe Programming – The Kotlin Way

Disclaimer: This ktor article was originally published in the Dzone Java Guide 2018, which can be downloaded here.

In this article, we will review the problems that may be caused by null pointers and how to avoid them in Java. After that, the article demonstrates how Kotlin nullability features work and how they improve your code.

As Java developers, we’re very accustomed to NullPointerExceptions (NPE) that are thrown at the runtime of an application. This almost always happens unintentionally in consequence of a bug, which is based on unrecognized references to null. The null reference is often used to indicate absent values, which isn’t obvious to the programmer in many cases. Although Java relies on strong static typing, it doesn’t let you distinguish between reference types that can and cannot hold a null reference. Have a look at the following code example:

Device audio = new DeviceProvider().getAudioDevice();
String audioName = audio.getName();

The method getAudioDevice returns an object of type Device but might return null in order to denote the absence of that device on particular systems. Well documented methods will describe exactly that behavior, which still requires the developer to be very attentive. Not knowing about the possibility of a returned null reference is going to cause an awful NullPointerException in the subsequent call to getName. Wouldn’t it actually be nice if we were able to identify the method’s returned Device type as nullable (or non-nullable respectively) firsthand?

Null-Safety in Java

We have to find strategies that help us avoiding unwanted bugs due to NPEs. A common approach is to defensively check for null references and handle these cases in a way that makes more sense, such as providing default values or throwing more meaningful exceptions. Applying this to the previous example brings us to the following solution:

Device audio = new DeviceProvider().getAudioDevice();
String audioName;
if (audio != null) {
    audioName = audio.getName();
} else {
    throw new IllegalStateException("This system does not provide an audio device.");
}

Constructs like these are part of any Java project and the approach works well unless you forget to add checks for certain variables. It’s a fairly error-prone approach and doesn’t mitigate the fact that using null as a representative for absent things is risky.

Does Java offer any more sophisticated solutions to this problem? It does, at least in some situations. The Java SE 8 introduced the Optional type that acts as a “container object which may or may not contain a non-null value”. This sounds very promising and needs to be considered next.

Java SE 8 Optional

The Optional type is a container that wraps another object, which can theoretically be null. Code that works on instances of Optional needs to handle the possible nullability in a rather explicit way:

Optional audio = new DeviceProvider().getAudioDevice();
String audioName = audio
    .flatMap(Device::getName)
    .orElseThrow(() -> new IllegalStateException("This system does not provide an audio device."));

The getAudioDevice method was adjusted to return an Optional, which doubtlessly indicates to the client that the device can be absent. Instead of carelessly accessing that nullable type, the Optional type can be used for handling the possible nullability in various ways. These include providing default values and throwing exceptions with the help of simple methods like orElse and orElseThrow respectively. Furthermore, it literally forces the client to think about the potential null case.

Unfortunately, the whole Optional story already ends with this use case very suddenly. As stated by Java language architect Brian Goetz in this StackOverflow post, the Optional type was not intended as a “general purpose Maybe […] type” but a way to let libraries and APIs express the absence of a return type (as we saw in the previous example).

For further usage examples of Optional, you should find a suitable article in your preferred search engine.

After all, the Optional type is a great way to provide more explicit APIs that let the corresponding callers know exactly when null handling is required just by observing the method’s signature. Nevertheless, it’s not a holistic solution since it isn’t meant to replace each and every null reference in the source code. Aside from that, can you safely rely on method return types, which are not marked as Optional?

Kotlin Nullability: Null-Safety in Kotlin

After we have seen the rather unsafe null handling in the Java language, this section will introduce an alternative approach: The Kotlin programming language, as an example, provides very sophisticated means for avoiding NullPointerExceptions.

The language’s type system differentiates between nullable and non-nullable types and every class can be used in both versions. By default, a reference of type String cannot hold null, whereas String? allows it. This distinction on its own doesn’t make a very big difference obviously. Therefore, whenever you choose to work with nullable types, the compiler forces you to handle possible issues, i.e. potential NPEs, appropriately.

//declare a variable with nullable String type, it's OK to assign `null` to it
var b: String? = "possiblyNull"

// 1. does not compile, could throw NPE
val len = b.length

// 2. Check nullability before access
if (b != null){
    b.length
}

// 3. Use safe operator
val len = b?.length

This code example shows different ways of working with nullable types (String? in this case). As demonstrated first, it’s not possible to access members of nullable types directly since this would lead to the same problems as in Java. Instead, traditionally checking whether that type is not null makes a difference. This action persuades the compiler to accept invocations on the variable (inside the if-block) as if it were not nullable. Note that this does not work with mutable vars because they could possibly be set to null from another thread between the check and first action in the block.
As an alternative to explicit checks, the safe call operator ?. can be used. The expression b?.length can be translated to “call length on b if b is not null, otherwise return null“. The return type of this expression is of type Int? because it may result in null. Chaining such calls is possible and very useful because it lets you safely omit a great number of explicit checks and makes the code much more readable:

person?.address?.city ?: throw IllegalStateException("No city associated to person.")

Another very useful operator is the elvis operator ?: that perfectly complements the safe call operator for handling else cases. If the left-hand expression is not null, the elvis operator returns it, otherwise, the right-hand expression will be called.

The last operator that you need to know is called not-null assertion operator !!. It converts any reference to a non-null type. This unchecked conversion may cause an NPE if that reference is null after all. The not-null assertion operator should only be used with care:

person!!.address!!.city //NPE will be thrown if person or address is null

In addition to the shown operators, the Kotlin library provides plenty of helpful functions like String?::IsNullOrEmpty(), String::toDoubleOrNull() and List::filterNotNull(), to name just a few. All of them support the developer in proper nullability handling and make NPEs almost impossible.

Interop between both languages

One of the key attributes of Kotlin is its fantastic interoperability with Java source code. You can easily mix both languages in a project and call Kotlin from Java and vice versa. How does that work in the case of null safety, though?

As learned earlier in this article, every Java reference can be null, which makes it hard for Kotlin to apply its safety principles to them meaningfully. A type coming from Java is called platform type, denoted with an exclamation mark, e.g. String!. For these platform types, the compiler isn’t able to determine whether it’s nullable or not due to missing information. As a developer, when a platform type is e.g. assigned to a variable, the correct type can be set explicitly. If you assign a platform type String! to a variable of the non-nullable type String, the compiler allows it and, as a consequence, safe access to that variable isn’t being enforced. Nevertheless, if that decision turns out to be wrong, i.e. a null reference is returned, NPEs will be thrown at runtime. Fortunately, there’s a solution that allows providing more information to the Kotlin compiler by applying certain annotations to the corresponding Java methods. This enables the compiler to determine actual nullability information and makes platform types unneeded.

Bottom Line

It’s important to understand that Kotlin does not try to avoid null references as such but raises the attention for null-related issues and especially NPEs enormously. If you take care of platform types and defensively decide to use them as nullable ones or apply the mentioned annotations to your Java libraries, you should be safe. NullPointerExceptions will be a thing of the past. As set out above, the Kotlin language provides many useful operators and other functions that simplify working with nullable types tremendously by making use of its clever type system. We can hope to see similar solutions in future Java versions soon.

Simon is a software engineer based in Germany with 7 years of experience writing code for the JVM and also with JavaScript. He’s very passionate about learning new things as often as possible and a self-appointed Kotlin enthusiast.

Please follow and like this Blog 🙂
Run Kotlin Scripts (kts) from regular Kotlin Programs

Run Kotlin Scripts (kts) from regular Kotlin Programs

Run Kotlin Scripts from Kotlin Programs

This article presents a way to run Kotlin scripts from Kotlin programs in order to leverage the power of DSLs.

Kotlin can be used as a scripting language. Simply write top-level executable code inside a file with .kts extension and run it with the kotlinc as described in the documentation. That’s also the format of Gradle build files that are used in combination with the Gradle Kotlin DSL like this gradle.build.kts. Gradle shows a fantastic example of a domain specific language that can be written standalone in .kts files to be read by the gradle tool later on. When we try to find a way to do the same with custom DSLs (Tutorial can be found here), we first need to know how to run Kotlin scripts from Kotlin programs. The article reveals how to do so.

The Java Scripting API (JSR-223)

The Java Scripting API is a tool for using scripting engines (such as Nashorn) from Java code. It enables users to write customizable scripting code that can be picked up by the Java application at runtime. In a way, the API is a neat way of writing extensible applications.

As of Kotlin 1.1, the corresponding JSR-223 is supported for Kotlin Scripts, too. That means that it’s possible to run Kotlin scripts from regular Kotlin programs in order to make applications customizable through these scripts.

Using the Kotlin Script Engine

In order to use the mentioned Kotlin script engine, a file called javax.script.ScriptEngineFactory has to be placed inside META-INF/services of your application. It should contain the following entry: org.jetbrains.kotlin.script.jsr223.KotlinJsr223JvmLocalScriptEngineFactory.
After that, the javax.script.ScriptEngineManager will be able to find the corresponding engine when looked up via ScriptEngineManager().getEngineByExtension("kts"). This code now finds the Kotlin ScriptEngine implementation, an instance that can be used to evaluate String-based scripts such as "5 + 2", or directly read scripts from the file system. Here’s a short example:

with(ScriptEngineManager().getEngineByExtension("kts")) {
    eval("val x = 3")
    val res2 = eval("x + 2")
    assertEquals(5, res2)
}

You could also compile scripts and evaluate them later:

val script = compile("""listOf(1,2,3).joinToString(":")""")
assertEquals(listOf(1, 2, 3).joinToString(":"), script.eval())

Wrapping the glue code in a library

As shown, executing Kotlin scripts from Kotlin programs is pretty easy due to the Java scripting API implementation for Kotlin. Nevertheless, since it’s a bit cumbersome to integrate the support into an application, I wrote a tiny library that encapsulates the Scripting API glue code. It is called KtsRunner and can be found on GitHub.

The KtsRunner is a lightweight tool for executing Kotlin scripts from your custom applications. The API, as of the very first version, provides a slim KtsObjectLoader class whose usage is shown in the following example:

data class ClassFromScript(val x: String)
import de.swirtz.ktsobjectloader.ClassFromScript

ClassFromScript("I was created in kts")

The previous snippets show the definition of some arbitrary data class and the code that instantiates an object of it. The object instantiation is basically what we write into a .kts file.

val scriptReader = Files.newBufferedReader(Paths.get("path/classDeclaration.kts"))
val loadedObj: ClassFromScript = KtsObjectLoader().load<ClassFromScript>(scriptReader)
assertEquals("I was created in kts", loadedObj.x)

Using the KtsObjectLoader makes it simple to load the correspoding object of ClassFromScript from the script file. Alternatively, the script could also be provided as a String:

val scriptContent = "5 + 10"
val result: Int = KtsObjectLoader().load<Int>(scriptContent))
assertEquals(15, result)

Adequate Usage Scenario

As mentioned in the beginning, it can make sense to make your application customizable through external scripts, similar to how Gradle can be extended with any custom build script. Imagine an application that provides a test suite runtime. The actual test cases are provided by technical testers who write their test scripts using a domain specific language that is provided by the main application. Since you don’t want testers to add source files (defining new test cases) to your application all the time, the test case creation is made in independent .kts files in which the DSL is utilized by the testing team. The test suite main application can use the presented KtsRunner library for loading the test cases provided in .kts files and process them further afterward.

An example

A pretty popular DSL for Kotlin is kotlinx.html, a language for describing type-safe HTML. You let the client of your application provide some arbitrary HTML that you want to render at a later time. The HTML DSL code is provided as .kts script files and might look like this:

import kotlinx.html.*
import kotlinx.html.dom.create
import org.w3c.dom.Element
import java.io.OutputStream
import java.io.OutputStreamWriter
import javax.xml.parsers.DocumentBuilderFactory

val document = DocumentBuilderFactory.newInstance().newDocumentBuilder().newDocument()
document.create.html {
    head {
        title("Hello world")
    }
    body {
        h1("h1Class") {
            style = "background-color:red"
            +"My header1"
        }
        p("pClass") {
            +"paragraph1"
        }
    }
}

When executed, an instance of org.w3c.dom.Element is created that contains the described HTML code in an XML document:

<?xml version="1.0" encoding="UTF-8"?><html>
    <head>
        <title>Hello world</title>
    </head>
    <body>
        <h1 class="h1Class" style="background-color:red">My header1</h1>
        <p class="pClass">paragraph1</p>
    </body>
</html>

That’s straightforward but the interesting part is that the script should actually be executed from the main program. For this purpose, we add the KtsRunner to the application by adding a repository and the dependency itself to the Gradle build file:

maven { 
    setUrl("https://dl.bintray.com/s1m0nw1/KtsRunner")
}
dependencies {
    //...
    compile("de.swirtz:ktsRunner:0.0.x")
}  

The final code for loading the Element from the external script looks as follows:

KtsObjectLoader().load<Element>(script)

Simple, isn’t it? Unfortunately, the shown Scripting API implementation for Kotlin is rather slow and you’ll definitely notice some performance constraints. Altogether, the KtsRunner is a very tiny tool that only encapsulates the glue code for enabling Kotlin Scripting support in random applications. The library is published on bintray and can therefore easily be used from your own application.

Simon is a software engineer based in Germany with 7 years of experience writing code for the JVM and also with JavaScript. He’s very passionate about learning new things as often as possible and a self-appointed Kotlin enthusiast.

Please follow and like this Blog 🙂
Publish Kotlin Library on Bintray using Gradle Kotlin DSL and Travis CI

Publish Kotlin Library on Bintray using Gradle Kotlin DSL and Travis CI

Distribute a Library on Bintray using Gradle Kotlin DSL

In my latest blog post, published a few weeks back, I informed about the usage of the Gradle Kotlin DSL and how it helps with describing build scripts. In another earlier post, I introduced a small library that can be utilized for simplifying the creation of TLS/SSL sockets using a custom Kotlin DSL: SeKurity.

In this post, we’ll investigate how such a library can be made available to others that actually want to make use of it inside other projects. Ultimately, it should be possible to list the SeKurity library as a simple dependency in a build script like Maven or Gradle. Since the library itself is already backed by Gradle, I’ll show a way of publishing the resulting artifacts at the end of that build.

Where to publish?

In order to make a library available to anybody else, it needs to be published to a publicly accessible e.g. Maven repository like the famous Maven Central. Unless you’re absolutely certain about the quality of your library, it’s recommended to share it on custom repositories rather than THE central one directly as you’ll kinda lose control of it once it’s out there. One alternative is called Bintray and it will also be used in the present article. After creating an account or simply signing in with your GitHub, Google or Twitter account, you can create your own repositories there.

Automating the Build

Before enabling our build to publish the built artifacts, the process should be automated in a CI tool like Travis CI, which interacts nicely with GitHub. To make it work, it must be enabled in the GitHub repo settings by adding it as a service (see Settings -> Integrations & services). Travis expects the repository to incorporate a .travis.yml configuration that tells it what to do. For a simple Java/Kotlin project using Gradle, the following will be sufficient:

language: java

before_install:
- chmod -R +x src
- chmod +x gradlew

script: ./gradlew clean build
deploy:
  provider: script
  script: ./gradlew bintrayUpload -PbintrayUser=$BINTRAY_USER
  -PbintrayApiKey=$BINTRAY_KEY
    on:
      branch: master

It’s normally not necessary to specify the script command explicitly as long as you’re using the default instructions. In the shown case, Gradle runs the build task first before it deploys via the gradle bintrayUpload taks, which we will learn about later. The deployment only happens for the master branch. The gradle deploy takes two arguments $BINTRAY_USER and $BINTRAY_KEY, which contain private data and are therefore securely stored in Travis directly. When the build is run, Travis takes care of replacing them.
You can finally enable the repository in Travis and it will start a build on every push immediately. For further information on Travis, the documentation is highly recommended.

The Gradle Build

As shown earlier, the bintrayUpload Gradle task gets invoked in the build. Where does it come from? Let’s have a look at the relevant parts of the gradle.build.kts file.

The Plugins


plugins {
    kotlin("jvm") version "1.2.30"
    id("com.github.johnrengelman.shadow") version "2.0.2"
    `maven-publish`
    id("com.jfrog.bintray") version "1.8.0"
}
  • The shadow plugin is used because we need the library to be bundled with all its dependencies in order to avoid classpath conflicts.
  • The maven-publish plugin helps with creating the relevant artifact and a relevant pom.xml file that will also be published.
  • The bintray plugin does all the heavy work at the end of the build. It refers to the maven-publish result and pushes the artifacts to Bintray.

Plugin Configuration

The build file also adds custom configurations for all previously shown plugins as shown next:

I Shadow


val artifactID = "sekurity"

val shadowJar: ShadowJar by tasks
shadowJar.apply {
    baseName = artifactID
    classifier = null
}

The shadow plugin uses the project’s name as the artifact’s baseName and the “all” qualifier by default, which is overridden here.

II Maven-Publish


fun MavenPom.addDependencies() = withXml {
    asNode().appendNode("dependencies").let { depNode ->
        configurations.compile.allDependencies.forEach {
            depNode.appendNode("dependency").apply {
                appendNode("groupId", it.group)
                appendNode("artifactId", it.name)
                appendNode("version", it.version)
            }
        }
    }
}

val publicationName = "tlslib"
publishing {
    publications.invoke {
        publicationName(MavenPublication::class) {
            artifactId = artifactID
            artifact(shadowJar)
            pom.addDependencies()
        }
    }
}

This one is a bit tricky. We reuse the result of shadow and add it to the publication which will be published under the artifactId that corresponds to the JAR’s baseName. By default, the generated POM does not list the library dependencies, which is done by mapping each compile dependency to a valid dependency XML node.

III Bintray


fun findProperty(s: String) = project.findProperty(s) as String?

bintray {
    user = findProperty("bintrayUser")
    key = findProperty("bintrayApiKey")
    publish = true
    setPublications(publicationName)
    pkg(delegateClosureOf {
        repo = "SeKurity"
        name = "SeKurity"
        userOrg = "s1m0nw1"
        websiteUrl = "https://kotlinexpertise.com"
        vcsUrl = "https://github.com/s1monw1/TlsLibrary"
        setLabels("kotlin")
        setLicenses("MIT")
    })
}

The bintray plugin is most important since it takes care of actually uploading the generated artifacts to Bintray itself. The user and key are obtained from the Gradle project properties, which are passed by Travis later on as already shown earlier. By referring to the publication created in maven-publish, all artifacts will be uploaded as specified. The pkg block describes the library’s attributes and where, i.e. to what repository, the publishing happens.

The Result

All these blocks merged result in a pretty nice Gradle build script, which can be observed in this Gist and also the SeKurity repository itself. The resulting artifact is made available here: https://bintray.com/s1m0nw1/SeKurity/SeKurity

Referring to the published Library

Now that the library is being uploaded to Bintray, it can be listed as a dependency in arbitrary build files. If we have another Gradle-backed project, we first need to add the relevant repository to the list of repositories:


repositories {
    mavenCentral()
    jcenter()

    maven {
        setUrl("https://dl.bintray.com/s1m0nw1/SeKurity/")
    }
}

After that, the library can be added as a simple compile dependency like this:


dependencies {
//...other dependencies
    compile("de.swirtz:sekurity:0.0.2")
}

Recap

As shown, Gradle plugins can be used for setting up a proper build that publishes a custom library to Bintray easily. In order to automate the whole process, CI tools like Travis can be used for executing the build whenever a new change has been made.

I hope the little tutorial helps with your build as well! If you like, have a look at my Twitter account and follow if you’re interested in more Kotlin stuff 🙂 Thanks a lot.

Simon is a software engineer based in Germany with 7 years of experience writing code for the JVM and also with JavaScript. He’s very passionate about learning new things as often as possible and a self-appointed Kotlin enthusiast.

Please follow and like this Blog 🙂
Kotlin Features I miss most in Java – Kotlin vs Java

Kotlin Features I miss most in Java – Kotlin vs Java

Let’s write an article that covers “Kotlin vs Java” topics – I want to tell you which Kotlin features I miss most when going back to Java.

My Life as a Java Dev

Although I’m a big supporter of the Kotlin programming language, I still do a lot of Java programming on a daily basis for my employer. Since I’m aware of the great functionalities of Kotlin, I’m often struggling with Java as it has some “pitfalls”, requires additional boilerplate and misses many features.
In this post, I’d like to describe which Kotlin features I miss most when coding in Java.

new and Semicolon

Ever since I’m doing Kotlin, there are two things I always forget when coding in Java: the new keyword for constructor invocations and the annoying ; to complete statements. Kotlin doesn’t have new and even semicolons are optional. I really appreciate this decision because it reduces the “syntactic noise“.

Data classes

In Kotlin, data classes are used for simple data containers, representing JSON objects or returning compound objects from functions amongst other use cases. Of course, Java doesn’t support this special type of classes yet. As a result, I often have to implement my own data class, which means a lot of boilerplate in Java.

One special use case is compound objects returned from functions. For example, let’s imagine a function that needs to return two objects. In Kotlin we could use a data class, or simpler, a Pair directly. In Java, I tend to create a value object, which is a class with several final fields, each of which instantiated through the constructor. Similar to Kotlin, I don’t implement getters and setters, but use the class’s fields directly as public properties. Unfortunately, this is not what we learned as best practice and most Java code style checkers will complain about it. I do not see any encapsulation issues here and it’s the least verbose approach in Java. The following shows such a compound object, the inner class Multi. In Kotlin this would be a one-liner.

public class MultiReturn {

    public static void main(String[] args) {
        new MultiReturn().useMulti();
    }

    public void useMulti() {
        Multi multi = helper();
        System.out.println("Multi with " + multi.count + " and " + multi.name);
    }

    private Multi helper() {
        return new Multi(2, "test");
    }
    
    private static class Multi {
        private final int count;
        private final String name;

        public Multi(int count, String name) {
            this.count = count;
            this.name = name;
        }
    }
}

Local Functions

In many situations, we tend to create private methods that are only used inside another single method in order to make this one more readable. In Kotlin, we can use local functions, i.e. functions inside functions (inside functions…), which enables some kind of scope. For me, this is a much cleaner approach, because the function is only accessible inside the function that actually uses the local one. Let’s look at an example.


fun deployVerticles() {

    fun deploy(verticleClassName: String) {
        vertx.deployVerticle(verticleClassName, opt, { deploy ->
            LOG.info("$verticleClassName has been deployed? ${deploy.succeeded()}")
        })
    }

    deploy("ServiceVerticle")
    deploy("WebVerticle")
}

It’s taken from a sample vert.x application and defines a local function that is reused twice afterward. A great way to simplify your code.

Single Expression Functions

We can create single expression functions in Kotlin, i.e. functions without an actual body. Whenever a function contains only a single expression, it can be placed after a = sign following the function declaration:


fun trueOrFalse() = Random().nextBoolean()

In Java, on the other hand, we always have to use a function body enclosed in {}, which ranges over at least three lines. This is also “syntactic noise” I don’t want to see anymore. To be fair, Java 1.8 makes it possible to define lambdas which can also solve this, less readable though (Can also be applied to local functions):


public class SingleExpFun {

    private BooleanSupplier trueOrFalse = new Random()::nextBoolean;

    private boolean getNext(){
        return trueOrFalse.getAsBoolean();
    }
}

Default Parameters

One very annoying part of Java is the way methods have to be overloaded. Let’s see an example:

public class Overloade
    public static void main(String[] args) {
        Overloader o = new Overloader();
        o.testWithoutPrint(2);
        o.test(2);
    }

    public void test(int a, boolean printToConsole) {
        if (printToConsole) System.out.println("int a: " + a);
    }

    public void testWithoutPrint(int a) {
        test(a, false);
    }

    public void test(int a) {
        test(a, true);
    }

}

We can see a class with a method test(int, boolean) that is overloaded for the default case and also a convenience method is available. For more complex examples, it can lead to a lot of redundant code, which is simpler in Kotlin by using default parameters.


fun test(a: Int, printToConsole: Boolean = true) {
    if (printToConsole) println("int a: " + a)
}

fun testWithoutPrint(a: Int) = test(a, false)

fun main(args: Array) {
    testWithoutPrint(2)
    test(2)
}

Calling multiple methods on an object instance (with)

Obviously, Kotlin is more functional than Java. It makes use of higher-order functions in incredibly many situations and provides many standard library functions that can be used as such. One of my favorites is with, which I miss a lot whenever I can’t use Kotlin. The with function can be used to create scopes that actually increase the readability of code. It’s always useful when you sequentially call multiple functions on a single object.


class Turtle {
    fun penDown()
    fun penUp()
    fun turn(degrees: Double)
    fun forward(pixels: Double)
}

with(Turtle()) {
    penDown()
    for(i in 1..4) {
        forward(100.0)
        turn(90.0)
    }
    penUp()
}

The great thing is the usage of lambdas with receiver, which you can read about in one of my other posts.

Null-Safety

Whenever I work with nullable types since the time I started with Kotlin, I actually miss the type system’s tools to prevent null-related errors. Kotlin did a very good job by distinguishing nullable types from not-nullable ones. If you strictly make use of these tools, there is no chance you’ll ever see a NullpointerException at runtime.

Lambdas and Collection Processing

Kotlin places a lot of value on its lambdas. As shown in the with example earlier, there’s special syntax available for lambdas that makes its usage even more powerful. I want to underline that the way functions and especially lambdas are treated in the language makes it dramatically superior to Java. Let’s see a simple example of Java’s Streams, which were introduced along with lambdas in Java 1.8:

List list = people.stream().map(Person::getName).collect(Collectors.toList());

It’s a rather simple example of a Stream that is used to get a list of names from a list of persons. Compared to what we did before 1.8, this is awesome. Still, it’s too noisy compared to a real functional approach as pursued by Kotlin:

val list = people.map { it.name }

Or yet another example, in which salaries of employees are summed up to a total amount:

int total = employees.stream()
                      .collect(Collectors.summingInt(Employee::getSalary)));

So much simpler in Kotlin:

val total = employees.sumBy { it.salary }

The Kotlin examples show how simple it can be. Java isn’t a functional language and has a hard time trying to adopt functional features like lambdas and streams as we can easily observe in the snippets. It really sucks to go back to Java, if you ever experienced the beauty of Kotlin. Have you ever tried to use Eclipse after being familiar with IntelliJ? You know what I mean then.

Wrap-up

In this short post, I presented you my top Kotlin features I always miss when coding in Java. It’s just a selection of things, which will hopefully find their way into the Java language soon. But to be honest, there’s no reason to always wait for Java, when there already is a much sweeter language available… I want to point out, that starting with Kotlin really made me a much better programmer because I began wondering about certain features in both languages and also try to find ways to use Kotlin-dedicated things in Java by finding workarounds like arranging my code differently.

I’d be interested in the features you like most, feel free to comment.
Also, if you like, have a look at my Twitter account and follow if you’re interested in more Kotlin stuff 🙂 Thanks a lot.

If you want to read more about Kotlin’s beautiful features I recommend the book Kotlin in Action and my other articles to you.

Simon is a software engineer based in Germany with 7 years of experience writing code for the JVM and also with JavaScript. He’s very passionate about learning new things as often as possible and a self-appointed Kotlin enthusiast.

Please follow and like this Blog 🙂
Create a DSL in Kotlin

Create a DSL in Kotlin

Kotlin as a programming language provides some very powerful features, which allow the creation of custom internal Domain Specific Languages (DSL).  One of these features, I also wrote about on this blog, is called Function Literals with Receiver, others are the invoke convention or infix notation. In this article, I will show how to create a Kotlin DSL by introducing a library that exposes a DSL as its API. I’ve often been struggling with Java’s API when I had to set up SSL/TLS connections in scenarios where I e.g. needed to implement HTTPS communication. I always felt like wanting to write a little library that can support me with this task, hiding away all the difficulties and of course the boilerplate needed for it.

Simon is a software engineer based in Germany with 7 years of experience writing code for the JVM and also with JavaScript. He’s very passionate about learning new things as often as possible and a self-appointed Kotlin enthusiast.

Please follow and like this Blog 🙂
Kotlin Coroutines Guide – Concurrent Programming in Kotlin

Kotlin Coroutines Guide – Concurrent Programming in Kotlin

Introduction and Motivation

In this article, I’d like to share my gathered insights on Kotlin Coroutines with you and hope to give a comprehensive overview. The shown code runs with Kotlin version 1.2.40 and kotlinx.coroutines version 0.22.5.
Kotlin coroutines are definitely one of the “bigger features” as indicated by the following quote, taken from JetBrains’ blog:

We all know that blocking is bad under a high load, that polling is a no-go, and the world is becoming more and more push-based and asynchronous. Many languages (starting with C# in 2012) support asynchronous programming through dedicated language constructs such as async/await keywords. In Kotlin, we generalized this concept so that libraries can define their own versions of such constructs, and async is not a keyword, but simply a function.
This design allows for integration of different asynchronous APIs: futures/promises, callback-passing, etc. It is also general enough to express lazy generators (yield) and cover some other use cases.

To put it simply, JetBrains introduced coroutines in order to provide simple means for concurrent programming. Probably many of you have already worked with Java, its Threads and concurrency classes; I myself did this quite a lot and actually, I’m really convinced of its maturity.

Java Concurrency vs. Kotlin Coroutines

Tip
If you still catch yourself struggling with Threading and Concurrency in Java, I can recommend the book Java Concurrency in Practice to you.

Although Java’s solution is really well-engineered, it’s often difficult to utilize and (of course) very verbose. Another problem is, that Java doesn’t directly encourage non-blocking programming. You often find yourself starting threads without having in mind, that they’re very expensive and introduce blocking computations quickly (due to locks, sleeps, waits, etc.). Applying non-blocking patterns alternatively is really hard and error-prone.

Kotlin Coroutines, on the other hand, are intended to be much easier and look like sequential code by hiding the complex stuff inside library functions. Yet, they provide a way to run asynchronous code without having to block threads, which provides new possibilities for applications [1]. Instead of blocking threads, computations are being suspended.
JetBrains tends to describe coroutines as “light-weight threads”; actually they are no Thread as we know them in Java. Compared to threads, coroutines are very cheap in the creation and the overhead introduced by threads isn’t around. One reason is, that they’re not directly mapped to native threads. As you will see, coroutines are executed in Threads managed by the library.
Another important difference is “limitation”: Threads are limited because they rely on available native threads, coroutines on the other side are almost free and thousands can be started at once.

Concurrent Programming Style

Different styles of asynchronous/concurrent programming styles exist in various languages, which for example are: callback-based (JavaScript), future/promise-based (Java, JavaScript), async/await-based (C#) and so on. All these concepts can be implemented with coroutines because Kotlin doesn’t dictate any style initially. Instead, all concepts are already, or at least could in future, (be) implemented using coroutines.
As one additional benefit, as opposed to e.g. callback-based programming, coroutines promote a sequential kind of asynchronous programming, i.e. although your coroutines may execute multiple parallel computations, your code still looks sequential and therefore acquainted.

The Concept of Kotlin Coroutines

The term and concept “Coroutine” is anything but new. According to the Wikipedia article, it was created in 1958 already. Many modern programming languages provide native support: C#, Go, Python, Ruby, etc. The implementation of coroutines, also in Kotlin, is often based on so-called “Continuations”, which are “an abstract representation of the control state of a computer program”. We’ll capture that in How does it work – Implementation of Coroutines again.

Getting Started – The Basics

There’s a comprehensive tutorial available on https://kotlinlang.com that describes how a project needs to be set up in order to work with coroutines. Please have a look there or just check out my Kotlin_Examples repository on GitHub.

Kotlin Coroutines Ingredients

As already hinted, the Kotlin coroutine library provides an understandable high-level API that lets us start quickly. One new modifier we need to learn is suspend, which is used to mark a method as “suspending”.
We’ll have a look at some easy examples using APIs from kotlinx.coroutines up next. But first, let’s learn what a suspending function is.

Suspending Functions

Coroutines rely on the keyword suspend, which is a modifier used in order to mark functions as “suspending” [2], i.e. that calls to such functions may suspend at any point. These can only be called from within coroutines, which in turn need at least one suspending function to be started.

suspend fun myMethod(p: String): Boolean {
    //...
}

As we can see in the example above, suspending functions look like regular functions plus the additional modifier. Keep in mind, that these methods can only be called from coroutines, other attempts will lead to compilation errors.

Coroutines can be thought of as a sequence of regular and suspending functions with an optional result being available after completion.

Hands-On

Let’s see some concrete coroutines in action. In a first example, the basics will be shown:

The first Coroutine

fun main(args: Array<String>) = runBlocking { //(1)
    val job = launch(CommonPool) { //(2)
        val result = suspendingFunction() //(3)
        print("$result")
    }
    print("The result: ")
    job.join() //(4)
}
>> prints "The result: 5"

In this example, two functions, (1) runBlocking and (2) launch, are used, which are examples of coroutine builders. Many different builders exist, each of which starts a coroutine with different purposes: launch (fire and forget), async (promise returned), runBlocking (blocks thread) and so on.

The inner coroutine started by (2) launch does the actual work: a (3) suspending function is called which might suspend at any time, the result is printed after completion. The main thread, after starting the coroutine, prints a String before the coroutine finishes.
Coroutines started by launch return a Job immediately, which can be used for canceling the computation or waiting for completion with (4) join() as we see here. Since calling join() may suspend, we need to wrap this call into another coroutine, which can often be achieved with runBlocking. This concrete coroutine builder (1)is designed to bridge regular blocking code to libraries that are written in suspending style, to be used in main functions and in tests” (Quoted API). If we removed the joining of the job, the program would stop before the coroutine can print the result.

Going deeper

A more lively example is the following: Imagine, you have to send an email from your application. Requesting the recipient address and rendering the message body are two very expensive tasks, which are independent of each other though. Being smart and using Kotlin, you want to make use of coroutines, performing both tasks in parallel, of course.
This is shown here:

email example async/await
suspend fun sendEmail(r: String, msg: String): Boolean { //(6)
    delay(2000)
    println("Sent '$msg' to $r")
    return true
}

suspend fun getReceiverAddressFromDatabase(): String { //(4)
    delay(1000)
    return "coroutine@kotlin.org"
}

suspend fun sendEmailSuspending(): Boolean {
    val msg = async(CommonPool) {             //(3)
        delay(500)
        "The message content"
    }
    val recipient = async(CommonPool) { 
        getReceiverAddressFromDatabase()  //(5)
    } 
    println("Waiting for email data")
    val sendStatus = async(CommonPool) {
        sendEmail(recipient.await(), msg.await()) //(7)
    }
    return sendStatus.await() //(8)
}

fun main(args: Array<String>) = runBlocking(CommonPool) { //(1)
    val job = launch(CommonPool) {
        sendEmailSuspending() //(2)
        println("Email sent successfully.")
    }
    job.join() //(9)
    println("Finished")
}

First, like already seen in the previous example, we use a (1) launch builder inside a runBlocking builder so that we can (9) wait for the coroutine’s completion. This isn’t new and neither is the (2) call to a suspending function sendEmailSuspending.
This method uses an (3) inner coroutine for getting the message content and (4) another suspend method getReceiverAddressFromDatabase for the address. Both tasks are executed in a separate coroutine build with (5) async. Note, that the calls to delay represent a non-blocking, coroutine suspending, alternative to Thread.sleep, which is used for mocking expensive computations here.

The async Coroutine Builder

The async builder is really simple and easy in its conception. As we know from many other languages, this method returns a promise, which is strictly speaking of type Deferred in Kotlin. All terms like promise, future, deferred or delay are often used interchangeably for describing the same concept: The async method promises to compute a value which we can wait for or request at any time.

We can see the “waiting” part of Kotlin’s Deferred objects in (7), where the suspending function (6) is called with the results of both prior computations. The method await() is called on instances of Deferred which suspends until the results become available [3]. The call to sendEmail is also wrapped in an async builder, for which completion we wait in (8) before returning its result.

The CoroutineContext

One important part of the above examples is the first parameter of the builder functions, which must be an instance of CoroutineContext. This context is what’s passed to a coroutine and provides access to the current Job (remember, this is what can be used outside a coroutine for cancellation or joining) for example.
Also, the current context may be utilized for launching inner coroutines, which has the effect that the sub-coroutine’s Job is a child of its surrounding one. That provides the possibility to cancel whole hierarchies of coroutines with a single cancellation request to the parent Job.
Different kinds of Elements are part of a CoroutineContext, one of which is CoroutineDispatcher.

In all examples shown, I used CommonPool, which is such a dispatcher. It makes sure, that the coroutines are executed in a thread pool managed by the framework. Alternatively, we could have used a confined thread, specially created, or implement our own pool for example. Contexts can even be combined easily using the overloaded + operator like so:

launch(CommonPool + CoroutineName("mycoroutine")){...}

Also, note that the CommonPool is the default dispatcher and it’s therefore defined as the default argument for the coroutine builder. We could have simply written launch { ... }, async { ... } and so on.

Shared Mutable State

Maybe, while reading the previous chapters, you have had concerns about synchronization between coroutines since I didn’t show mention any of this before. I, at least, had this concern because coroutines make use of thread pools (like CommonPool) to be dispatched in and therefore could work on shared state concurrently. It’s quite evident that synchronization is just as important as we know it from other languages like Java. We can make use of acquainted strategies like thread-safe data structures, confining execution to a single thread or using locks (see Mutex for further details).
Besides the common patterns, Kotlin coroutines encourage the use of a “share by communication”-style (see QA).

Concretely, an “actor” can be shared between coroutines. They can be used by coroutines, which may send/take messages to/from it. Let’s see how this works:

Actors

actor message passing example
sealed class CounterMsg {
    object IncCounter : CounterMsg() // one-way message to increment counter
    class GetCounter(val response: SendChannel<Int>) : CounterMsg() // a request with channel for reply.
}

fun counterActor() = actor<CounterMsg>(CommonPool) { //(1)
    var counter = 0 //(9) actor state, not shared
    for (msg in channel) { // handle incoming messages
        when (msg) {
            is CounterMsg.IncCounter -> counter++ //(4)
            is CounterMsg.GetCounter -> msg.response.send(counter) //(3)
        }
    }
}

suspend fun getCurrentCount(counter: SendChannel<CounterMsg>): Int { //(8)
    val response = Channel<Int>() //(2)
    counter.send(CounterMsg.GetCounter(response))
    val receive = response.receive()
    println("Counter = $receive")
    return receive
}

fun main(args: Array<String>) = runBlocking<Unit> {
    val counter = counterActor()

    launch(CommonPool) { //(5)
            while(getCurrentCount(counter) < 100){
                delay(100)
                println("sending IncCounter message")
                counter.send(CounterMsg.IncCounter) //(7)
            }
        }

    launch(CommonPool) { //(6)
        while ( getCurrentCount(counter) < 100) {
            delay(200)
        }
    }.join()
    counter.close() // shutdown the actor
}

This example shows the usage of an (1) Actor, which is a coroutine itself working on any context. The actor is holding the (9) relevant state of this sample application, which is counter. Another important feature, we haven’t considered so far, is a (2) Channel:of


Channels

Channels provide a way to transfer a stream of values, similar to what we know as BlockingQueue (enables producer-consumer pattern) in Java but without any blocking methods. Instead, send and receive are suspending functions used for providing and consuming objects from the channel, implemented with FIFO strategy, making them fair.


The actor is, by default, associated to such a channel, which can be used in other coroutines (7) for sending messages to it. In the example, the actor iterates over the stream of messages from its channel (for works with suspending calls) handling them according to their type: (4) IncCounter messages make the actor change its state by incrementing the counter while (3) GetCounter makes the actor return its counter state by sending an independent message to the GetCounter‘s SendChannel.
The first coroutine (5) in main, just for the sake of convenience, launches a task which sends (7) IncCounter messages to the actor as long as the counter is less than 100. The second (6) just waits until the counter reaches 100. Both coroutines make use of the suspending function (8) getCurrentCounter, which sends a GetCounter message to the actor and suspends by waiting on receive to return.

As we can see, the whole relevant state is confined to the specific actor coroutine. This solves the problem of shared mutable state.

More Features and Examples

If you really want to dive into coroutines and start working with it, I recommend the commonly known Kotlin documentation and especially want to suggest this fantastic guide.

How does it work – Implementation of Kotlin Coroutines

I cannot go too much into detail here because this would exceed the post’s intention. I’m planning to write a follow-up with more detailed information on implementation, considering the generated bytecode of coroutines as well, in the next weeks.
For now, let’s limit the following description to a “bird’s eye view”-ish one.

Coroutines do not rely on features of the operating system or the JVM. Instead, coroutines and suspend functions are transformed by the compiler producing a state machine capable of handling suspensions in general and passing around suspending coroutines keeping their state. This is enabled by Continuations, which are added as a parameter to each and every suspending function by the compiler; this technique is called “Continuation-passing style”.
If you can’t wait to know any details, you need to read this explanation.

Pro Tips by Roman Elizarov

I’ve talked to Roman Elizarov from JetBrains, who’s highly responsible for Kotlin coroutines and want to share the gathered information with you:

The first question I had: When am I supposed to use coroutines and are there any use cases that still require threads to be used?

A: Rule of thumb by Roman:
Coroutines are for asynchronous tasks that wait for something most of the time. Threads are for CPU-intensive tasks.

I mentioned, that the phrase “light-weight thread” sounds kind of inappropriate to me as it obscures the fact, that coroutines rely on threads since they are executed in a pool of threads. In my opinion, coroutines are rather a “task” being executed, stopped etc.

A: Roman answered, that the phrase “light-weight threads” is rather superficial, and that “coroutines are in many ways like threads from user’s standpoint.”

As a last question, I wanted to know about synchronization. If coroutines are alike threads, there must be the necessity of synchronizing shared state between different coroutines.

A: Roman told me, that known patterns of synchronization may be used, but it is recommended not to have any mutable shared state at all when we use coroutines. Instead, coroutines “encourage […​] to adopt “share by communication” style.

Conclusion

Kotlin Coroutines once again demonstrate Kotlin’s magnificence. As opposed to Java, Kotlin encourages a totally different style of concurrent programming, which is non-blocking and naturally doesn’t make us start huge amounts of native threads.
In Java, it’s mostly normal to just start another thread or create new pools without having in mind, that this introduces a huge overhead and can even make our application slow due to blocking code for example. Coroutines, as an alternative, are said to be “light-weight threads”, which describes the fact, that they’re not mapped to native threads and therefore don’t drag along all the risks and problems we usually have to deal with (deadlocks, starvation e.g.). As we’ve seen, with coroutines, we normally don’t have to worry about blocking threads, synchronization is much more straightforward and not even necessary ideally as long as we pursue “share by communication”.

Coroutines also enable us to work with several different kinds of concurrent programming, each of which is either available in the library already (kotlinx.coroutine) or at least could be implemented easily.
Java developers, in particular, might most likely be acquainted with submitting tasks to a thread pool and waiting for results of futures then (ExecutorService), which we easily achieve by using async/await style. Yet, it’s not just an equal replacement, but a big improvement to what we know already.
Think about your concurrent Java code, all those checked exceptions, defensive locking strategies and a lot of boilerplate code. With coroutines, it’s normal to write code sequentially by calling suspend functions, communicating with other coroutines, waiting for results, canceling coroutines and more.

Perspective

Although I’m convinced that coroutines are truly fantastic, time will show whether it’s actually mature enough for highly concurrent applications. Many programmers will have to rethink and apply totally different concepts to their programs. I’m really curious about its future. As of now, Kotlin coroutines are just experimental, which means JetBrains might adjust them in upcoming releases in order to adapt feedback from the community after playing around or even adopting coroutines in productive projects.

Please don’t hesitate to get in touch, feedback’s always appreciated 🙂 Also if you like, have a look at my [Twitter](https://twitter.com/s1m0nw1) and follow if you’re interested in more Kotlin stuff 🙂 Thanks a lot.If you want to read more about Kotlin’s beautiful features I highly recommend the book Kotlin in Action and my other articles to you.

 

Simon


1. Non-blocking programming became more and more popular in the last time because reactive programming gained influence.
2. Actually, it’s “possibly suspending” since such a function may suspend, but doesn’t have to.
3. Keep in mind, that Java’s Future blocks a thread as soon as we call get() on it.

 

Simon is a software engineer based in Germany with 7 years of experience writing code for the JVM and also with JavaScript. He’s very passionate about learning new things as often as possible and a self-appointed Kotlin enthusiast.

Please follow and like this Blog 🙂
Kotlin Generics and Variance (Compared to Java)

Kotlin Generics and Variance (Compared to Java)

This article covers the concepts of Generics and Variance in Kotlin and compares it to Java. Kotlin Generics differ from Java’s in how users can define their type of variance. As opposed to Java, Kotlin allows defining variance on declaration-site, whereas Java only knows use-site variance.

Kotlin Generics – What is Variance?

Many programming languages support the concept of subtyping, which allows implementing hierarchies that represent relationships like “A Cat IS-An Animal“. In Java, we can either use the extends keyword in order to change/expand behavior of an existing class (inheritance) or use implements to provide implementations for an interface. According to Liskov’s substitution principle, every instance of a class A can be substituted by instances of its subtype B. The word variance, often referred to in mathematics as well, is used to describe how subtyping in complex aspects like method return types, type declarations, generic types or arrays relates to the direction of inheritance of the involved classes. There are three terms we need to take into account: Covariance, Contravariance, and Invariance.

Read More Read More

Simon is a software engineer based in Germany with 7 years of experience writing code for the JVM and also with JavaScript. He’s very passionate about learning new things as often as possible and a self-appointed Kotlin enthusiast.

Please follow and like this Blog 🙂
Setup Vert.x Application written in Kotlin with Gradle – Kotlin Reactive Programming

Setup Vert.x Application written in Kotlin with Gradle – Kotlin Reactive Programming

I decided to write a Vert.x application in combination with Kotlin in a simple example because I’m really interested in Reactive Programming and love to use Kotlin. In this post, I will give some basic information on Vert.x as a tool set for writing reactive applications on the JVM and also introduce Kotlin a bit. In the end, I want to demonstrate how this application can be set up in Gradle.

Read More Read More

Simon is a software engineer based in Germany with 7 years of experience writing code for the JVM and also with JavaScript. He’s very passionate about learning new things as often as possible and a self-appointed Kotlin enthusiast.

Please follow and like this Blog 🙂
Java Oracle Certification

Java Oracle Certification

Java Oracle Certification – Should you consider it?

I took the OCA SE 7 Programmer I Certification exam two years ago in 2015, which I passed successfully with a score of 86%. This year I will be taking the second exam of this certification path called OCP SE 7 Programmer II. In this post, I want to reflect why taking these kind of exams isn’t a waste of time but can be a great advantage for you as a Java developer. Of course, being certified in whatever area will increase your chances of getting a better job because certificates represent some kind of guaranty of your skills to a company. No doubt about that.

Read More Read More

Simon is a software engineer based in Germany with 7 years of experience writing code for the JVM and also with JavaScript. He’s very passionate about learning new things as often as possible and a self-appointed Kotlin enthusiast.

Please follow and like this Blog 🙂