This got me really interested, because I maintain Jelly, also a binary RDF format. Jelly, however, focuses on very high read-write performance, and while it is pretty compact, I would not expect it to be comparable to CBOR-LD in terms of size.

How does CBOR-LD work?

Quite honestly, I have a limited understanding of that, but here’s what I’ve figured out so far. In JSON-LD there is the notion of a “context”, which is basically a dictionary that maps JSON keys to more complex structures.

Here is an example from the JSON-LD 1.1 spec. We can take an input document like this:

[{
  "http://schema.org/name": [{"@value": "Manu Sporny"}],
  "http://schema.org/url": [{ "@id": "http://manu.sporny.org/" }],
  "http://schema.org/image": [{ "@id": "http://manu.sporny.org/images/manu.png" }]
}]

…add a context like this:

{
  "@context": {
    "name": "http://schema.org/name",
    "image": {
      "@id": "http://schema.org/image",
      "@type": "@id"
    },
    "homepage": {
      "@id": "http://schema.org/url",
      "@type": "@id"
    }
  }
}

…and we get a compacted document like this:

{
  "@context": ... ,
  "name": "Manu Sporny",
  "homepage": "http://manu.sporny.org/",
  "image": "http://manu.sporny.org/images/manu.png"
}

I’ve replaced the context with ... here, because there are actually two ways you can use a context in JSON-LD: you can either inline it (put the whole @context map there) or you can refer to it by a URL. The latter is a clever trick, because you are removing a bunch of information from the document and putting it somewhere else. The producer of the data and the consumer must agree on the context, but once they do, the data can be very compact. This behavior is in fact unique to JSON-LD among standardized RDF formats. The others (like Turtle) never put data outside of the document.

CBOR-LD is first of all a translation of JSON-LD to binary, which already saves a few bytes. But it also takes this context mechanism to the extreme with what it calls semantic compression. The context dictionary is then indexed by integers (not strings), and you can also put more pattern types in the dictionary than with JSON-LD. CBOR-LD also uses value encodings of literals (so, instead of a string “1234” you would have an actual integer), which is another way to save space, while losing a bit of information (RDF purists might not like that).

My explanation is probably not very accurate, so I recommend you check out the presentation slides I mentioned earlier or the draft spec for more details.

How does Jelly work?

That’s really a story for a few other blogposts! 😆 But, in short, Jelly is much closer to more traditional RDF formats (like Turtle) than CBOR-LD. Internally, it looks a lot like an N-Triples/N-Quads file (just a long list of triples), but the triples are interspersed with dictionary entries.

For example, if we are going to use the IRI http://xmlns.com/foaf/0.1/knows, we put the prefix http://xmlns.com/foaf/0.1/ in the prefix lookup under, say, ID 4, and then we put the string knows in the name lookup under ID 2. The IRI then looks like this: RdfIri(4, 2). The lookups are updated as we go along through the stream, evicting old entries if needed. They are sent via the same byte stream as the triples/quads, so you get the entire data in one file – just like in Turtle or N-Triples.

There are also a few other techniques used, including compressing repeating terms and some tricks with differential compression of lookup references… You can find the full spec on Jelly’s website, if you are interested. The nice part about it is that it’s very fast, works automatically with any RDF data (no need to think about contexts), can handle data of any size (with bounded memory!), and is sufficiently compact for most use cases.

Experiment 1 – IoT data

Like I mentioned, the two formats have very different use cases. Being shameless and curious, I first tried them with something that Jelly was built for – streaming IoT data. I took the assist-iot-weather-graphs dataset from RiverBench. The dataset is a series of weather observations, with each measurement being a named RDF graph with a bunch of triples describing the sensors, wind speed, temperatures, and so on (you can find samples on its description page in RiverBench).

The code for the experiments is here. It uses the new shiny integration of Jelly-JVM with the Titanium RDF API. The new Titanium API is intended to be a minimalistic bridge between different RDF libraries and I think it does a great job at that – linking Jelly-JVM to Titanium JSON-LD was a breeze.

I first needed to encode the data in JSON-LD, and then compact it. For that I needed a context. Now – I’m bad at JSON-LD, so I used the context creation algorithm that Apache Jena uses, which is to simply use all explicitly defined prefixes and build a context like this:

{
  "@context": {
    "aiot": "https://assist-iot.eu/ontologies/aiot#",
    "aiot_p2": "https://assist-iot.eu/ontologies/aiot_p2#",
    "geosparql": "http://www.opengis.net/ont/geosparql#",
    "om": "http://www.ontology-of-units-of-measure.org/resource/om-2/",
    "pilot": "https://assist-iot.eu/pilot2_rdf/",
    "prov": "http://www.w3.org/ns/prov#",
    "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
    "sosa": "http://www.w3.org/ns/sosa/",
    "ssn": "http://www.w3.org/ns/ssn/",
    "xsd": "http://www.w3.org/2001/XMLSchema#"
  }
}

Note that this is likely not even close to being optimal, but it is simple to build and analogous to Turtle. Also note that Jelly does not need you to manually specify the prefixes, it figures them out automatically from the raw data.

I then put the context under a local URL (in a file on disk), so that the context is just an URL reference, and compressed it with CBOR-LD.

Here are the results, in bytes:

One thing I didn’t mention is that Jelly can work with RDF streams, so it is able to encode multiple RDF datasets (in this case, 1 dataset = 1 weather observation) as a single stream. This is why the first frame in Jelly’s stream is larger – it contains the prefixes and other lookup entries. The subsequent frames are much smaller, because they only contain the dictionary references and literals.

CBOR-LD seems to be much worse here, but like I mentioned, I did a very bad job at preparing the JSON-LD context. If I used something really fine-tuned to the kind of data I’m working with, the results would be much better (probably better than Jelly!). If someone wants to try their hand at this, I would be very interested in updating the blogpost with them. I think it should be possible to get it down below 1000 bytes.

Experiment 2 – Verifiable Credentials

Now, this is what CBOR-LD was really built for. Verifiable Credentials (VCs) contain information about, for example, your driver’s license or vaccination status. They are meant to be stored in QR/barcodes or NFC tags, so the smaller the better.

The ld-cli tool has a great example of something like this – only 145 bytes for the whole thing! I wanted to see how Jelly would do with it.

The code for this experiment is in the same file as before.

Remember, CBOR-LD is putting essentially the entire dictionary (IRIs and stuff) in the context, which is not part of the message. In Jelly, I ran the same VC through the encoder a few times, to get a similar effect as with the IoT data. The first frame is larger, because it contains the prefixes and dictionary entries, but the subsequent frames are much smaller.

So – if you treated the first Jelly frame as a “context” of sorts that is pre-shared between the producer and the consumer, you would get a very similar mechanism as in CBOR-LD. In that case I’d say that the comparison between the 331 bytes of Jelly to the 145 bytes of CBOR-LD is a fair one.

I also tried applying binary compression (gzip and zstd) to the outputs (see the code):

Setting higher compression levels did not make the files smaller. We can see that you can shave off a few bytes from the Jelly file, but it’s still far from being as compact as CBOR-LD. On the other hand, the CBOR file gets a bit larger, because the data is already very tightly compressed.

Concluding thoughts

Jelly and CBOR-LD are both binary RDF formats, but with very different goals, use cases, and strengths.

CBOR-LD can be really, really compact, if you put in the work of defining a good context. Jelly does everything automatically for you (“one size fits all”), but is not as compact. There are things that could be improved in Jelly to make it a bit smaller (and I’m working on it!), but it often does require sacrificing some interoperability or speed.

On the very far extreme end, you could define a binary format that has a context that’s a pattern of triples with a couple of blanks to fill in (like a template). The binary representation would be just the serialized field contents, in sequence. Literally zero overhead. Flexibility would be zero as well, but it would be the most compact format possible. At this point I start to wonder if it wouldn’t make sense to just use a dedicated binary format for the specific use case, and then link it to an RML template or something like that. 😆 Quite honestly, I think that this kind of approach may make quite a lot sense for narrowly-defined messages – from what I understand this is similar to what the SmartEdge EU project is working on.

The point is that there is always a trade-off between compactness, speed, and flexibility. Jelly is optimized for speed and flexibility, while CBOR-LD is optimized for compactness. The choice of format depends on the use case. I think CBOR-LD is an excellent idea overall, but one with relatively narrow use cases. There is also the issue of how to keep track of the registered contexts, and the related security implications – this was brought up at the last W3C JSON-LD CG meeting. I will keep working on making Jelly a bit more compact, but there is only so much that you can do without sacrificing the other properties.

Caveat: I did not compare speed here, at all. While Jelly is insanely fast, especially for IoT data, I would expect CBOR-LD to be way slower, because it’s another layer of complexity on an already complex format (JSON-LD). I guess that’s a topic for future blogpost.

Shout-outs

Many thanks to filip26 for his help in explaining to me how CBOR-LD works and how to use it. He made the amazing Titanium RDF API I used to convert between Jelly and CBOR-LD, the iridium-cbor-ld library, and the ld-cli tool, which was super helpful.

Also, shout out to the W3C JSON-LD Community Group, through which I got to know CBOR-LD better!

Welcome to the first post in the “gotta go fast” series about Scala 3 and JVM performance. If you don’t want to miss future posts, consider subscribing to my RSS/Atom feed.

In this post, we explore the ways in which you can avoid the overhead of allocating things on the heap, which for me turned out to be the place where I’ve achieved the most noticeable performance gains.

First things first – please remember that premature optimization is the root of all evil. Modern JVMs are pretty smart and can optimize a lot of things for you. Do not optimize your Scala code before you find a clear issue with its performance. Many techniques described here go against “the Scala way”, may make your code uglier, or make it easier to introduce bugs into your code. But, they make it go faster. 😉 You have been warned.

With this disclaimer out of the way…

The horror of heap allocations

At some point you may run in to a situation where your application spends the majority of CPU time allocating objects in the heap and then freeing them with the garbage collector (GC). You can check this easily with any decent profiler. Sure, the allocation mechanism in modern JVMs is pretty efficient¹, and there is a whole science dedicated to choosing the right GC for a given job, but an allocation is an allocation, and it must come with some performance cost. If you are for example allocating millions of objects per seconds, then it may be worth taking a closer look at.

OK, so we have too many allocations – the solution then is obvious: allocate less! But how?

The Scala “bloat”: Option, Either, Try

Option[T] was created as a functional and type-safe alternative to null in Java, which admittedly is one of the most annoying parts of the language. Same with Either[T1, T2] for unions and Try[T] for exceptions. Let’s look at an example:

import scala.util.{Success, Try}

val cat: String = "Cat!"
val someCat: Option[String] = Some(cat)
val rightCat: Either[Unit, String] = Right(cat)
val successCat: Try[String] = Success(cat)

Seems pretty normal. But, this ordinary snippet contains four (!) allocations: one String, one Some[String], one Right[String], and one Success[String]. So, every time you simply want to say “yes, this variable really does have a value” and use Some(value), you make an allocation. This can add up very, very quickly in a hot path.

You can get around this by simply not using these features of Scala. I definitely do not recommend doing this for public APIs, only for internal code. Please also take care to test your code properly, because these great Scala features were designed to protect your code from bugs.

• Replace Option[T] with just T. Instead of None use null. Note that using a None by itself does not allocate anything new, as None is a Scala object (singleton), so you only save allocations on removing Some(...).
• Replace Either[T1, T2] with a union type: T1 | T2. Scala supports these pretty well. The disadvantage is that instead of nice functional methods for accessing the right/left value of an Either, you must use match expressions instead. Internally, matching by type uses isInstanceOf, which is really fast on modern JVMs. On the other hand, union types may be seen as nicer in some use cases, as they support the type algebra.
• Replace Try[T] with good ol’ exceptions… or something else entirely, like a union type. You should not rely on exceptions for the “normal” control flow of your program, but if what you are catching really are exceptions, then it’s probably fine.

This will make your code look more like Java than Scala – it’s up to you to decide if the speedup is worth it.

Pre-allocation

I’ve already mentioned that using None by itself is free in terms of allocations – it is allocated as a singleton, so every time you use it, you get a reference to the exact same object on the heap. You can use this trick in a few other situations as well.

Default instance

Let’s take a simple example:

case class Cat(friends: List[Cat] = List())

def getLonelyCat: Cat = Cat()

Each time we call the getLonelyCat method, a new Cat instance is allocated on the heap. However, as the class is immutable, this does not benefit us in any way. We could have 50 different callers obtain a new lonely cat in this way and none of them would be able to modify their instance of the Cat. Why not instead give them the exact same instance, saving 49 allocations? They won’t realize the difference, after all…

object Cat:
  val defaultInstance: Cat = Cat()

case class Cat(friends: List[Cat] = List())

def getLonelyCat: Cat = Cat.defaultInstance

Here we have created a companion object to the Cat class, which allocates a single new lonely cat at application startup (in Java lingo: static initializer block). We do not allocate any new lonely cats at runtime.

Commonly-used instances

You can take the default instance trick further by pre-allocating more than one instance of a class. This admittedly has a narrower range of use cases, but is useful where you have some frequent patterns in the objects you allocate. Let’s consider a temperature sensor which reports temperature measurements in Kelvin and the status of the sensor:

enum TempSensorStatus:
  case Normal, TooCold, TooHot, Error, Unresponsive

// value is in Kelvin
case class TemperatureReading(value: Double, status: TempSensorStatus)

object TemperatureReading:
  val tooCold = TemperatureReading(0.0, TempSensorStatus.TooCold)
  val tooHot = TemperatureReading(1000.0, TempSensorStatus.TooHot)
  val error = TemperatureReading(Double.NaN, TempSensorStatus.Error)
  val unresponsive = TemperatureReading(Double.NaN, TempSensorStatus.Unresponsive)

When reading the temperature, you can now just return one of the pre-allocated instances, instead of allocating a new one every time.

You could also use an instance cache. When you need to allocate a new object, check if it’s already in your cache. If yes, then return it to the caller. If not, allocate a new one, store it in the cache, and return to the caller. Note that this is much more complex and it may be hard to get any performance improvement out of this. A lot depends on the efficiency of your cache implementation and generally it only makes sense for large classes with many internal allocations and complex initializer logic.

Reuse, recycle

In the previous section we exploited immutability to our advantage, but the same characteristic can also cause us headaches elsewhere. A good way to avoid allocating objects would be to just reuse what you have already allocated. But, with immutability, you can’t just overwrite the old contents with new data!

Mutable collections

Things are immutable in Scala for a reason, but there are also mutable collection types which may make sense in some use cases. So, instead of:

val twoCats: List[String] = List("Black cat", "Orange cat (smart)")
// This allocates new instance of List[String]
val threeCats: List[String] = twoCats :+ "White cat (dirty)"

You could use:

val cats = collection.mutable.ListBuffer("Black cat", "Orange cat (smart)")
cats += "White cat (dirty)"

…which optimistically saves us one allocation. Of course, if the underlying collection needs to be expanded, it will internally do some allocations.

Mutable fields

You can also consider using mutable fields in your classes, and reusing previously allocated instances that you don’t need anymore. This is useful for example in caches, where instead of allocating a new value for the cache, you can just reuse an object you already had there, as you are going to evict it anyway.

Instead of:

case class Dog(name: String)

val dogs = collection.mutable.ArrayBuffer(Dog("Rex"), Dog("Max"), Dog("Woof"))
// Replace dog at index 2 with a new one (allocation & deletion)
dogs(2) = Dog("Cat-Chaser the Dog")

You could do:

case class Dog(var name: String)

val dogs = Array(Dog("Rex"), Dog("Max"), Dog("Woof"))
// Replace dog at index 2 with a new one (no allocation)
dogs(2).name = "Cat-Chaser the Dog"

Of course, then you have to be 100% sure that you control the lifetime of these instances and that you won’t, for example, rename someone’s dog by accident. That would be very unethical.

Summary

In short: the less you allocate, the better. I hope these tricks will be useful to someone, but please try to use them responsibly and only if there is a clear performance issue.

In future parts of this mini-series I will tackle topics like Scala compiler quirks, polymorphism, and inlining. Stay tuned!

Footnotes

Welcome to the first post on my blog! 👋 This one is very technical, but I want to cover very diverse topics in the future. Let’s see how it goes.

Frustrations with Pandas

If you are familiar with data science and Python, then you surely have used the Pandas library. It’s a de-facto standard tool for manipulating data frames – tables of data with columns and rows. It has a simple, easy-to-learn API and is well-integrated with the rest of the Python ecosystem. It was also a source of my many frustrations since I started using it, making it one of my most dreaded and, at the same time, most used tools.

Pandas is great, but over time, I started noticing more and more of its limitations. One of the most obvious ones is the inconsistent and confusing API. I could never properly memorize in which contexts the square braces ([]) were for selecting columns or rows, or when does a method return a view (and can this view be modified) over a DataFrame or its copy. Filters are janky. The groupby method has a few different styles of argument specification that I always have to look up – same with apply and a few others. Indexes (especially multi-level indexes) are confusing to use and don’t seem to be useful in most cases. The list goes on – in general, the API looks like it evolved organically over time, without a specific design philosophy. For me, this makes Pandas harder to use, which slows down work and wastes time.

The second big frustration I had was with handling larger-than-memory datasets. Pandas always loads everything into memory, making it plain impossible to work on some relatively common workloads, like processing a large dump of sensor data from experiments. This can be alleviated by pre-processing the data using something simpler (like the GNU command-line tools that come with Linux), or rolling out the big guns – something like Dask. Instead of using eager evaluation (like Pandas), Dask builds a computation graph (i.e., query plan) that is then lazily evaluated. It can work with data residing in files, has a built-in optimizer, and can even distribute workloads across multiple machines. The issues I had with it was that it had a limited feature set, still inherited the other downsides of Pandas, and added its own layer of complexity. It does work, but for me it’s just wonky.

Finally, Pandas is just slow. In more complex scenarios, you may need to introduce some Python code to handle a more complicated task, which will instantly reduce the performance dramatically. Even if you avoid this and try the many interesting performance tricks, the processing speed for me always left something to be desired. Dask doesn’t help this much, especially for workloads that are inherently single-threaded.

The broader context

These are not new issues, they are well-known to the community, and many solutions for them were proposed, of varying usefulness.

I was often left wondering if I could just use the R language and then all my problems would go away. I was always under the impression that the R data table system was much more consistent and thought-through – same goes for the excellent ggplot2 library versus the incredibly messy matplotlib in Python.

Although R is a near-perfect ecosystem for data science, it lacks one thing that for me made it impossible to make the switch: the vast collection of libraries written in Python. If I worked only with data science and machine learning, it would be fine – but what about libraries for knowledge graphs, HTTP servers, MediaWiki parsers, MQTT connectors, computer vision, SSH remoting, connecting to weather stations over serial, processing architectural models (BIM), and many, many more?¹ I’m sure you can do all of this somehow in R… but I think we can all agree that Python does have an outstanding ecosystem of libraries for basically everything. This alone makes it worth sticking with, at least for me.

What we needed all this time: Polars

Recently I’ve stumbled upon the (relatively) new Polars library, and it turned out to be everything I needed all this time. It is incredibly fast, by the virtue of smart vectorization, using columnar storage (same idea as DuckDB – another hot topic), and being written in Rust. It has a consistent and well-designed API that actually makes sense. It supports not only lazy evaluation with a query optimizer, but also (experimentally) streaming evaluation. Stream processing is especially interesting, because it could allow for supporting datasets of arbitrary size, even on a single machine. I will come back to the topic of stream processing in a future post.

So far I have used Polars in a few data science notebooks, and the results were very good. I had no trouble learning the new syntax and the performance is just as advertised – blazing-fast. I will certainly replace Pandas with Polars in all my future data science projects.

The one thing that Polars does not implement is distributed computing, which is offered by Dask. For me, this is not an issue (more of an advantage, really, due to lower complexity), as I do not work with data that would require distributing the workload. Most of the time I’ve found you can squeeze out a lot of performance by efficiently using the resources of a single node (multithreading, vectorization). With modern CPUs that can crunch vectors with impressive throughput, a lot more can be done on a single node, if you can use this potential. Polars can.

The future

Pandas is a great tool, don’t get me wrong. I very highly respect the incredible amount of work put into it by its developers. But, I see it as a step in the evolution of the data science ecosystem, which must move forward to stay relevant. Sometimes, it is better to drop a set of assumptions that don’t work so well anymore, start from scratch, and build something drastically better. I hope that Polars will be this thing.

I think the long-term success of Polars will depend on its integration with the wider Python ecosystem. What I really, really, really want to see is an implementation of the Seaborn statistical plotting library over Polars, with a completely redesigned API, akin to R’s ggplot. There is in fact some progress in this direction with Seaborn experimenting with a completely new API for plotting, and there being already some rudimentary support for plotting from Polars DataFrames. The latter is unfortunately realized by making a full copy of the data to a Pandas DataFrame… so we are not getting any of the performance advantages. I hope the developers of Seaborn will see this opportunity and catch on to it. Then, Python data science will be truly unbeatable. 💪

Polars also has an equivalent API in Rust, so it may be possible to introduce support for more languages in the future. I’m especially curious if one compile it to WebAssembly and use as a basis for future data-crunching Cloud-Edge-IoT applications (Sowiński et al., 2023). But this is a rather distant future. Or is it? 🤔

Footnotes