My name is Joost – I am an engineer and this is my digital garden where I write sometimes about things that interest me. Also I keep lists of books I have read 📚 and things and tools that I use 🔧.
I am completely into drums 🥁 and espresso ☕.
Skiing in the Alps
While skiing in the Swiss Alps last winter, I got into a discussion about speed. Following my engineering knowledge, your skiing speed does not depend on your own mass. Without the intention to offend anyone, skiing is basically a form of falling at an angle. We can build a model based on Newton’s second law where acceleration times mass is equal to the resultant force. We can see that mass ($m$) will drop out of the equation:...
Overview of Spark configurations
Find myself looking for an overview too often. So let’s create a rough overview of common used config for Spark. As a start, create a Spark Session with default config: from pyspark.sql import SparkSession spark = SparkSession.builder \ .master(SPARK_MASTER) \ .appname("app name") \ .getOrCreate() The Spark Context represents the connection to the cluster; communicaties with lower-level API’s and RDDs. Some resource settings on the driver: ... .config("spark.driver.memory", "8g") ... .config("spark.cores.max", "4") ....
Local development on Rancher Desktop
Through Hackernews I found out about the recent release of Rancher Desktop and I was curious if this would be a good alternative to Docker desktop for the develop of web applications on my local machine. I don’t really have a problem with Docker desktop, just good to try something new every now and then and it is open-source. Running some containers really gets some steam out of my Mac so hopefully it has some improvement there....
Async method decorator
Had a complete headache trying to figure out how a decorator as a class can maintain the possible async properties of a method. The solution is actually very simple. When called, use inspect.iscoroutinefunction to check whether it is a coroutine, and return again an async method! The example adds given paths to a registry, import inspect from functools import wraps paths_registry = [] class route(object): def __init__(self, path: str, **kwargs) -> None: self....
A Simple Factory for Domain Events
This is a simple demonstration of a domain event factory in Python. I assume you are familiar with the Factory Method Pattern. I also use the pydantic package for attribute validation. When implemented, we can use the factory to create immutable domain events with a homogenous data structure across instances of the same type. The metadata is generated by the underlying BaseEvent. In this approach we always produces complete events....