Find myself looking for an overview too often. So let’s create a rough overview of common used config for Spark. As a start, create a Spark Session with default config: from pyspark.sql import SparkSession spark = SparkSession.builder \ .master(SPARK_MASTER) \ .appname("app name") \ .getOrCreate() The Spark Context represents the connection to the cluster; communicaties with lower-level API’s and RDDs. Some resource settings on the driver: ... .config("spark.driver.memory", "8g") ... .config("spark.cores.max", "4") ....

Through Hackernews I found out about the recent release of Rancher Desktop and I was curious if this would be a good alternative to Docker desktop for the develop of web applications on my local machine. I don’t really have a problem with Docker desktop, just good to try something new every now and then and it is open-source. Running some containers really gets some steam out of my Mac so hopefully it has some improvement there....

Had a complete headache trying to figure out how a decorator as a class can maintain the possible async properties of a method. The solution is actually very simple. When called, use inspect.iscoroutinefunction to check whether it is a coroutine, and return again an async method! The example adds given paths to a registry, import inspect from functools import wraps paths_registry = [] class route(object): def __init__(self, path: str, **kwargs) -> None: self....

This is a simple demonstration of a domain event factory in Python. I assume you are familiar with the Factory Method Pattern. I also use the pydantic package for attribute validation. When implemented, we can use the factory to create immutable domain events with a homogenous data structure across instances of the same type. The metadata is generated by the underlying BaseEvent. In this approach we always produces complete events....

While reading into implementing DDD, there is often a plea for the use of immutable objects. The main motivation is that an object that is initially valid always remains valid; there is no later verification or validation required. Secondly, working with an immutable object cannot cause any side effects. Some data objects in Python are immutable, the dataclasses themselve are not. Let’s have this simple class: class SimpleClass: def __init__(self, attr1: int): self....