Skip to content

Friday, 14 February 2025

About Me

I’m Rishav Ray Chaudhury, a third year Electrical Engineering undergrad from India. I have participated in Season of KDE this year. I am tasked with implementing a variant of the Mancala board game called Kalah under the guidance of João Gouveia and Benson Muite.

Why I chose this project

As with any aspiring developer, I too started out with game development. A couple of my friends and I started to make games using C# and the Unity framework in my first year of University. The feeling of making games that my friends enjoyed was exhilarating to say the least. This was also around the same time that I learned about open-source software. Apparently, a lot of software that developers frequently use, was developed by a group of passionate programmers. I tried contributing to some repos but was unsuccessful, mainly because I was completely unfamiliar with the projects. That all changed when I started using Arch Linux. For Arch, the desktop environment that I chose was KDE Plasma. After using it for some time, I came to know about Season of KDE and finally took the initiative to start contributing to software that I frequently used. Of the projects, the game development project was the one that caught my eye and now here I am developing a game for it.

Modern distributed systems need to process massive amounts of data efficiently while maintaining strict ordering guarantees. This is especially challenging when scaling horizontally across multiple nodes. How do we ensure messages from specific sources are processed in order while still taking advantage of parallelism and fault tolerance?

Elixir, with its robust concurrency model and distributed computing capabilities, is well-suited for solving this problem. In this article, we’ll build a scalable, distributed message pipeline that:

  • Bridges RabbitMQ and Google Cloud PubSub, delivering messages from RabbitMQ queues to a PubSub topic.
  • Ensures message ordering for each RabbitMQ queue.
  • Scales horizontally across multiple nodes.
  • Distribute the message pipelines evenly across the Elixir cluster.
  • Gracefully handles failures and network partitions.

Many modern applications require processing large volumes of data while preserving message order from individual sources. Consider, for example, IoT systems where sensor readings must be processed in sequence, or multi-tenant applications where each tenant’s data requires sequential processing.

The solution we’ll build addresses these requirements by treating each RabbitMQ queue as an ordered data source.

Let’s explore how to design this system using Elixir’s distributed computing libraries: Broadway, Horde, and libcluster.

Architecture overview

The system consists of multiple Elixir nodes forming a distributed cluster. Each node runs one or more Broadway pipelines to process messages from RabbitMQ queues and forward them to Google Cloud PubSub. To maintain message ordering, each queue has exactly one pipeline instance running across the cluster at any time. If a node fails the system must redistribute its pipelines to other nodes automatically, and if a new node joins the cluster then the existing pipelines should be redistributed to ensure a balanced load.

Elixir natively supports the ability to cluster multiple nodes together so that processes and distributed components within the cluster can communicate seamlessly. We will employ the libcluster library since it provides several strategies to automatize cluster formation and healing.

For the data pipelines, the Broadway library provides a great framework to support multi-stage data processing while handling back-pressure, batching, fault tolerance and other good features.

To correctly maintain the distribution of data pipelines across the Elixir nodes, the Horde library comes to the rescue by providing the building blocks we need: a distributed supervisor that we can use to distribute and maintain healthy pipelines on the nodes, and a distributed registry that we use directly to track which pipelines exist and on which nodes they are.

Finally, a PipelineManager component will take care of monitoring RabbitMQ for new queues and starting/stopping corresponding pipelines dynamically across the cluster.

Technical implementation

Let’s initiate a new Elixir app with a supervision tree.

mix new message_pipeline --sup

First, we’ll need to add our library dependencies in mix.exs and run mix deps.get:

defmodule MessagePipeline.MixProject do
use Mix.Project

def project do
[
app: :message_pipeline,
version: "0.1.0",
elixir: "~> 1.17",
start_permanent: Mix.env() == :prod,
deps: deps()
]
end

def application do
[
extra_applications: [:logger],
mod: {MessagePipeline.Application, []}
]
end

defp deps do
[
{:libcluster, "~> 3.3"},
{:broadway, "~> 1.0"},
{:broadway_rabbitmq, "~> 0.7"},
{:google_api_pub_sub, "~> 0.34"},
{:goth, "~> 1.4"},
{:tesla, "~> 1.12"},
{:jason, "~> 1.4"},
{:amqp, "~> 4.0"}
]
end
end

Rolling our own PubSub client

Since we just need to publish messages to Google Cloud PubSub and the API is straightforward, we don’t need fancy libraries here.

Let’s start by adding Goth to the supervision tree to manage Google’s service account credentials.

defmodule MessagePipeline.Application do
use Application

def start(_type, _args) do
credentials =
"GOOGLE_APPLICATION_CREDENTIALS_PATH"
|> System.fetch_env!()
|> File.read!()
|> Jason.decode!()

children = [
{Goth, name: MessagePipeline.Goth, source: {:service_account, credentials}},
# Other children...
]

Supervisor.start_link(children, strategy: :one_for_one)
end
end

And here’s our HTTP client to publish messages to Google Cloud PubSub

defmodule MessagePipeline.GooglePubsub do
@google_pubsub_base_url "https://pubsub.googleapis.com"

def publish_messages(messages) when is_list(messages) do
project_id = System.fetch_env!("GOOGLE_PUBSUB_PROJECT_ID")
topic_name = System.fetch_env!("GOOGLE_PUBSUB_TOPIC")

topic = "projects/#{project_id}/topics/#{topic_name}"

request = %{
messages: Enum.map(messages, &%{data: Base.encode64(&1)})
}

with {:ok, auth_token} <- generate_auth_token(),
client = client(auth_token),
{:ok, response} <- Tesla.post(client, "/v1/#{topic}:publish", request) do
%{body: %{"messageIds" => message_ids}} = response
{:ok, message_ids}
end
end

defp client(auth_token) do
middleware = [
{Tesla.Middleware.BaseUrl, @google_pubsub_base_url},
Tesla.Middleware.JSON,
{Tesla.Middleware.Headers, [{"Authorization", "Bearer " <> auth_token}]}
]

Tesla.client(middleware)
end

defp generate_auth_token do
with {:ok, %{token: token}} <- Goth.fetch(MessagePipeline.Goth) do
{:ok, token}
end
end
end

Clustering with libcluster

We’ll use libcluster to establish communication between our Elixir nodes. Here’s an example configuration that uses the Gossip strategy to form a cluster between nodes:

defmodule MessagePipeline.Application do
use Application

def start(_type, _args) do
topologies = [
gossip_example: [
strategy: Elixir.Cluster.Strategy.Gossip
]
]

children = [
{Cluster.Supervisor, [topologies, [name: MessagePipeline.ClusterSupervisor]]},
# Other children...
]

Supervisor.start_link(children, strategy: :one_for_one)
end
end

Distributed process management with Horde

We’ll use Horde to manage our Broadway pipelines across the cluster. Horde ensures that each pipeline runs on exactly one node and handles redistribution when nodes fail.

Let’s add Horde’s supervisor and registry to the application’s supervision tree.

The UniformQuorumDistribution distribution strategy distributes processes using a hash mechanism among all reachable nodes. In the event of a network partition, it enforces a quorum and will shut down all processes on a node if it is split from the rest of the cluster: the unreachable node is drained and the pipelines can be resumed on the other cluster nodes.

defmodule MessagePipeline.Application do
use Application

def start(_type, _args) do
children = [
{Horde.Registry, [
name: MessagePipeline.PipelineRegistry,
members: :auto,
keys: :unique
]},
{Horde.DynamicSupervisor, [
name: MessagePipeline.PipelineSupervisor,
members: :auto,
strategy: :one_for_one,
distribution_strategy: Horde.UniformQuorumDistribution
]}
# Other children...
]

Supervisor.start_link(children, strategy: :one_for_one)
end
end

Broadway pipeline implementation

Each pipeline uses Broadway to consume messages from RabbitMQ and publish them to Google PubSub.

A strict, per-queue ordering is guaranteed by setting a concurrency of 1.

defmodule MessagePipeline.Pipeline do
use Broadway

alias Broadway.Message

def child_spec(opts) do
queue_name = Keyword.fetch!(opts, :queue_name)
pipeline_name = pipeline_name(queue_name)

%{
id: pipeline_name,
start: {__MODULE__, :start_link, opts}
}
end

def start_link(opts) do
queue_name = Keyword.fetch!(opts, :queue_name)
pipeline_name = pipeline_name(queue_name)

pipeline_opts = [
name: {:via, Horde.Registry, {MessagePipeline.PipelineRegistry, pipeline_name}},
producer: [
module: {
BroadwayRabbitMQ.Producer,
queue: queue_name,
connection: [
host: System.fetch_env!("RABBITMQ_HOST"),
port: String.to_integer(System.fetch_env!("RABBITMQ_PORT")),
username: System.fetch_env!("RABBITMQ_USER"),
password: System.fetch_env!("RABBITMQ_PASSWORD")
]
},
concurrency: 1
],
processors: [
default: [
concurrency: 1
]
],
batchers: [
default: [
batch_size: 100,
batch_timeout: 200,
concurrency: 1
]
]
]

case Broadway.start_link(__MODULE__, pipeline_opts) do
{:ok, pid} ->
{:ok, pid}

{:error, {:already_started, _pid}} ->
:ignore
end
end

def pipeline_name(queue_name) do
String.to_atom("pipeline_#{queue_name}")
end

@impl true
def handle_message(_, message, _) do
message
|> Message.update_data(&process_data/1)
end

@impl true
def handle_batch(_, messages, _, _) do
case publish_to_pubsub(messages) do
{:ok, _message_ids} -> messages
{:error, reason} ->
# Mark messages as failed
Enum.map(messages, &Message.failed(&1, reason))
end
end

defp process_data(data) do
# Transform message data as needed
data
end

defp publish_to_pubsub(messages) do
MessagePipeline.GooglePubsub.publish_messages(messages)
end
end

Queue discovery and pipeline management

Finally, we need a process to monitor RabbitMQ queues and ensure pipelines are running for each one.

The Pipeline Manager periodically queries RabbitMQ for existing queues. If a new queue appears, it starts a Broadway pipeline only if one does not already exist in the cluster. If a queue is removed, the corresponding pipeline is shut down.

defmodule MessagePipeline.PipelineManager do
use GenServer

@timeout :timer.minutes(1)

def start_link(opts) do
GenServer.start_link(__MODULE__, opts, name: __MODULE__)
end

def init(_opts) do
state = %{managed_queues: MapSet.new()}

{:ok, state, {:continue, :start}}
end

def handle_continue(:start, state) do
state = manage_queues(state)

{:noreply, state, @timeout}
end

def handle_info(:timeout, state) do
state = manage_queues(state)

{:noreply, state, @timeout}
end

def manage_queues(state) do
{:ok, new_queues} = discover_queues()
current_queues = state.managed_queues

queues_to_add = MapSet.difference(new_queues, current_queues)
queues_to_remove = MapSet.difference(current_queues, new_queues)

Enum.each(queues_to_add, &start_pipeline/1)
Enum.each(queues_to_remove, &stop_pipeline/1)

%{state | managed_queues: new_queues}
end

defp discover_queues do
{:ok, conn} =
AMQP.Connection.open(
host: System.fetch_env!("RABBITMQ_HOST"),
port: String.to_integer(System.fetch_env!("RABBITMQ_PORT")),
username: System.fetch_env!("RABBITMQ_USER"),
password: System.fetch_env!("RABBITMQ_PASSWORD")
)
{:ok, chan} = AMQP.Channel.open(conn)
{:ok, queues} = AMQP.Queue.list(chan)

# Filter out system queues
queues
|> Enum.reject(fn %{name: name} ->
String.starts_with?(name, "amq.") or
String.starts_with?(name, "rabbit")
end)
|> Enum.map(& &1.name)
|> MapSet.new()
end

defp start_pipeline(queue_name) do
pipeline_name = MessagePipeline.Pipeline.pipeline_name(queue_name)

case Horde.Registry.lookup(MessagePipeline.PipelineRegistry, pipeline_name) do
[{pid, _}] ->
{:error, :already_started}
[] ->
Horde.DynamicSupervisor.start_child(
MessagePipeline.PipelineSupervisor,
{MessagePipeline.Pipeline, queue_name: queue_name}
)
end
end

defp stop_pipeline(queue_name) do
pipeline_name = MessagePipeline.Pipeline.pipeline_name(queue_name)

case Horde.Registry.lookup(MessagePipeline.PipelineRegistry, pipeline_name) do
[{pid, _}] ->
Horde.DynamicSupervisor.terminate_child(MessagePipeline.PipelineSupervisor, pid)
[] ->
{:error, :not_found}
end
end
end

Let’s not forget to also add the pipeline manager to the application’s supervision tree.

defmodule MessagePipeline.Application do
use Application

def start(_type, _args) do
children = [
{MessagePipeline.PipelineManager, []}
# Other children...
]

Supervisor.start_link(children, strategy: :one_for_one)
end
end

Test the system

We should now have a working and reliable system. To quickly test it out, we can configure a local RabbitMQ broker, a Google Cloud PubSub topic, and finally a couple of Elixir nodes to verify that distributed pipelines are effectively run to forward messages between RabbitMQ queues and PubSub.

Let’s start by running RabbitMQ with the management plugin. RabbitMQ will listen for connections on the 5672 port, while also exposing the management interface at http://localhost:15672. The default credentials are guest/guest.

docker run -d --name rabbitmq \
-p 5672:5672 \
-p 15672:15672 \
rabbitmq:3-management

Next, install and use the gcloud CLI to create a Google Cloud project, a PubSub topic, and a a service account to access PubSub programmatically.

# Login to Google Cloud
gcloud auth login

# Create a new project (or use an existing one)
gcloud projects create message-pipeline-test
gcloud config set project message-pipeline-test

# Enable PubSub API
gcloud services enable pubsub.googleapis.com

# Create a topic
gcloud pubsub topics create test-topic

# Create service account for local testing
gcloud iam service-accounts create local-test-sa

# Generate and download credentials
gcloud iam service-accounts keys create ./google-credentials.json \
--iam-account local-test-sa@message-pipeline-test.iam.gserviceaccount.com

# Grant publish permissions
gcloud pubsub topics add-iam-policy-binding test-topic \
--member="serviceAccount:local-test-sa@message-pipeline-test.iam.gserviceaccount.com" \
--role="roles/pubsub.publisher"

It’s now time to start two terminal sessions where we can export the needed environment variables before running two Elixir nodes.

# In terminal 1

export RABBITMQ_HOST=localhost
export RABBITMQ_PORT=5672
export RABBITMQ_USER=guest
export RABBITMQ_PASSWORD=guest
export GOOGLE_PUBSUB_PROJECT_ID=message-pipeline-test
export GOOGLE_PUBSUB_TOPIC=test-topic
export GOOGLE_APPLICATION_CREDENTIALS_PATH=$(pwd)/google-credentials.json
export RELEASE_COOKIE=my-secret-cookie

iex --sname node1 -S mix

# In terminal 2 (same variables)

export RABBITMQ_HOST=localhost
export RABBITMQ_PORT=5672
export RABBITMQ_USER=guest
export RABBITMQ_PASSWORD=guest
export GOOGLE_PUBSUB_PROJECT_ID=message-pipeline-test
export GOOGLE_PUBSUB_TOPIC=test-topic
export GOOGLE_APPLICATION_CREDENTIALS_PATH=$(pwd)/google-credentials.json
export RELEASE_COOKIE=my-secret-cookie

iex --sname node2 -S mix

To verify cluster formation, from one of the nodes:

# List all nodes in the cluster, should show the other node
Node.list()

# Check Horde supervisor distribution
:sys.get_state(MessagePipeline.PipelineSupervisor)

Now let’s create some test queues on RabbitMQ and start publishing some messages.

# Download rabbitmqadmin if not already available
wget http://localhost:15672/cli/rabbitmqadmin
chmod +x rabbitmqadmin

# Create queues
./rabbitmqadmin declare queue name=test-queue-1
./rabbitmqadmin declare queue name=test-queue-2

# Publish test messages
./rabbitmqadmin publish routing_key=test-queue-1 payload="Message 1 for queue 1"
./rabbitmqadmin publish routing_key=test-queue-1 payload="Message 2 for queue 1"
./rabbitmqadmin publish routing_key=test-queue-2 payload="Message 1 for queue 2"

# List queues and their message counts
./rabbitmqadmin list queues name messages_ready messages_unacknowledged

# Get messages (without consuming them)
./rabbitmqadmin get queue=test-queue-1 count=5 ackmode=reject_requeue_true

One can also use the RabbitMQ management interface at http://localhost:15672, authenticate with the guest/guest default credentials, go to the “Queues” tab, click “Add a new queue”, and create “test-queue-1” and “test-queue-2”.

After a minute, the Elixir nodes should automatically start some pipelines corresponding to the RabbitMQ queues.

# List all registered pipelines
Horde.Registry.select(MessagePipeline.PipelineRegistry, [{{:"$1", :"$2", :"$3"}, [], [:"$2"]}])

# Check specific pipeline
pipeline_name = :"pipeline_test-queue-1"
Horde.Registry.lookup(MessagePipeline.PipelineRegistry, pipeline_name)

Now, if we publish messages on the RabbitMQ queues, we should see them appear on the PubSub topic.

We can verify it from Google Cloud Console, or by creating a subscription, publishing some messages on RabbitMQ, and then pulling messages from the PubSub subscription.

gcloud pubsub subscriptions create test-sub --topic test-topic
# ...Publish messages on RabbitMQ queues...
gcloud pubsub subscriptions pull test-sub --auto-ack

If we stop one of the Elixir nodes (Ctrl+C twice in its IEx session) to simulate a failure, the pipelines should be redistributed in the remaining node:

# Check updated node list
Node.list()

# Check pipeline distribution
Horde.Registry.select(MessagePipeline.PipelineRegistry, [{{:"$1", :"$2", :"$3"}, [], [:"$2"]}])

Rebalancing pipelines on new nodes

With our current implementation, pipelines are automatically redistributed when a node fail but they are not redistributed when a new node joins the cluster.

Fortunately, Horde supports precisely this functionality from v0.8+, and we don’t have to manually stop and re-start our pipelines to have them landing on other nodes.

All we need to do is enable the option process_distribution: :active on Horde’s supervisor to automatically rebalance processes on node joining / leaving. The option runs each child spec through the choose_node/2 function of the preferred distribution strategy, detects which processes should be running on other nodes considering the new cluster configuration, and specifically restarts those particular processes such that they run on the correct node.

defmodule MessagePipeline.Application do
use Application

def start(_type, _args) do
children = [
{Horde.Registry, [
name: MessagePipeline.PipelineRegistry,
members: :auto,
keys: :unique
]},
{Horde.DynamicSupervisor, [
name: MessagePipeline.PipelineSupervisor,
members: :auto,
strategy: :one_for_one,
distribution_strategy: Horde.UniformQuorumDistribution,
process_redistribution: :active
]}
# Other children...
]

Supervisor.start_link(children, strategy: :one_for_one)
end
end

Conclusion

This architecture provides a robust solution for processing ordered message streams at scale. The combination of Elixir’s distributed capabilities, Broadway’s message processing features, and careful coordination across nodes enables us to build a system that can handle high throughput while maintaining message ordering guarantees.

To extend this solution for your specific needs, consider these enhancements:

  • Adopt a libcluster strategy suitable for a production environment, such as Kubernetes.
  • Tune queue discovery latency, configuring the polling interval based on how frequently new queues are created. Better yet, instead of polling RabbitMQ, consider setting up RabbitMQ event notifications to detect queue changes in real-time.
  • Declare AMQP queues as durable and make sure that publishers mark published messages as persisted, in order to survive broker restarts and improve delivery guarantees. Use publisher confirms to ensure messages are safely received by the broker. Deploy RabbitMQ in a cluster with queue mirroring or quorum queues for additional reliability.
  • Add monitoring, instrumenting Broadway and Horde with Telemetry metrics.
  • Enhance error handling and retry mechanisms. For example, retry message publication to PubSub N times before failing the messages, thus invalidating the (possibly costly) processing operation.
  • Unit & e2e testing. Consider that the gcloud CLI (gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators) contains a PubSub emulator that may come in handy: e.g. gcloud beta emulators pubsub start — project=test-project — host-port=0.0.0.0:8085
  • Leverage an HorizontalPodAutoscaler for automated scaling on Kubernetes environments based on resource demand.
  • Evaluate the use of Workload Identities if possible. For instance, you can provide your workloads with access to Google Cloud resources by using federated identities instead of a service account key. This approach frees you from the security concerns of manually managing service account credentials.

Thursday, 13 February 2025

Kaidan supports Qt 6 now! In addition, this release improves the user interface and fixes some bugs. Have a look at the changelog for more details.

Changelog

Features:

  • Highlight public XMPP provider titles while card is expanded (melvo)
  • Round corners of cards and buttons (melvo)
  • Add fading in/out hover effect to map previews (melvo)
  • Collapse contact profiles by default if they have more than 3 entries (melvo)
  • Show colored check mark for delivered messages instead of none to avoid message bubble resizing (melvo)

Bugfixes:

  • Fix opening public MUC-based group chats via another XMPP client (melvo)
  • Fix playing voice messages and changing playing position (melvo)
  • Fix updating message reactions that could not be sent instead of adding them a second time (melvo)
  • Fix updating group chat users in user interface (melvo)
  • Fix displaying message reaction details (melvo)
  • Update filtering contacts by labels even if label list is not open anymore (melvo)
  • Fix scrolling media overview (melvo)
  • Fix updating draft messages (melvo)

Notes:

  • Kaidan requires Qt 6.6 now (mlaurent, melvo, fazevedo, plata)

Download

Or install Kaidan for your distribution:

Packaging status

PySide6, Qt for Python

In this article, we’re going to have a look at Qt for Python, how it integrates with Qt Creator, and why you might want to consider using it, too.

Continue reading PySide6, Qt for Python at basysKom GmbH.

Monday, 10 February 2025

This is the release schedule the release team agreed on

https://community.kde.org/Schedules/KDE_Gear_25.04_Schedule

Dependency freeze is in around 3 weeks (March 6) and feature freeze one
after that. Get your stuff ready!
 

This week, I focused on integrating the Monte Carlo Tree Search (MCTS) algorithm into the MankalaEngine. The primary goal was to test the performance of the MCTS-based agent against various existing algorithms in the engine. Let's dive into what MCTS is, how it works, and what I discovered during the testing phase.

What is Monte Carlo Tree Search (MCTS)?

The Monte Carlo Tree Search (MCTS) is a heuristic search algorithm used for decision-making in sequential decision problems. It incrementally builds a search tree and simulates multiple random moves at each step to evaluate potential outcomes. These simulations help the algorithm determine the most promising move to make.

How Does MCTS Work?

MCTS operates through four key steps:

1. Selection

The algorithm starts at the root node (representing the current game state) and traverses down the tree to a leaf node (an unexplored state). During this process, the algorithm selects child nodes using a specific strategy.

A popular strategy for node selection is Upper Confidence Bounds for Trees (UCT). The UCT formula helps balance exploration and exploitation by selecting nodes based on the following equation:

UCT = mean + C × sqrt(ln(N) / n)

Where:

  • mean is the average reward (or outcome) of a node.
  • N is the total number of simulations performed for the parent node.
  • n is the number of simulations performed for the current child node.
  • C is a constant that controls the level of exploration.

2. Expansion

Once the algorithm reaches a leaf node, it expands the tree by adding one or more child nodes representing potential moves or decisions that can be made from the current state.

3. Simulation

The algorithm then performs a simulation or rollout from the newly added child node. During this phase, the algorithm randomly plays out a series of moves (typically following a simple strategy) until the game reaches a terminal state (i.e., win, loss, or draw).

This is where the Monte Carlo aspect of MCTS shines. By simulating many random games, the algorithm gains insights into the likely outcomes of different actions.

4. Backpropagation

After the simulation ends, the results are propagated back up the tree, updating the nodes with the outcome of the simulation. This allows the algorithm to adjust the expected rewards of the parent nodes based on the result of the child node’s simulation.

read more here - MCTS

Implementing MCTS in C++ for MankalaEngine

With a solid understanding of the algorithm, I began implementing MCTS in C++. The initial step involved integrating the MCTS logic into the benchmark utility of the MankalaEngine. After resolving a series of issues and running multiple tests, the code was functioning as expected.

Testing Results

I compared the performance of the MCTS agent against other existing agents in the MankalaEngine, such as Minimax, MTDF, and Random agents. Here’s how the MCTS agent performed:

Random Agent (Player 1) vs. MCTS (Player 2)

  • MCTS won 80% of the time

MCTS (Player 1) vs. Random Agent (Player 2)

  • MCTS won 60% of the time

MCTS vs. Minimax & MTDF

  • Unfortunately, MCTS consistently lost against both Minimax and MTDF agents. 😞

Key Improvements for MCTS

While MCTS performed well against the Random Agent, there is still room for improvement, especially in its simulation phase. Currently, the algorithm uses a random policy for simulations, which can be inefficient. To improve performance, we can:

  • Use more efficient simulation policies that simulate only promising moves, rather than randomly selecting moves.
  • At the start of the Selection step, focus on moves that have historically been good opening strategies (this requires further research to identify these moves, especially in Pallanguli).
  • Fine-tune the exploration-exploitation balance to improve decision-making.

Upcoming Tasks

In the upcoming week, I plan to:

  • Write test cases for the Pallanguli implementation.
  • Review the implementation of Pallanguli.
Moving KDE's styling into the future

Image
A screenshot of Kate, NeoChat, Plasma and KClock
Four applications, four different ways of styling.

Last year during Akademy I gave a talk called Union: The Future of Styling in KDE?!. In this talk I presented a problem: We currently have four ways of styling our applications. Not only that, but some of these approaches are quite hard to work with, especially for designers who lack programming skills. This all leads to it being incredibly hard to make changes to our application styling currently, which is not only a problem for something like the Plasma Next Initiative, but even smaller changes take a lot of effort.

This problem is not new; we already identified it several years ago. Unfortunately, it also is not easy to solve. Some of the reasons it got to this state are simply inertia. Some things like Plasma's SVG styling were developed as a way to improve styling in an era where a lot of the technologies we currently use did not exist yet. The solutions developed in those days have now existed for a pretty long time so we cannot suddenly drop them. Other reasons are more technical in nature, such as completely different rendering stacks.

Introducing Union

Those different rendering stacks are actually one of the core issues that makes this hard to solve. It means that we cannot simply use the same rendering code for everything, but have to come up with a tricky compatibility layer to make that work. This is what we currently do, and while it works, it means we need to maintain said compatibility layer. It also means we are not utilizing the rendering stack to its full potential.

However, there is another option, which is to take a step back and realise that we actually may not even want to share the rendering code, given that they are quite different. Instead, we need a description of what the element should look like, and then we can have specific rendering code that implements how to render that in the best way for a certain technology stack.

This idea is at the core of a project I called Union, which is a styling system intended to unify all our separate approaches into a single unified styling engine that can support all the different technologies we use for styling our applications.

Image
A diagram showing the three separate parts of Union
The three separate parts of Union

Union consists of three parts: an input layer, an intermediate layer and an output layer. The input layer consists of plugins that can read and interpret some input file format containing a style description and turn it into a more abstract desciption of what to render. How to do that is defined by the middle intermediate layer, which is a library containing the description of the data model and a method of defining which elements to apply things to. Finally, the output layer consists of plugins that use the data from the intermediate layer and turn it into actual rendering commands, as needed for a specific rendering stack.

Implementing Things

This sounds nice on paper, but implementing it is easier said than done. For starters, everything depends on the intermediate layer being both flexible enough to handle varying use cases but at the same time rigid enough that it becomes hard to - intentionally or unintentionally - create dependencies between the input and output layers. Apart from that, replacing the entire styling stack is simply going to be a lot of work.

Image
A screenshot of Inkscape showing some of a Plasma SVG theme
Plasma's SVG styling uses specially-marked SVG items for styling.

To allow us to focus more on the core we needed to break things down into more manageable parts. We chose to focus on the intermediate layer first, by using Plasma's SVG themes as an input format and a QtQuick Style as output. This means we are working with an input format that we already know how to deal with. It also means we have a clear picture of what the output should look like, as it should ultimately look just like how Plasma looks.

At this point, a lot of this work has now been done. While Union does not yet implement a full QtQuick style, it implements most of the basic controls to allow something such as Discover to run without looking completely alien. Focusing on the intermediate layer proved very useful, we encountered and managed to solve several pretty tricky technical issues that would have been even trickier if we did not know what things should look like.

Image
A screenshot of Plasma Discover running with the Union style
Plasma Discover running using Union.

Union Needs You!

All that said, there is still a lot to be done. For starters, to be an actual unified styling system for KDE we need a QtWidgets implementation. Some work on that has started, but it is going to be a lot harder than the QtQuick implementation. We also need a different input format. While Plasma's SVG styling works, it is not ideal for developing new styles with. I would personally like to investigate using CSS as input format as it has most of what we need while also being familiar to a lot of people. Unfortunately, finding a good CSS parser library turns out to be quite hard.

However, at this stage we are at a point where we have multiple tasks that can be done in parallel. This means it is now at a point where it would be great if we had more people developing code, as well as some initial testing and feedback on the systen. If you are interested in helping out, the code can be found at invent.kde.org/plasma/union. There is also a Matrix channel for more realtime disucssions.

Discuss this article on KDE Discuss.

ahiemstra

Thursday, 6 February 2025

Glaxnimate 0.6.0 Beta has finally been released for testing!

It has been a while since the last release of Glaxnimate, but in the background we worked hard to make this first release under the KDE umbrella happen!

Please help us testing and report any issue you may encounter on https://bugs.kde.org/enter_bug.cgi?product=glaxnimate

Glaxnimate joins KDE

The Glaxnimate team is proud to announce Glaxnimate is now part of KDE. Glaxnimate benefits from the shared KDE build and distribution infrastructure, the collective knowledge of the community and libraries such as KDE Frameworks. This way the developers can spend more time on the code to fix bugs and develop new features for you!

Changes

Editing

  • The rotation handle now preserves rotation direction and multiple full rotations
  • Alt + click on keyframes cycles between built-in easing curves
  • Alt + click on bezier points cycles between tangent symmetry modes (Ctrl+click still works)
  • Changing a bezier point from corner to smooth will add tangents if they are missing
  • The import image dialog now allows importing multiple images at once

I/O

  • Added support for SVG text-anchor

User Interface

  • Middle mouse drag now pans the timeline
  • There is an icon on the timeline to quickly toggle keyframes
  • Buttons to jump to the next/previous keyframe in the timeline
  • Improved LottieFiles import dialog
  • Improved autosave recovery process
  • Script console now supports basic autocompletion

Scripting

  • Exposed method to add new compositions

Misc

  • Switched to an even/odd version numbering scheme
  • Integration with KDE Frameworks

Bug Fixes

  • Fixed keyframe context menu showing the wrong "after" transition
  • When drawing bezier points that don't have tangents are correctly marked as corner
  • The play button now resumes from the current frame rather than resetting to the start
  • Fixed saving custom templates
  • Toggling visibility / lock of a layer by clicking on its icon now adds an undo/redo action
  • Fixed LottieFiles import
  • Fixed dropping file as object
  • Fixed closing compositions from the tab bar
  • Fixed loading colors from older lotties
  • Shape modifiers marked as not visible are now correctly ignored
  • Fixed rendering of round corners modifier
  • Fixed "New Composition" action creating an invisible layer
  • Fixed repeater opacity not being applied correctly
  • Improved handling of repeater with stroke
  • Fixed SVG animation export
  • Fixed animated raster plugin I/O

How to get it

Note that this is a beta release. Most Linux distributions do not package unstable releases.

We recommend to test this release with one of the binaries we provide:

Packager Section

The source code tarball are available from the KDE servers:

URL: https://download.kde.org/unstable/glaxnimate/0.5.80

Source: glaxnimate-0.5.80.tar.xz

Signed by: 97B71AA02D63EA6C5C44C23B962AC48EF0501F0B Julius Künzel julius.kuenzel@kde.org

Monday, 3 February 2025

KStars v3.7.5 is released on 2025.02.03 for Windows, MacOS & Linux. It's a bi-monthly bug-fix release with a couple of exciting features. Here are the release notes organized by developer.

Jasem Mutlaq

  • Added dome slit visualization on sky map. Specify the Dome Measurement parameters in the INDI Dome driver to see a live dome slit overlay in the Sky Map.

  • Implemented generic DBus methods for KStars options
  • Added SchedulerSleeping event
  • Added mutex protection for multi-threaded resources
  • Enhanced scheduler loading and settings management
  • Improved filter manager operations
  • Fixed video subframing. Up to 50x improvement in subframed video feeds.
  • Fixed multiple profile editor issues
  • Added VSCode development setup support

Hy Murveit

  • Fixed DMS delta angle calculation
  • Added mandatory settle to PAA
  • Improved imaging planner stability


  • Fixed pierside placeholder directory usage
  • Added START_AT scheduler test
  • Fixed Abell planetary nebula lookup
  • Enhanced PAA adjustment estimation

Wolfgang Reissenberger

  • Implemented video sequence capture. Preliminary support for capturing Video files as regular sequences in the Capture module. Great news for EAA.

  • Fixed focus options
  • Improved remote directory handling
  • Fixed flats with wall position
  • Enhanced filter wheel integration

John Evans

  • Enabled focuser controls when camera disconnected
  • Improved focus measure framing
  • Fixed focus advisor code warnings
  • Updated aberration inspector functionality

Toni Schriber

  • Fixed overshooting cosine in CachingDms calculation
  • Implemented calibration reuse after rotation. Guide calibration data can now be re-used between sessions after rotation.

Ben Cooksley

  • Removed CMake trace/debug logs from CI runs

György Balló

  • Set window icon

Oliver Kellogg

  • Fixed typo in FITS Viewer configuration

Akarsh Simha

  • Fixed right-click popup menu on deep stars

Technical Highlights

  • Improved capture sequence stability: Set 5-minute timeout for transient operations (dome motion, mount parking/unparking, dust cap operations, focusing, filter wheel changes) to prevent indefinite sequence stalling.
  • Improved mount rotation processing
  • Enhanced scheduler loading mechanism
  • Added mutex protection for multi-threaded resources
  • Improved capture operation timeout handling

Sunday, 2 February 2025

A new Craft cache has just been published. The update is already available for KDE's CD, CI (Windows/Android) will follow in the next days.

Please note that this only applies to the Qt6 cache. The Qt5 cache is in LTS mode since April 2024 and does not recieve major updates anymore. We highly recommend to port your Qt5 app packaged by Craft to Qt6 as soon as possible!

Changes (highlights)

General

We added CI for flake8, isort and black with the help of tox (which makes it easy to run them locally too) to all Craft repositories. To be able to do so we did a lot of best pratice cleanup beforehand like eg. removing star imports.

Craft Core

  • Drop support for MSVC 2017
  • Introduced a CraftBool helper. This allows handy things like self.subinfo.options.dynamic.withMyLib.asOnOff instead of 'ON' if self.subinfo.options.dynamic.withMyLib else 'OFF'
  • Fix: let the Meson build system respect the buildStatic option
  • Handle --enable-static --enable-shared in AutoToolsPackageBase instead of in every single blueprint
  • Python packages (Linux and Windows MSVC; macOS is work in progress):
    • Build them ourself instead of using the pre-build binaries from pypi.org
    • Use proper staging
    • Allow to deploy/package them
  • Properly set Craft env when branch is switched (eg. with Craft Master in CI)

Blueprints

  • libjpeg-turbo 3.0.3
  • Multiple fixes for build of shared vs. static libs
  • libvpx 1.15.0
  • Add minGW 14.2 (not the default yet!)
  • 7z 24.09
  • KShimgen 0.6.1
  • linuxdeploy-plugin-qt 2.0.0-alpha-1-20250119
  • qtkeychain 0.15.0

About KDE Craft

KDE Craft is an open source meta-build system and package manager. It manages dependencies and builds libraries and applications from source on Windows, macOS, Linux, FreeBSD and Android.

Learn more on https://community.kde.org/Craft or join the Matrix room #kde-craft:kde.org