Modern distributed systems need to process massive amounts of data efficiently while maintaining strict ordering guarantees. This is especially challenging when scaling horizontally across multiple nodes. How do we ensure messages from specific sources are processed in order while still taking advantage of parallelism and fault tolerance?

Elixir, with its robust concurrency model and distributed computing capabilities, is well-suited for solving this problem. In this article, we’ll build a scalable, distributed message pipeline that:

• Bridges RabbitMQ and Google Cloud PubSub, delivering messages from RabbitMQ queues to a PubSub topic.
• Ensures message ordering for each RabbitMQ queue.
• Scales horizontally across multiple nodes.
• Distribute the message pipelines evenly across the Elixir cluster.
• Gracefully handles failures and network partitions.

Many modern applications require processing large volumes of data while preserving message order from individual sources. Consider, for example, IoT systems where sensor readings must be processed in sequence, or multi-tenant applications where each tenant’s data requires sequential processing.

The solution we’ll build addresses these requirements by treating each RabbitMQ queue as an ordered data source.

Let’s explore how to design this system using Elixir’s distributed computing libraries: Broadway, Horde, and libcluster.

Architecture overview

The system consists of multiple Elixir nodes forming a distributed cluster. Each node runs one or more Broadway pipelines to process messages from RabbitMQ queues and forward them to Google Cloud PubSub. To maintain message ordering, each queue has exactly one pipeline instance running across the cluster at any time. If a node fails the system must redistribute its pipelines to other nodes automatically, and if a new node joins the cluster then the existing pipelines should be redistributed to ensure a balanced load.

Elixir natively supports the ability to cluster multiple nodes together so that processes and distributed components within the cluster can communicate seamlessly. We will employ the libcluster library since it provides several strategies to automatize cluster formation and healing.

For the data pipelines, the Broadway library provides a great framework to support multi-stage data processing while handling back-pressure, batching, fault tolerance and other good features.

To correctly maintain the distribution of data pipelines across the Elixir nodes, the Horde library comes to the rescue by providing the building blocks we need: a distributed supervisor that we can use to distribute and maintain healthy pipelines on the nodes, and a distributed registry that we use directly to track which pipelines exist and on which nodes they are.

Finally, a PipelineManager component will take care of monitoring RabbitMQ for new queues and starting/stopping corresponding pipelines dynamically across the cluster.

Technical implementation

Let’s initiate a new Elixir app with a supervision tree.

mix new message_pipeline --sup

First, we’ll need to add our library dependencies in mix.exs and run mix deps.get:

defmodule MessagePipeline.MixProject do
  use Mix.Project

  def project do
    [
      app: :message_pipeline,
      version: "0.1.0",
      elixir: "~> 1.17",
      start_permanent: Mix.env() == :prod,
      deps: deps()
    ]
  end

  def application do
    [
      extra_applications: [:logger],
      mod: {MessagePipeline.Application, []}
    ]
  end

  defp deps do
    [
      {:libcluster, "~> 3.3"},
      {:broadway, "~> 1.0"},
      {:broadway_rabbitmq, "~> 0.7"},
      {:google_api_pub_sub, "~> 0.34"},
      {:goth, "~> 1.4"},
      {:tesla, "~> 1.12"},
      {:jason, "~> 1.4"},
      {:amqp, "~> 4.0"}
    ]
  end
end

Rolling our own PubSub client

Since we just need to publish messages to Google Cloud PubSub and the API is straightforward, we don’t need fancy libraries here.

Let’s start by adding Goth to the supervision tree to manage Google’s service account credentials.

defmodule MessagePipeline.Application do
  use Application

  def start(_type, _args) do
    credentials =
      "GOOGLE_APPLICATION_CREDENTIALS_PATH"
      |> System.fetch_env!()
      |> File.read!()
      |> Jason.decode!()

    children = [
      {Goth, name: MessagePipeline.Goth, source: {:service_account, credentials}},
      # Other children...
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

And here’s our HTTP client to publish messages to Google Cloud PubSub

defmodule MessagePipeline.GooglePubsub do
  @google_pubsub_base_url "https://pubsub.googleapis.com"

  def publish_messages(messages) when is_list(messages) do
    project_id = System.fetch_env!("GOOGLE_PUBSUB_PROJECT_ID")
    topic_name = System.fetch_env!("GOOGLE_PUBSUB_TOPIC")

    topic = "projects/#{project_id}/topics/#{topic_name}"

    request = %{
      messages: Enum.map(messages, &%{data: Base.encode64(&1)})
    }

    with {:ok, auth_token} <- generate_auth_token(),
         client = client(auth_token),
         {:ok, response} <- Tesla.post(client, "/v1/#{topic}:publish", request) do
      %{body: %{"messageIds" => message_ids}} = response
      {:ok, message_ids}
    end
  end

  defp client(auth_token) do
    middleware = [
      {Tesla.Middleware.BaseUrl, @google_pubsub_base_url},
      Tesla.Middleware.JSON,
      {Tesla.Middleware.Headers, [{"Authorization", "Bearer " <> auth_token}]}
    ]

    Tesla.client(middleware)
  end

  defp generate_auth_token do
    with {:ok, %{token: token}} <- Goth.fetch(MessagePipeline.Goth) do
      {:ok, token}
    end
  end
end

Clustering with libcluster

We’ll use libcluster to establish communication between our Elixir nodes. Here’s an example configuration that uses the Gossip strategy to form a cluster between nodes:

defmodule MessagePipeline.Application do
  use Application

  def start(_type, _args) do
    topologies = [
      gossip_example: [
        strategy: Elixir.Cluster.Strategy.Gossip
      ]
    ]

    children = [
      {Cluster.Supervisor, [topologies, [name: MessagePipeline.ClusterSupervisor]]},
      # Other children...
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

Distributed process management with Horde

We’ll use Horde to manage our Broadway pipelines across the cluster. Horde ensures that each pipeline runs on exactly one node and handles redistribution when nodes fail.

Let’s add Horde’s supervisor and registry to the application’s supervision tree.

The UniformQuorumDistribution distribution strategy distributes processes using a hash mechanism among all reachable nodes. In the event of a network partition, it enforces a quorum and will shut down all processes on a node if it is split from the rest of the cluster: the unreachable node is drained and the pipelines can be resumed on the other cluster nodes.

defmodule MessagePipeline.Application do
  use Application
  
  def start(_type, _args) do
    children = [
      {Horde.Registry, [
          name: MessagePipeline.PipelineRegistry,
          members: :auto,
          keys: :unique
        ]},
      {Horde.DynamicSupervisor, [
          name: MessagePipeline.PipelineSupervisor,
          members: :auto,
          strategy: :one_for_one,
          distribution_strategy: Horde.UniformQuorumDistribution
        ]}
      # Other children...
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

Broadway pipeline implementation

Each pipeline uses Broadway to consume messages from RabbitMQ and publish them to Google PubSub.

A strict, per-queue ordering is guaranteed by setting a concurrency of 1.

defmodule MessagePipeline.Pipeline do
  use Broadway

  alias Broadway.Message

  def child_spec(opts) do
    queue_name = Keyword.fetch!(opts, :queue_name)
    pipeline_name = pipeline_name(queue_name)

    %{
      id: pipeline_name,
      start: {__MODULE__, :start_link, opts}
    }
  end

  def start_link(opts) do
    queue_name = Keyword.fetch!(opts, :queue_name)
    pipeline_name = pipeline_name(queue_name)

    pipeline_opts = [
      name: {:via, Horde.Registry, {MessagePipeline.PipelineRegistry, pipeline_name}},
      producer: [
        module: {
          BroadwayRabbitMQ.Producer,
          queue: queue_name,
          connection: [
            host: System.fetch_env!("RABBITMQ_HOST"),
            port: String.to_integer(System.fetch_env!("RABBITMQ_PORT")),
            username: System.fetch_env!("RABBITMQ_USER"),
            password: System.fetch_env!("RABBITMQ_PASSWORD")
          ]
        },
        concurrency: 1
      ],
      processors: [
        default: [
          concurrency: 1
        ]
      ],
      batchers: [
        default: [
          batch_size: 100,
          batch_timeout: 200,
          concurrency: 1
        ]
      ]
    ]
    
    case Broadway.start_link(__MODULE__, pipeline_opts) do
      {:ok, pid} ->
        {:ok, pid}

      {:error, {:already_started, _pid}} ->
        :ignore
    end
  end

  def pipeline_name(queue_name) do
    String.to_atom("pipeline_#{queue_name}")
  end

  @impl true
  def handle_message(_, message, _) do
    message
    |> Message.update_data(&process_data/1)
  end

  @impl true
  def handle_batch(_, messages, _, _) do
    case publish_to_pubsub(messages) do
      {:ok, _message_ids} -> messages
      {:error, reason} -> 
        # Mark messages as failed
        Enum.map(messages, &Message.failed(&1, reason))
    end
  end

  defp process_data(data) do
    # Transform message data as needed
    data
  end

  defp publish_to_pubsub(messages) do
    MessagePipeline.GooglePubsub.publish_messages(messages)
  end
end

Queue discovery and pipeline management

Finally, we need a process to monitor RabbitMQ queues and ensure pipelines are running for each one.

The Pipeline Manager periodically queries RabbitMQ for existing queues. If a new queue appears, it starts a Broadway pipeline only if one does not already exist in the cluster. If a queue is removed, the corresponding pipeline is shut down.

defmodule MessagePipeline.PipelineManager do
  use GenServer

  @timeout :timer.minutes(1)

  def start_link(opts) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  def init(_opts) do
    state = %{managed_queues: MapSet.new()}

    {:ok, state, {:continue, :start}}
  end

  def handle_continue(:start, state) do
    state = manage_queues(state)

    {:noreply, state, @timeout}
  end

  def handle_info(:timeout, state) do
    state = manage_queues(state)

    {:noreply, state, @timeout}
  end

  def manage_queues(state) do
    {:ok, new_queues} = discover_queues()
    current_queues = state.managed_queues

    queues_to_add = MapSet.difference(new_queues, current_queues)
    queues_to_remove = MapSet.difference(current_queues, new_queues)

    Enum.each(queues_to_add, &start_pipeline/1)
    Enum.each(queues_to_remove, &stop_pipeline/1)

    %{state | managed_queues: new_queues}
  end

  defp discover_queues do
    {:ok, conn} =
      AMQP.Connection.open(
        host: System.fetch_env!("RABBITMQ_HOST"),
        port: String.to_integer(System.fetch_env!("RABBITMQ_PORT")),
        username: System.fetch_env!("RABBITMQ_USER"),
        password: System.fetch_env!("RABBITMQ_PASSWORD")
      )
    {:ok, chan} = AMQP.Channel.open(conn)
    {:ok, queues} = AMQP.Queue.list(chan)

    # Filter out system queues
    queues
    |> Enum.reject(fn %{name: name} -> 
        String.starts_with?(name, "amq.") or
        String.starts_with?(name, "rabbit")
      end)
    |> Enum.map(& &1.name)
    |> MapSet.new()
  end

  defp start_pipeline(queue_name) do
    pipeline_name = MessagePipeline.Pipeline.pipeline_name(queue_name)

    case Horde.Registry.lookup(MessagePipeline.PipelineRegistry, pipeline_name) do
      [{pid, _}] ->
        {:error, :already_started}
      [] ->
        Horde.DynamicSupervisor.start_child(
          MessagePipeline.PipelineSupervisor,
          {MessagePipeline.Pipeline, queue_name: queue_name}
        )
    end
  end

  defp stop_pipeline(queue_name) do
    pipeline_name = MessagePipeline.Pipeline.pipeline_name(queue_name)

    case Horde.Registry.lookup(MessagePipeline.PipelineRegistry, pipeline_name) do
      [{pid, _}] ->
        Horde.DynamicSupervisor.terminate_child(MessagePipeline.PipelineSupervisor, pid)
      [] ->
        {:error, :not_found}
    end
  end
end

Let’s not forget to also add the pipeline manager to the application’s supervision tree.

defmodule MessagePipeline.Application do
  use Application

  def start(_type, _args) do
    children = [
      {MessagePipeline.PipelineManager, []}
      # Other children...
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

Test the system

We should now have a working and reliable system. To quickly test it out, we can configure a local RabbitMQ broker, a Google Cloud PubSub topic, and finally a couple of Elixir nodes to verify that distributed pipelines are effectively run to forward messages between RabbitMQ queues and PubSub.

Let’s start by running RabbitMQ with the management plugin. RabbitMQ will listen for connections on the 5672 port, while also exposing the management interface at http://localhost:15672. The default credentials are guest/guest.

docker run -d --name rabbitmq \
  -p 5672:5672 \
  -p 15672:15672 \
  rabbitmq:3-management

Next, install and use the gcloud CLI to create a Google Cloud project, a PubSub topic, and a a service account to access PubSub programmatically.

# Login to Google Cloud
gcloud auth login

# Create a new project (or use an existing one)
gcloud projects create message-pipeline-test
gcloud config set project message-pipeline-test

# Enable PubSub API
gcloud services enable pubsub.googleapis.com

# Create a topic
gcloud pubsub topics create test-topic

# Create service account for local testing
gcloud iam service-accounts create local-test-sa

# Generate and download credentials
gcloud iam service-accounts keys create ./google-credentials.json \
  --iam-account local-test-sa@message-pipeline-test.iam.gserviceaccount.com

# Grant publish permissions
gcloud pubsub topics add-iam-policy-binding test-topic \
  --member="serviceAccount:local-test-sa@message-pipeline-test.iam.gserviceaccount.com" \
  --role="roles/pubsub.publisher"

It’s now time to start two terminal sessions where we can export the needed environment variables before running two Elixir nodes.

# In terminal 1

export RABBITMQ_HOST=localhost
export RABBITMQ_PORT=5672
export RABBITMQ_USER=guest
export RABBITMQ_PASSWORD=guest
export GOOGLE_PUBSUB_PROJECT_ID=message-pipeline-test
export GOOGLE_PUBSUB_TOPIC=test-topic
export GOOGLE_APPLICATION_CREDENTIALS_PATH=$(pwd)/google-credentials.json
export RELEASE_COOKIE=my-secret-cookie

iex --sname node1 -S mix

# In terminal 2 (same variables)

export RABBITMQ_HOST=localhost
export RABBITMQ_PORT=5672
export RABBITMQ_USER=guest
export RABBITMQ_PASSWORD=guest
export GOOGLE_PUBSUB_PROJECT_ID=message-pipeline-test
export GOOGLE_PUBSUB_TOPIC=test-topic
export GOOGLE_APPLICATION_CREDENTIALS_PATH=$(pwd)/google-credentials.json
export RELEASE_COOKIE=my-secret-cookie

iex --sname node2 -S mix

To verify cluster formation, from one of the nodes:

# List all nodes in the cluster, should show the other node
Node.list()

# Check Horde supervisor distribution
:sys.get_state(MessagePipeline.PipelineSupervisor)

Now let’s create some test queues on RabbitMQ and start publishing some messages.

# Download rabbitmqadmin if not already available
wget http://localhost:15672/cli/rabbitmqadmin
chmod +x rabbitmqadmin

# Create queues
./rabbitmqadmin declare queue name=test-queue-1
./rabbitmqadmin declare queue name=test-queue-2

# Publish test messages
./rabbitmqadmin publish routing_key=test-queue-1 payload="Message 1 for queue 1"
./rabbitmqadmin publish routing_key=test-queue-1 payload="Message 2 for queue 1"
./rabbitmqadmin publish routing_key=test-queue-2 payload="Message 1 for queue 2"

# List queues and their message counts
./rabbitmqadmin list queues name messages_ready messages_unacknowledged

# Get messages (without consuming them)
./rabbitmqadmin get queue=test-queue-1 count=5 ackmode=reject_requeue_true

One can also use the RabbitMQ management interface at http://localhost:15672, authenticate with the guest/guest default credentials, go to the “Queues” tab, click “Add a new queue”, and create “test-queue-1” and “test-queue-2”.

After a minute, the Elixir nodes should automatically start some pipelines corresponding to the RabbitMQ queues.

# List all registered pipelines
Horde.Registry.select(MessagePipeline.PipelineRegistry, [{{:"$1", :"$2", :"$3"}, [], [:"$2"]}])

# Check specific pipeline
pipeline_name = :"pipeline_test-queue-1"
Horde.Registry.lookup(MessagePipeline.PipelineRegistry, pipeline_name)

Now, if we publish messages on the RabbitMQ queues, we should see them appear on the PubSub topic.

We can verify it from Google Cloud Console, or by creating a subscription, publishing some messages on RabbitMQ, and then pulling messages from the PubSub subscription.

gcloud pubsub subscriptions create test-sub --topic test-topic
# ...Publish messages on RabbitMQ queues...
gcloud pubsub subscriptions pull test-sub --auto-ack

If we stop one of the Elixir nodes (Ctrl+C twice in its IEx session) to simulate a failure, the pipelines should be redistributed in the remaining node:

# Check updated node list
Node.list()

# Check pipeline distribution
Horde.Registry.select(MessagePipeline.PipelineRegistry, [{{:"$1", :"$2", :"$3"}, [], [:"$2"]}])

Rebalancing pipelines on new nodes

With our current implementation, pipelines are automatically redistributed when a node fail but they are not redistributed when a new node joins the cluster.

Fortunately, Horde supports precisely this functionality from v0.8+, and we don’t have to manually stop and re-start our pipelines to have them landing on other nodes.

All we need to do is enable the option process_distribution: :active on Horde’s supervisor to automatically rebalance processes on node joining / leaving. The option runs each child spec through the choose_node/2 function of the preferred distribution strategy, detects which processes should be running on other nodes considering the new cluster configuration, and specifically restarts those particular processes such that they run on the correct node.

defmodule MessagePipeline.Application do
  use Application
  
  def start(_type, _args) do
    children = [
      {Horde.Registry, [
          name: MessagePipeline.PipelineRegistry,
          members: :auto,
          keys: :unique
        ]},
      {Horde.DynamicSupervisor, [
          name: MessagePipeline.PipelineSupervisor,
          members: :auto,
          strategy: :one_for_one,
          distribution_strategy: Horde.UniformQuorumDistribution,
          process_redistribution: :active
        ]}
      # Other children...
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

Conclusion

This architecture provides a robust solution for processing ordered message streams at scale. The combination of Elixir’s distributed capabilities, Broadway’s message processing features, and careful coordination across nodes enables us to build a system that can handle high throughput while maintaining message ordering guarantees.

To extend this solution for your specific needs, consider these enhancements:

• Adopt a libcluster strategy suitable for a production environment, such as Kubernetes.
• Tune queue discovery latency, configuring the polling interval based on how frequently new queues are created. Better yet, instead of polling RabbitMQ, consider setting up RabbitMQ event notifications to detect queue changes in real-time.
• Declare AMQP queues as durable and make sure that publishers mark published messages as persisted, in order to survive broker restarts and improve delivery guarantees. Use publisher confirms to ensure messages are safely received by the broker. Deploy RabbitMQ in a cluster with queue mirroring or quorum queues for additional reliability.
• Add monitoring, instrumenting Broadway and Horde with Telemetry metrics.
• Enhance error handling and retry mechanisms. For example, retry message publication to PubSub N times before failing the messages, thus invalidating the (possibly costly) processing operation.
• Unit & e2e testing. Consider that the gcloud CLI (gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators) contains a PubSub emulator that may come in handy: e.g. gcloud beta emulators pubsub start — project=test-project — host-port=0.0.0.0:8085
• Leverage an HorizontalPodAutoscaler for automated scaling on Kubernetes environments based on resource demand.
• Evaluate the use of Workload Identities if possible. For instance, you can provide your workloads with access to Google Cloud resources by using federated identities instead of a service account key. This approach frees you from the security concerns of manually managing service account credentials.

The second maintenance release of the 24.12 cycle is out with multiple bug fixes. Notable changes include fixes for crashes, UI resizing issues, effect stack behavior, proxy clip handling, and rendering progress display, along with improvements to Speech-to-text in Flatpak and macOS packages.

For the full changelog continue reading on kdenlive.org.

Kaidan supports Qt 6 now! In addition, this release improves the user interface and fixes some bugs. Have a look at the changelog for more details.

Changelog

Features:

• Highlight public XMPP provider titles while card is expanded (melvo)
• Round corners of cards and buttons (melvo)
• Add fading in/out hover effect to map previews (melvo)
• Collapse contact profiles by default if they have more than 3 entries (melvo)
• Show colored check mark for delivered messages instead of none to avoid message bubble resizing (melvo)

Bugfixes:

• Fix opening public MUC-based group chats via another XMPP client (melvo)
• Fix playing voice messages and changing playing position (melvo)
• Fix updating message reactions that could not be sent instead of adding them a second time (melvo)
• Fix updating group chat users in user interface (melvo)
• Fix displaying message reaction details (melvo)
• Update filtering contacts by labels even if label list is not open anymore (melvo)
• Fix scrolling media overview (melvo)
• Fix updating draft messages (melvo)

Notes:

• Kaidan requires Qt 6.6 now (mlaurent, melvo, fazevedo, plata)

Download

• Source code (.tar.xz) (sig signed with 04EFAD0F7A4D9724)
• Linux (Flatpak on Flathub)

Or install Kaidan for your distribution:

In this article, we’re going to have a look at Qt for Python, how it integrates with Qt Creator, and why you might want to consider using it, too.

Continue reading PySide6, Qt for Python at basysKom GmbH.

Welcome to the @Krita-promo team's January 2025 development and community update.

Development Report

Krita 5.2.9 Released

A new bugfix release is out! Check out the Krita 5.2.9 release post and keep up-to-date with the latest version.

Qt6 Port Progress

Krita can now be compiled (MR!2306) and run (Mastodon post) with Qt6 on Linux, a major milestone on the long road of porting from the outdated Qt5 framework. However, it's still a long way to go to get things working correctly, and it will be some time before any pre-alpha builds are available for the far-off Krita 6.0.

Community Report

January 2025 Monthly Art Challenge Results

For the "Magical Adventure" theme, 14 members submitted 20 original artworks. And the winner is… Magical Adventure by @Mythmaker

The February Art Challenge is Open Now

For the February Art Challenge, @Mythmaker has chosen "Fabulous Flora" as the theme, with the optional challenge of using natural texture. See the full brief for more details, and bring some color into bloom.

Featured Artwork

Best of Krita-Artists - December 2024/January 2025

Nine images were submitted to the Best of Krita-Artists Nominations thread, which was open from December 14th to January 11th. When the poll closed on January 14th, these five wonderful works made their way onto the Krita-Artists featured artwork banner:

Coven Camille | League Of Legends Fan Art by @Dehaf

Still Evening by @MangooSalade

Themis by @DavB

Flying Pig Squadron by @Yaroslavus_Artem

Oniwakamaru and the giant carp by @GioArtworks

Best of Krita-Artists - January/February 2025

Voting is open until February 15th!

Ways to Help Krita

Krita is Free and Open Source Software developed by an international team of sponsored developers and volunteer contributors.

Visit Krita's funding page to see how user donations keep development going, and explore a one-time or monthly contribution. Or check out more ways to Get Involved, from testing, coding, translating, and documentation writing, to just sharing your artwork made with Krita.

The Krita-promo team has put out a call for volunteers, come join us and help keep these monthly updates going.

Notable Changes

Notable changes in Krita's development builds from Jan. 16 - Feb. 12, 2025.

Unstable branch (5.3.0-prealpha):

Bug fixes:

• Blending Modes: Rewrite blending modes to properly support float and HDR colorspaces. (bug report) (Change, by Dmitry Kazakov)
• Brush Engines: Fix Filter Brush engine to work with per- and cross-channel filters. (Change, by Dmitry Kazakov)
• Filters: Screentone: Change default screentone interpolation type to Linear. (Change, by Emmet O'Neill)
• Scripting: Fix Node.paint script functions to use the given node instead of active node. (Change, by Freya Lupen)

Features:

• Text: Load font families as resources and display a preview in the font chooser. (Change, by Wolthera van Hövell)
• Filters: Random Noise: Add grayscale noise option and improve performance. (Change, by Maciej Jesionowski)
• Blending Modes: Add a new HSY blending mode, "Tint", which colorizes and slightly lightens. It's suggested to be used with the Fast Color Overlay filter. (Change 1, Change 2 by Maciej Jesionowski)

Nightly Builds

Pre-release versions of Krita are built every day for testing new changes.

Get the latest bugfixes in Stable "Krita Plus" (5.2.10-prealpha): Linux - Windows - macOS (unsigned) - Android arm64-v8a - Android arm32-v7a - Android x86_64

Or test out the latest Experimental features in "Krita Next" (5.3.0-prealpha). Feedback and bug reports are appreciated!: Linux - Windows - macOS (unsigned) - Android arm64-v8a - Android arm32-v7a - Android x86_64

Tellico 4.1.1 is available, with a few fixes.

Improvements

• Updated Filmaffinity and ComicVine data sources.
• Added configurable image size to Discogs and Moviemeter data sources.
• Fixed ISBN searching with Colnect.

This is the release schedule the release team agreed on

https://community.kde.org/Schedules/KDE_Gear_25.04_Schedule

Dependency freeze is in around 3 weeks (March 6) and feature freeze one
after that. Get your stuff ready!
 

This week, I focused on integrating the Monte Carlo Tree Search (MCTS) algorithm into the MankalaEngine. The primary goal was to test the performance of the MCTS-based agent against various existing algorithms in the engine. Let's dive into what MCTS is, how it works, and what I discovered during the testing phase.

What is Monte Carlo Tree Search (MCTS)?

The Monte Carlo Tree Search (MCTS) is a heuristic search algorithm used for decision-making in sequential decision problems. It incrementally builds a search tree and simulates multiple random moves at each step to evaluate potential outcomes. These simulations help the algorithm determine the most promising move to make.

How Does MCTS Work?

MCTS operates through four key steps:

1. Selection

The algorithm starts at the root node (representing the current game state) and traverses down the tree to a leaf node (an unexplored state). During this process, the algorithm selects child nodes using a specific strategy.

A popular strategy for node selection is Upper Confidence Bounds for Trees (UCT). The UCT formula helps balance exploration and exploitation by selecting nodes based on the following equation:

UCT = mean + C × sqrt(ln(N) / n)

Where:

• mean is the average reward (or outcome) of a node.
• N is the total number of simulations performed for the parent node.
• n is the number of simulations performed for the current child node.
• C is a constant that controls the level of exploration.

2. Expansion

Once the algorithm reaches a leaf node, it expands the tree by adding one or more child nodes representing potential moves or decisions that can be made from the current state.

3. Simulation

The algorithm then performs a simulation or rollout from the newly added child node. During this phase, the algorithm randomly plays out a series of moves (typically following a simple strategy) until the game reaches a terminal state (i.e., win, loss, or draw).

This is where the Monte Carlo aspect of MCTS shines. By simulating many random games, the algorithm gains insights into the likely outcomes of different actions.

4. Backpropagation

After the simulation ends, the results are propagated back up the tree, updating the nodes with the outcome of the simulation. This allows the algorithm to adjust the expected rewards of the parent nodes based on the result of the child node’s simulation.

read more here - MCTS

Implementing MCTS in C++ for MankalaEngine

With a solid understanding of the algorithm, I began implementing MCTS in C++. The initial step involved integrating the MCTS logic into the benchmark utility of the MankalaEngine. After resolving a series of issues and running multiple tests, the code was functioning as expected.

Testing Results

I compared the performance of the MCTS agent against other existing agents in the MankalaEngine, such as Minimax, MTDF, and Random agents. Here’s how the MCTS agent performed:

Random Agent (Player 1) vs. MCTS (Player 2)

• MCTS won 80% of the time

MCTS (Player 1) vs. Random Agent (Player 2)

• MCTS won 60% of the time

MCTS vs. Minimax & MTDF

• Unfortunately, MCTS consistently lost against both Minimax and MTDF agents. 😞

Key Improvements for MCTS

While MCTS performed well against the Random Agent, there is still room for improvement, especially in its simulation phase. Currently, the algorithm uses a random policy for simulations, which can be inefficient. To improve performance, we can:

• Use more efficient simulation policies that simulate only promising moves, rather than randomly selecting moves.
• At the start of the Selection step, focus on moves that have historically been good opening strategies (this requires further research to identify these moves, especially in Pallanguli).
• Fine-tune the exploration-exploitation balance to improve decision-making.

Upcoming Tasks

In the upcoming week, I plan to:

• Write test cases for the Pallanguli implementation.
• Review the implementation of Pallanguli.

Kasts polishing, progress on Krita Qt6 port and Kdenlive fundraising report

Welcome to a new issue of "This Week in KDE Apps"! Every week we cover as much as possible of what's happening in the world of KDE apps. This issue contains change from the last two weeks.

Much happened the past two weeks, we had a successful KDE presence at FOSDEM, we now have a location and date for this year's edition of Linux App Summit (April 25-26, 2025 in Tirana, Albania) and also continued to improve our apps. Let's dive in!

Releases

• KDE Gear 24.12.2 is out with some bugfixes.
• Glaxnimate 0.6.0 beta is out. Glaxnimate is a 2d animation software and 0.6.0 is the first version of Glaxnimate as part of KDE. Checkout Glaxnimate's website.
• KStars 3.7.5 is out with mostly bugfixes and performance improvements.
• GCompris 25.0 is out. This is a big release containing 5 new activities.
• Krita 5.2.9 is out. This is a bug fix release, containing all bugfixes of our bug hunt efforts back in November. Major bug-fixes include fixes to clone-layers, fixes to opacity handling, in particular for file formats like Exr, a number of crash fixes and much more!

Akonadi Background service for KDE PIM apps

We fixed an issue where loading tags was broken and would result in a constant 100% CPU usage. (Carl Schwan, 24.12.3. Link)

Elisa Play local music and listen to online radio

We now correctly re-open Elisa when it was minimized to the system tray. (Pedro Nishiyama, 24.12.2. Link)

Dolphin Manage your files

We made it possible to rename tabs in Dolphin. This action is available in each tab's context menu. This is useful for very long tab names or when it is difficult to identify a tab by a folder's name alone. (ambar chakravartty, 25.04.0. Link)

We also improved the keyboard based selection of items. Typing a letter on the keyboard usually selects the item in the view which starts with that letter. Diacritics are now ignored here, so you will for example be able to press the "U" key to select a file starting with an "Ü". (Thomas Moerschell, 24.12.3. Link)

We changed the three view buttons to a single menu button. (Akseli Lahtinen, 25.04.0. Link)

We made the "Empty Trash" icon red in conformance to our HIG as it is a destructive operation. (Nate Graham, 25.04.0. Link)

We improved getting the information from supported version control systems (e.g. Git). It is now faster and happens earlier. (Méven Car, 25.04.0. Link)

Falkon Web Browser

We added input methods hints to input fields. This is mostly helpful when using different input methods than a traditional keyboard (e.g. a virtual keyboard). (Juraj Oravec. Link)

KDE Itinerary Digital travel assistant

We continued to improve the coverage of Itinerary in Poland. This week we added support for the train operator Polregio, fixed and refactored the extractor for Koleo and rewrote the extractor for PKP-app to support the ticket layouts. (Grzegorz Mu, 24.12.3. Link 1, link 2, and link 3)

We also added support for CitizenM hotel bookings. (Joshua Goins, 24.12.3. Link)

We also started working on an online version of the ticket extractor. A preview is available on Carl's website.

Volker also published a recap of the past two months in Itinerary. This contains also some orthogonal topics like the free software routing service Transitous.

Kasts Podcast application

We fixed the vertical alignment of the queue header. (Joshua Goin, 25.04.0. Link)

We are now using Kirigami.UrlButton for links and Kirigami.SelectableLabel for the text description in the podcast details page to improve visual and behavior consistency with other Kirigami applications. (Joshua Goins, 25.04.0. Link)

We also improved the look of the search bar in the discovery page. It's now properly separated from the rest of the content. (Joshua Goins, 25.04.0. Link)

We added the ability to force the app to mobile/desktop mode. (Bart De Vries, 25.04.0. Link)

We fixed the sort order of the podcasts episodes. (Bart De Vries, 24.12.3. Link)

Finally we made various improvements to our usage of QML in Kasts to use newer QML constructs. This should improve slighly the performance while reducing the technical debt. (Tobias Fella, 25.04.0. Link 1, link 2, link 3, link 4, link 5, and link 6)

Kate Advanced text editor

We fixed some issues with the list of commits displayed in Kate. The highlight color is now correct and the margins consistent. (Leo Ruggeri, 25.04.0. Link)

We improved the diff widget of Kate. The toolbar icon sizes are now the same as other toolbars in Kate. (Leo Ruggeri, 25.04.0. Link)

Kdenlive Video editor

The Kdenlive team published a report about the result of their last fundraising. It contains a huge amount of great improvements, so go read it!

Added checkerboard option in clip monitor background (Julius Künzel, 25.04.0, Link)

Konqueror KDE File Manager & Web Browser

We fixed the handling of the cookie policy when no policy has been explicitly set. (Stefano Crocco, 24.12.3. Link)

Krita Digital Painting, Creative Freedom

The Krita team continued porting Krita to Qt6/KF6. The application now compiles and run with Qt6, but there are still some uni tests not working. Link to Mastodon thread

Ramon published a video about "Memileo Brushes" on YouTube.

KRDC Connect with RDP or VNC to another computer

We implemented the dynamic resolution mode from the remote desktop protocol (RDP). This means we now resize the remote desktop to fit the current KRDC window. This works for Windows >= 8.1. (Fabio Bas, 25.04.0. Link)

We added support for the domain field in the authentication process. (Fabio Fas, 25.04.0. Link)

We adapted the code to work with FreeRDP 3.11. (Fabio Bas, 25.04.0. Link)

Marknote Write down your thoughts

We fixed the list of "Sort Notes List" option not being set by default. (Joshua Goins. Link)

We now properly capitalize the "undo" and "redo" actions. (Joshua Goins. Link)

We removed internal copies of some Kirigami Addons components in Marknote. (Joshua Goins. Link)

Okular View and annotate documents

We added a way to filter the list of certificate to only show certificates for "Qualified Signatures" in the certificate selection. (Sune Vuorela, 25.04.0. Link)

PlasmaTube Watch YouTube videos

We improved the placeholder messages for empty views. (Joshua Goins, 25.04.0. Link 1 and link 2)

We fixed displaying thumbnails and avatars when using the Peertube backend. (Joshua Goins, 24.12.3. Link, link 2, and link 3)

Barcode Scanner Scan and create QR-Codes

Qrca can now scan a QR code directly from an image instead of just from the camera. (Onuralp Sezer, 25.04.0. Link)

Tokodon Browse the Fediverse

We are now using more fitting icons for the "Embed" and "Open in Browser" actions in Tokodon's context menu. We also removed the duplicated "Copy to Clipboard" action from that context menu. (Joshua Goins, 24.12.3. Link and link 2)

Following the improvements from two weeks ago, we did even even more accessibility/screen reader improvements to Tokodon. (Joshua Goins, 24.12.3. Link)

…And Everything Else

This blog only covers the tip of the iceberg! If you’re hungry for more, check out Nate's blog about Plasma and be sure not to miss his This Week in Plasma series, where every Saturday he covers all the work being put into KDE's Plasma desktop environment.

For a complete overview of what's going on, visit KDE's Planet, where you can find all KDE news unfiltered directly from our contributors.

Get Involved

The KDE organization has become important in the world, and your time and contributions have helped us get there. As we grow, we're going to need your support for KDE to become sustainable.

You can help KDE by becoming an active community member and getting involved. Each contributor makes a huge difference in KDE — you are not a number or a cog in a machine! You don’t have to be a programmer either. There are many things you can do: you can help hunt and confirm bugs, even maybe solve them; contribute designs for wallpapers, web pages, icons and app interfaces; translate messages and menu items into your own language; promote KDE in your local community; and a ton more things.

You can also help us by donating. Any monetarnky contribution, however small, will help us cover operational costs, salaries, travel expenses for contributors and in general just keep KDE bringing Free Software to the world.

To get your application mentioned here, please ping us in invent or in Matrix.