Modern distributed systems need to process massive amounts of data efficiently while maintaining strict ordering guarantees. This is especially challenging when scaling horizontally across multiple nodes. How do we ensure messages from specific sources are processed in order while still taking advantage of parallelism and fault tolerance?

Elixir, with its robust concurrency model and distributed computing capabilities, is well-suited for solving this problem. In this article, we’ll build a scalable, distributed message pipeline that:

• Bridges RabbitMQ and Google Cloud PubSub, delivering messages from RabbitMQ queues to a PubSub topic.
• Ensures message ordering for each RabbitMQ queue.
• Scales horizontally across multiple nodes.
• Distribute the message pipelines evenly across the Elixir cluster.
• Gracefully handles failures and network partitions.

Many modern applications require processing large volumes of data while preserving message order from individual sources. Consider, for example, IoT systems where sensor readings must be processed in sequence, or multi-tenant applications where each tenant’s data requires sequential processing.

The solution we’ll build addresses these requirements by treating each RabbitMQ queue as an ordered data source.

Let’s explore how to design this system using Elixir’s distributed computing libraries: Broadway, Horde, and libcluster.

Architecture overview

The system consists of multiple Elixir nodes forming a distributed cluster. Each node runs one or more Broadway pipelines to process messages from RabbitMQ queues and forward them to Google Cloud PubSub. To maintain message ordering, each queue has exactly one pipeline instance running across the cluster at any time. If a node fails the system must redistribute its pipelines to other nodes automatically, and if a new node joins the cluster then the existing pipelines should be redistributed to ensure a balanced load.

Elixir natively supports the ability to cluster multiple nodes together so that processes and distributed components within the cluster can communicate seamlessly. We will employ the libcluster library since it provides several strategies to automatize cluster formation and healing.

For the data pipelines, the Broadway library provides a great framework to support multi-stage data processing while handling back-pressure, batching, fault tolerance and other good features.

To correctly maintain the distribution of data pipelines across the Elixir nodes, the Horde library comes to the rescue by providing the building blocks we need: a distributed supervisor that we can use to distribute and maintain healthy pipelines on the nodes, and a distributed registry that we use directly to track which pipelines exist and on which nodes they are.

Finally, a PipelineManager component will take care of monitoring RabbitMQ for new queues and starting/stopping corresponding pipelines dynamically across the cluster.

Technical implementation

Let’s initiate a new Elixir app with a supervision tree.

mix new message_pipeline --sup

First, we’ll need to add our library dependencies in mix.exs and run mix deps.get:

defmodule MessagePipeline.MixProject do
  use Mix.Project

  def project do
    [
      app: :message_pipeline,
      version: "0.1.0",
      elixir: "~> 1.17",
      start_permanent: Mix.env() == :prod,
      deps: deps()
    ]
  end

  def application do
    [
      extra_applications: [:logger],
      mod: {MessagePipeline.Application, []}
    ]
  end

  defp deps do
    [
      {:libcluster, "~> 3.3"},
      {:broadway, "~> 1.0"},
      {:broadway_rabbitmq, "~> 0.7"},
      {:google_api_pub_sub, "~> 0.34"},
      {:goth, "~> 1.4"},
      {:tesla, "~> 1.12"},
      {:jason, "~> 1.4"},
      {:amqp, "~> 4.0"}
    ]
  end
end

Rolling our own PubSub client

Since we just need to publish messages to Google Cloud PubSub and the API is straightforward, we don’t need fancy libraries here.

Let’s start by adding Goth to the supervision tree to manage Google’s service account credentials.

defmodule MessagePipeline.Application do
  use Application

  def start(_type, _args) do
    credentials =
      "GOOGLE_APPLICATION_CREDENTIALS_PATH"
      |> System.fetch_env!()
      |> File.read!()
      |> Jason.decode!()

    children = [
      {Goth, name: MessagePipeline.Goth, source: {:service_account, credentials}},
      # Other children...
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

And here’s our HTTP client to publish messages to Google Cloud PubSub

defmodule MessagePipeline.GooglePubsub do
  @google_pubsub_base_url "https://pubsub.googleapis.com"

  def publish_messages(messages) when is_list(messages) do
    project_id = System.fetch_env!("GOOGLE_PUBSUB_PROJECT_ID")
    topic_name = System.fetch_env!("GOOGLE_PUBSUB_TOPIC")

    topic = "projects/#{project_id}/topics/#{topic_name}"

    request = %{
      messages: Enum.map(messages, &%{data: Base.encode64(&1)})
    }

    with {:ok, auth_token} <- generate_auth_token(),
         client = client(auth_token),
         {:ok, response} <- Tesla.post(client, "/v1/#{topic}:publish", request) do
      %{body: %{"messageIds" => message_ids}} = response
      {:ok, message_ids}
    end
  end

  defp client(auth_token) do
    middleware = [
      {Tesla.Middleware.BaseUrl, @google_pubsub_base_url},
      Tesla.Middleware.JSON,
      {Tesla.Middleware.Headers, [{"Authorization", "Bearer " <> auth_token}]}
    ]

    Tesla.client(middleware)
  end

  defp generate_auth_token do
    with {:ok, %{token: token}} <- Goth.fetch(MessagePipeline.Goth) do
      {:ok, token}
    end
  end
end

Clustering with libcluster

We’ll use libcluster to establish communication between our Elixir nodes. Here’s an example configuration that uses the Gossip strategy to form a cluster between nodes:

defmodule MessagePipeline.Application do
  use Application

  def start(_type, _args) do
    topologies = [
      gossip_example: [
        strategy: Elixir.Cluster.Strategy.Gossip
      ]
    ]

    children = [
      {Cluster.Supervisor, [topologies, [name: MessagePipeline.ClusterSupervisor]]},
      # Other children...
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

Distributed process management with Horde

We’ll use Horde to manage our Broadway pipelines across the cluster. Horde ensures that each pipeline runs on exactly one node and handles redistribution when nodes fail.

Let’s add Horde’s supervisor and registry to the application’s supervision tree.

The UniformQuorumDistribution distribution strategy distributes processes using a hash mechanism among all reachable nodes. In the event of a network partition, it enforces a quorum and will shut down all processes on a node if it is split from the rest of the cluster: the unreachable node is drained and the pipelines can be resumed on the other cluster nodes.

defmodule MessagePipeline.Application do
  use Application
  
  def start(_type, _args) do
    children = [
      {Horde.Registry, [
          name: MessagePipeline.PipelineRegistry,
          members: :auto,
          keys: :unique
        ]},
      {Horde.DynamicSupervisor, [
          name: MessagePipeline.PipelineSupervisor,
          members: :auto,
          strategy: :one_for_one,
          distribution_strategy: Horde.UniformQuorumDistribution
        ]}
      # Other children...
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

Broadway pipeline implementation

Each pipeline uses Broadway to consume messages from RabbitMQ and publish them to Google PubSub.

A strict, per-queue ordering is guaranteed by setting a concurrency of 1.

defmodule MessagePipeline.Pipeline do
  use Broadway

  alias Broadway.Message

  def child_spec(opts) do
    queue_name = Keyword.fetch!(opts, :queue_name)
    pipeline_name = pipeline_name(queue_name)

    %{
      id: pipeline_name,
      start: {__MODULE__, :start_link, opts}
    }
  end

  def start_link(opts) do
    queue_name = Keyword.fetch!(opts, :queue_name)
    pipeline_name = pipeline_name(queue_name)

    pipeline_opts = [
      name: {:via, Horde.Registry, {MessagePipeline.PipelineRegistry, pipeline_name}},
      producer: [
        module: {
          BroadwayRabbitMQ.Producer,
          queue: queue_name,
          connection: [
            host: System.fetch_env!("RABBITMQ_HOST"),
            port: String.to_integer(System.fetch_env!("RABBITMQ_PORT")),
            username: System.fetch_env!("RABBITMQ_USER"),
            password: System.fetch_env!("RABBITMQ_PASSWORD")
          ]
        },
        concurrency: 1
      ],
      processors: [
        default: [
          concurrency: 1
        ]
      ],
      batchers: [
        default: [
          batch_size: 100,
          batch_timeout: 200,
          concurrency: 1
        ]
      ]
    ]
    
    case Broadway.start_link(__MODULE__, pipeline_opts) do
      {:ok, pid} ->
        {:ok, pid}

      {:error, {:already_started, _pid}} ->
        :ignore
    end
  end

  def pipeline_name(queue_name) do
    String.to_atom("pipeline_#{queue_name}")
  end

  @impl true
  def handle_message(_, message, _) do
    message
    |> Message.update_data(&process_data/1)
  end

  @impl true
  def handle_batch(_, messages, _, _) do
    case publish_to_pubsub(messages) do
      {:ok, _message_ids} -> messages
      {:error, reason} -> 
        # Mark messages as failed
        Enum.map(messages, &Message.failed(&1, reason))
    end
  end

  defp process_data(data) do
    # Transform message data as needed
    data
  end

  defp publish_to_pubsub(messages) do
    MessagePipeline.GooglePubsub.publish_messages(messages)
  end
end

Queue discovery and pipeline management

Finally, we need a process to monitor RabbitMQ queues and ensure pipelines are running for each one.

The Pipeline Manager periodically queries RabbitMQ for existing queues. If a new queue appears, it starts a Broadway pipeline only if one does not already exist in the cluster. If a queue is removed, the corresponding pipeline is shut down.

defmodule MessagePipeline.PipelineManager do
  use GenServer

  @timeout :timer.minutes(1)

  def start_link(opts) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  def init(_opts) do
    state = %{managed_queues: MapSet.new()}

    {:ok, state, {:continue, :start}}
  end

  def handle_continue(:start, state) do
    state = manage_queues(state)

    {:noreply, state, @timeout}
  end

  def handle_info(:timeout, state) do
    state = manage_queues(state)

    {:noreply, state, @timeout}
  end

  def manage_queues(state) do
    {:ok, new_queues} = discover_queues()
    current_queues = state.managed_queues

    queues_to_add = MapSet.difference(new_queues, current_queues)
    queues_to_remove = MapSet.difference(current_queues, new_queues)

    Enum.each(queues_to_add, &start_pipeline/1)
    Enum.each(queues_to_remove, &stop_pipeline/1)

    %{state | managed_queues: new_queues}
  end

  defp discover_queues do
    {:ok, conn} =
      AMQP.Connection.open(
        host: System.fetch_env!("RABBITMQ_HOST"),
        port: String.to_integer(System.fetch_env!("RABBITMQ_PORT")),
        username: System.fetch_env!("RABBITMQ_USER"),
        password: System.fetch_env!("RABBITMQ_PASSWORD")
      )
    {:ok, chan} = AMQP.Channel.open(conn)
    {:ok, queues} = AMQP.Queue.list(chan)

    # Filter out system queues
    queues
    |> Enum.reject(fn %{name: name} -> 
        String.starts_with?(name, "amq.") or
        String.starts_with?(name, "rabbit")
      end)
    |> Enum.map(& &1.name)
    |> MapSet.new()
  end

  defp start_pipeline(queue_name) do
    pipeline_name = MessagePipeline.Pipeline.pipeline_name(queue_name)

    case Horde.Registry.lookup(MessagePipeline.PipelineRegistry, pipeline_name) do
      [{pid, _}] ->
        {:error, :already_started}
      [] ->
        Horde.DynamicSupervisor.start_child(
          MessagePipeline.PipelineSupervisor,
          {MessagePipeline.Pipeline, queue_name: queue_name}
        )
    end
  end

  defp stop_pipeline(queue_name) do
    pipeline_name = MessagePipeline.Pipeline.pipeline_name(queue_name)

    case Horde.Registry.lookup(MessagePipeline.PipelineRegistry, pipeline_name) do
      [{pid, _}] ->
        Horde.DynamicSupervisor.terminate_child(MessagePipeline.PipelineSupervisor, pid)
      [] ->
        {:error, :not_found}
    end
  end
end

Let’s not forget to also add the pipeline manager to the application’s supervision tree.

defmodule MessagePipeline.Application do
  use Application

  def start(_type, _args) do
    children = [
      {MessagePipeline.PipelineManager, []}
      # Other children...
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

Test the system

We should now have a working and reliable system. To quickly test it out, we can configure a local RabbitMQ broker, a Google Cloud PubSub topic, and finally a couple of Elixir nodes to verify that distributed pipelines are effectively run to forward messages between RabbitMQ queues and PubSub.

Let’s start by running RabbitMQ with the management plugin. RabbitMQ will listen for connections on the 5672 port, while also exposing the management interface at http://localhost:15672. The default credentials are guest/guest.

docker run -d --name rabbitmq \
  -p 5672:5672 \
  -p 15672:15672 \
  rabbitmq:3-management

Next, install and use the gcloud CLI to create a Google Cloud project, a PubSub topic, and a a service account to access PubSub programmatically.

# Login to Google Cloud
gcloud auth login

# Create a new project (or use an existing one)
gcloud projects create message-pipeline-test
gcloud config set project message-pipeline-test

# Enable PubSub API
gcloud services enable pubsub.googleapis.com

# Create a topic
gcloud pubsub topics create test-topic

# Create service account for local testing
gcloud iam service-accounts create local-test-sa

# Generate and download credentials
gcloud iam service-accounts keys create ./google-credentials.json \
  --iam-account local-test-sa@message-pipeline-test.iam.gserviceaccount.com

# Grant publish permissions
gcloud pubsub topics add-iam-policy-binding test-topic \
  --member="serviceAccount:local-test-sa@message-pipeline-test.iam.gserviceaccount.com" \
  --role="roles/pubsub.publisher"

It’s now time to start two terminal sessions where we can export the needed environment variables before running two Elixir nodes.

# In terminal 1

export RABBITMQ_HOST=localhost
export RABBITMQ_PORT=5672
export RABBITMQ_USER=guest
export RABBITMQ_PASSWORD=guest
export GOOGLE_PUBSUB_PROJECT_ID=message-pipeline-test
export GOOGLE_PUBSUB_TOPIC=test-topic
export GOOGLE_APPLICATION_CREDENTIALS_PATH=$(pwd)/google-credentials.json
export RELEASE_COOKIE=my-secret-cookie

iex --sname node1 -S mix

# In terminal 2 (same variables)

export RABBITMQ_HOST=localhost
export RABBITMQ_PORT=5672
export RABBITMQ_USER=guest
export RABBITMQ_PASSWORD=guest
export GOOGLE_PUBSUB_PROJECT_ID=message-pipeline-test
export GOOGLE_PUBSUB_TOPIC=test-topic
export GOOGLE_APPLICATION_CREDENTIALS_PATH=$(pwd)/google-credentials.json
export RELEASE_COOKIE=my-secret-cookie

iex --sname node2 -S mix

To verify cluster formation, from one of the nodes:

# List all nodes in the cluster, should show the other node
Node.list()

# Check Horde supervisor distribution
:sys.get_state(MessagePipeline.PipelineSupervisor)

Now let’s create some test queues on RabbitMQ and start publishing some messages.

# Download rabbitmqadmin if not already available
wget http://localhost:15672/cli/rabbitmqadmin
chmod +x rabbitmqadmin

# Create queues
./rabbitmqadmin declare queue name=test-queue-1
./rabbitmqadmin declare queue name=test-queue-2

# Publish test messages
./rabbitmqadmin publish routing_key=test-queue-1 payload="Message 1 for queue 1"
./rabbitmqadmin publish routing_key=test-queue-1 payload="Message 2 for queue 1"
./rabbitmqadmin publish routing_key=test-queue-2 payload="Message 1 for queue 2"

# List queues and their message counts
./rabbitmqadmin list queues name messages_ready messages_unacknowledged

# Get messages (without consuming them)
./rabbitmqadmin get queue=test-queue-1 count=5 ackmode=reject_requeue_true

One can also use the RabbitMQ management interface at http://localhost:15672, authenticate with the guest/guest default credentials, go to the “Queues” tab, click “Add a new queue”, and create “test-queue-1” and “test-queue-2”.

After a minute, the Elixir nodes should automatically start some pipelines corresponding to the RabbitMQ queues.

# List all registered pipelines
Horde.Registry.select(MessagePipeline.PipelineRegistry, [{{:"$1", :"$2", :"$3"}, [], [:"$2"]}])

# Check specific pipeline
pipeline_name = :"pipeline_test-queue-1"
Horde.Registry.lookup(MessagePipeline.PipelineRegistry, pipeline_name)

Now, if we publish messages on the RabbitMQ queues, we should see them appear on the PubSub topic.

We can verify it from Google Cloud Console, or by creating a subscription, publishing some messages on RabbitMQ, and then pulling messages from the PubSub subscription.

gcloud pubsub subscriptions create test-sub --topic test-topic
# ...Publish messages on RabbitMQ queues...
gcloud pubsub subscriptions pull test-sub --auto-ack

If we stop one of the Elixir nodes (Ctrl+C twice in its IEx session) to simulate a failure, the pipelines should be redistributed in the remaining node:

# Check updated node list
Node.list()

# Check pipeline distribution
Horde.Registry.select(MessagePipeline.PipelineRegistry, [{{:"$1", :"$2", :"$3"}, [], [:"$2"]}])

Rebalancing pipelines on new nodes

With our current implementation, pipelines are automatically redistributed when a node fail but they are not redistributed when a new node joins the cluster.

Fortunately, Horde supports precisely this functionality from v0.8+, and we don’t have to manually stop and re-start our pipelines to have them landing on other nodes.

All we need to do is enable the option process_distribution: :active on Horde’s supervisor to automatically rebalance processes on node joining / leaving. The option runs each child spec through the choose_node/2 function of the preferred distribution strategy, detects which processes should be running on other nodes considering the new cluster configuration, and specifically restarts those particular processes such that they run on the correct node.

defmodule MessagePipeline.Application do
  use Application
  
  def start(_type, _args) do
    children = [
      {Horde.Registry, [
          name: MessagePipeline.PipelineRegistry,
          members: :auto,
          keys: :unique
        ]},
      {Horde.DynamicSupervisor, [
          name: MessagePipeline.PipelineSupervisor,
          members: :auto,
          strategy: :one_for_one,
          distribution_strategy: Horde.UniformQuorumDistribution,
          process_redistribution: :active
        ]}
      # Other children...
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

Conclusion

This architecture provides a robust solution for processing ordered message streams at scale. The combination of Elixir’s distributed capabilities, Broadway’s message processing features, and careful coordination across nodes enables us to build a system that can handle high throughput while maintaining message ordering guarantees.

To extend this solution for your specific needs, consider these enhancements:

• Adopt a libcluster strategy suitable for a production environment, such as Kubernetes.
• Tune queue discovery latency, configuring the polling interval based on how frequently new queues are created. Better yet, instead of polling RabbitMQ, consider setting up RabbitMQ event notifications to detect queue changes in real-time.
• Declare AMQP queues as durable and make sure that publishers mark published messages as persisted, in order to survive broker restarts and improve delivery guarantees. Use publisher confirms to ensure messages are safely received by the broker. Deploy RabbitMQ in a cluster with queue mirroring or quorum queues for additional reliability.
• Add monitoring, instrumenting Broadway and Horde with Telemetry metrics.
• Enhance error handling and retry mechanisms. For example, retry message publication to PubSub N times before failing the messages, thus invalidating the (possibly costly) processing operation.
• Unit & e2e testing. Consider that the gcloud CLI (gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators) contains a PubSub emulator that may come in handy: e.g. gcloud beta emulators pubsub start — project=test-project — host-port=0.0.0.0:8085
• Leverage an HorizontalPodAutoscaler for automated scaling on Kubernetes environments based on resource demand.
• Evaluate the use of Workload Identities if possible. For instance, you can provide your workloads with access to Google Cloud resources by using federated identities instead of a service account key. This approach frees you from the security concerns of manually managing service account credentials.

The second maintenance release of the 24.12 cycle is out with multiple bug fixes. Notable changes include fixes for crashes, UI resizing issues, effect stack behavior, proxy clip handling, and rendering progress display, along with improvements to Speech-to-text in Flatpak and macOS packages.

For the full changelog continue reading on kdenlive.org.

Kaidan supports Qt 6 now! In addition, this release improves the user interface and fixes some bugs. Have a look at the changelog for more details.

Changelog

Features:

• Highlight public XMPP provider titles while card is expanded (melvo)
• Round corners of cards and buttons (melvo)
• Add fading in/out hover effect to map previews (melvo)
• Collapse contact profiles by default if they have more than 3 entries (melvo)
• Show colored check mark for delivered messages instead of none to avoid message bubble resizing (melvo)

Bugfixes:

• Fix opening public MUC-based group chats via another XMPP client (melvo)
• Fix playing voice messages and changing playing position (melvo)
• Fix updating message reactions that could not be sent instead of adding them a second time (melvo)
• Fix updating group chat users in user interface (melvo)
• Fix displaying message reaction details (melvo)
• Update filtering contacts by labels even if label list is not open anymore (melvo)
• Fix scrolling media overview (melvo)
• Fix updating draft messages (melvo)

Notes:

• Kaidan requires Qt 6.6 now (mlaurent, melvo, fazevedo, plata)

Download

• Source code (.tar.xz) (sig signed with 04EFAD0F7A4D9724)
• Linux (Flatpak on Flathub)

Or install Kaidan for your distribution:

In this article, we’re going to have a look at Qt for Python, how it integrates with Qt Creator, and why you might want to consider using it, too.

Continue reading PySide6, Qt for Python at basysKom GmbH.

Tellico 4.1.1 is available, with a few fixes.

Improvements

• Updated Filmaffinity and ComicVine data sources.
• Added configurable image size to Discogs and Moviemeter data sources.
• Fixed ISBN searching with Colnect.

This is the release schedule the release team agreed on

https://community.kde.org/Schedules/KDE_Gear_25.04_Schedule

Dependency freeze is in around 3 weeks (March 6) and feature freeze one
after that. Get your stuff ready!
 

This week, I focused on integrating the Monte Carlo Tree Search (MCTS) algorithm into the MankalaEngine. The primary goal was to test the performance of the MCTS-based agent against various existing algorithms in the engine. Let's dive into what MCTS is, how it works, and what I discovered during the testing phase.

What is Monte Carlo Tree Search (MCTS)?

The Monte Carlo Tree Search (MCTS) is a heuristic search algorithm used for decision-making in sequential decision problems. It incrementally builds a search tree and simulates multiple random moves at each step to evaluate potential outcomes. These simulations help the algorithm determine the most promising move to make.

How Does MCTS Work?

MCTS operates through four key steps:

1. Selection

The algorithm starts at the root node (representing the current game state) and traverses down the tree to a leaf node (an unexplored state). During this process, the algorithm selects child nodes using a specific strategy.

A popular strategy for node selection is Upper Confidence Bounds for Trees (UCT). The UCT formula helps balance exploration and exploitation by selecting nodes based on the following equation:

UCT = mean + C × sqrt(ln(N) / n)

Where:

• mean is the average reward (or outcome) of a node.
• N is the total number of simulations performed for the parent node.
• n is the number of simulations performed for the current child node.
• C is a constant that controls the level of exploration.

2. Expansion

Once the algorithm reaches a leaf node, it expands the tree by adding one or more child nodes representing potential moves or decisions that can be made from the current state.

3. Simulation

The algorithm then performs a simulation or rollout from the newly added child node. During this phase, the algorithm randomly plays out a series of moves (typically following a simple strategy) until the game reaches a terminal state (i.e., win, loss, or draw).

This is where the Monte Carlo aspect of MCTS shines. By simulating many random games, the algorithm gains insights into the likely outcomes of different actions.

4. Backpropagation

After the simulation ends, the results are propagated back up the tree, updating the nodes with the outcome of the simulation. This allows the algorithm to adjust the expected rewards of the parent nodes based on the result of the child node’s simulation.

read more here - MCTS

Implementing MCTS in C++ for MankalaEngine

With a solid understanding of the algorithm, I began implementing MCTS in C++. The initial step involved integrating the MCTS logic into the benchmark utility of the MankalaEngine. After resolving a series of issues and running multiple tests, the code was functioning as expected.

Testing Results

I compared the performance of the MCTS agent against other existing agents in the MankalaEngine, such as Minimax, MTDF, and Random agents. Here’s how the MCTS agent performed:

Random Agent (Player 1) vs. MCTS (Player 2)

• MCTS won 80% of the time

MCTS (Player 1) vs. Random Agent (Player 2)

• MCTS won 60% of the time

MCTS vs. Minimax & MTDF

• Unfortunately, MCTS consistently lost against both Minimax and MTDF agents. 😞

Key Improvements for MCTS

While MCTS performed well against the Random Agent, there is still room for improvement, especially in its simulation phase. Currently, the algorithm uses a random policy for simulations, which can be inefficient. To improve performance, we can:

• Use more efficient simulation policies that simulate only promising moves, rather than randomly selecting moves.
• At the start of the Selection step, focus on moves that have historically been good opening strategies (this requires further research to identify these moves, especially in Pallanguli).
• Fine-tune the exploration-exploitation balance to improve decision-making.

Upcoming Tasks

In the upcoming week, I plan to:

• Write test cases for the Pallanguli implementation.
• Review the implementation of Pallanguli.

by Alexander Bokovoy and Andreas Schneider

FOSDEM 2025 is just behind us and it was a great event as always. Alexander and I had a chance to talk
about the local authentication hub project. Our FOSDEM talk was “localkdc – a general local authentication hub”. You can watch it and come back here for more details.

But before going into details, let us provide a bit of a background. It is 2025 now and we should go almost three decades back (ugh!).

History dive

Authentication on Linux systems is interwoven with the identity of the users. Once a user logged in, a process is running under a certain POSIX account identity. Many applications validate the presence of the account prior to the authentication itself. For example, the OpenSSH server does check the POSIX account and its properties and if the user was not found, will intentionally corrupt the password passed to the PAM authentication stack request. An authentication request will fail but the attempt will be recorded in the system journal.

This joint operation between authentication and identification sources in Linux makes it important to maintain a coherent information state. No wonder that in corporate environments it is often handled centrally: user and group identities stored at a central server and sourced from that one by a local software, such as SSSD. In order to consume these POSIX users and groups, SSSD needs to be registered with the centralized authority or, in other words, enrolled into the domain. Domain enrollment allows not only identity and authentication of users: both the central server and the enrolled client machine can mutually authenticate each other and be sure they talk to the right authority when authenticating the user.

FreeIPA provides a stable mechanism for building a centralized domain management system. Each user account has POSIX attributes associated with it and each user account is represented by the Kerberos principal. Kerberos authentication can be used to transfer the authentication state across multiple services and provides a chance for services to discover user identity information beyond POSIX. It also makes strong linking between the POSIX level identity and authentication structure possible: for example, a Kerberos service may introspect a Kerberos ticket presented by a user’s client application to see how this user was authenticated originally: with a password or some specific passwordless mechanism. Or, perhaps, that a client application performs operations on behalf of the user after claiming it was authenticated using a different (non-Kerberos) authentication.

Local user accounts’ use lacks this experience. Each individual service needs to reauthenticate a user again and again. Local system login: authenticate. Elevating privileges through SUDO? Authenticate again, if not explicitly configured otherwise. Details of the user session state, like how long this particular session is active, is not checked by the applications, making it also harder to limit access. There is no information on how this user was authenticated. Finally, overall user experience between local (standalone) authentication and domain-enrolled one differs, making it harder to adjust and educate users.

Local authentication is also typically password-based. This is not a bad thing in itself but depending on applications and protocols, worse choices could be made, security-wise. For example, contemporary SMB 3.11 protocol is quite secure if authenticated using Kerberos. For non-Kerberos usage, however, it is left to rely on NTLM authentication protocol which requires use of RC4 stream cipher. There are multiple attacks known to break RC4-based encryption, yet it is still used in majority of non-domain joined communications using SMB protocol simply because there was no (so far) alternative. To be correct, there was always an alternative, use of Kerberos protocol, but setting it up for individual isolated systems wasn’t practical.

The Kerberos protocol assumes the use of three different parties: a client, a service, and a key distribution center (KDC). In corporate environments a KDC is part of the domain controller system, a client and a service are both domain members, computers are enrolled in the domain. The client authenticates to KDC and obtains a Kerberos ticket granting ticket (TGT). It then requests a service ticket from the KDC by presenting its TGT and then presents this service ticket to the service. The service application, on its side, is able to decrypt the service ticket presented by the client and authenticate the request.

In the late 2000s Apple realised that for individual computers a number of user accounts is typically small and a KDC can be run as a service on the individual computer itself. When both the client and server are on the same computer, this works beautifully. The only problem is that when a user needs to authenticate to a different computer’s service, the client cannot reach the KDC hosted on the other computer because it is not exposed to the network directly. Luckily, MIT Kerberos folks already thought about this problem a decade prior to that: in 1997 a first idea was published for a Kerberos extension that allowed to tunnel Kerberos requests over a different application protocol. This specification became later known as “Initial and Pass Through Authentication Using Kerberos V5 and the GSS-API” (IAKerb). An initial implementation for MIT Kerberos was done in 2009/2010 while Apple introduced it in 2007 to enable remote access to your own Mac across the internet. It came in MacOS X 10.5 as a “Back to My Mac” feature and even got specified in RFC 6281, only to be retired from MacOS in 2019.

Modern days

In the 2020s Microsoft continued to work on NTLM removal. In 2023 they announced that all Windows systems will have a local KDC as their local authentication source, accessible externally via selected applications through the IAKerb mechanism. By the end of 2024, we have only seen demos published by Microsoft engineers at various events but this is a promising path forward. Presence of the local KDC in Windows raises an interoperability requirement: Linux systems will have to handle access to Windows machines in a standalone environment over SMB protocol. Authentication is currently done with NTLM, it will eventually be removed, thus we need to support the IAKerb protocol extension.

The NTLM removal for Linux systems requires several changes. First, the Samba server will need to learn how to accept authentication with the IAKerb protocol extension. Then, Samba client code needs to be able to establish a client connection and advertise IAKerb protocol extension. For kernel level access, the SMB filesystem driver needs to learn how to use IAKerb as well, this will also need to be implemented in the user space cifs-utils package. Finally, to be able to use the same feature in a pure Linux environment, we need to be able to deploy Kerberos KDC locally and do it in an easy manner on each machine.

This is where we had an idea. If we are going to have a local KDC running on each system, maybe we should use it to handle all authentication and not just for the NTLM removal? This way we can make both the local and domain-enrolled user experience the same and provide access locally to a whole set of authentication methods we support for FreeIPA: passwords, smartcards, one-time passwords and remote RADIUS server authentication, use of FIDO2 tokens, and authentication against an external OAuth2 Identity Provider using a device authorization grant flow.

How “local” a local KDC should be?

On standalone systems it is often not desirable to run daemons continuously. Also, it is not desirable to expose these services to the connected network if they really don’t need to be exposed. A common approach to solve this problem is by providing a local inter-process communication (IPC) mechanism to communicate with the server components. We chose to expose a local KDC via UNIX domain sockets. A UNIX domain socket is a well-known mechanism and has known security properties. With the help of a systemd feature called socket activation, we also can start local KDC on demand, when a Kerberos client connects over the UNIX domain socket. Since on local systems actual authentication requests don’t happen often, this helps to reduce memory and CPU usage in the long run.

If a local KDC is only accessible over a UNIX domain socket, remote applications could not get access to it directly. This means they would need to have help from a server application that can utilize the IAKerb mechanism to pass-through the communication between a client and the KDC. It would enable us to authenticate as a local user remotely from a different machine. Due to how the IAKerb mechanism is designed and integrated into GSS-API, this only allows password-based authentication. Anything that requires passwordless methods cannot obtain initial Kerberos authentication over IAKerb, at least at this point.

Here is a small demo on Fedora, using our localkdc tool to start a local KDC, obtain a Kerberos ticket upon login. The tickets can then be used effortlessly to authenticate to local services such as SUDO or Samba. For remote access we rely on Samba support for IAKerb and authenticate with GSSAPI but local smbclient uses a password first to obtain the initial ticket over IAKerb. This is purely a limitation of the current patches we have to Samba.

Make a pause here and think about the implications. We have an initial Kerberos ticket from the local system. The Kerberos ticket embeds details of how this authentication happened. We might have used a password to authenticate, or a smartcard. Or any other supported pre-authentication methods. We could reuse the same methods FreeIPA already provides in the centralized environment.

The Kerberos ticket also can contain details about the user session, including current group membership. It does not current have that in the local KDC case but we aim to fix that. This ticket can be used to authenticate to any GSS-API or Kerberos-aware service on this machine. If a remote machine accepts Kerberos, it theoretically could accept a ticket presented by a client application running on the local machine as well. Only, to do that it needs to be able to communicate with our local KDC and it couldn’t access it.

Trust management

Luckily, a local KDC deployment is a full-featured Kerberos realm and thus can establish cross-realm agreements with other Kerberos realms. If two “local” KDC realms have trust agreements between each other, they can issue cross-realm Kerberos tickets which applications can present over IAKerb to the remote “local” KDC. Then a Kerberos ticket to a service running on the target system can be requested and issued by the system’s local KDC.

Thus, we can achieve passwordless authentication locally on Linux systems and have the ability to establish peer to peer agreements across multiple systems, to allow authentication requests to flow and operate on commonly agreed credentials. A problem now moves to the management area: how to manage these peer to peer agreements and permissions in an easy way?

Systemd User/Group API support

MIT Kerberos KDC implementation provides a flexible way to handle Kerberos principals’ information. A database backend (KDB) implementation can be dynamically loaded and replaced. This is already used by both FreeIPA and Samba AD to integrate MIT Kerberos KDC with their own database backends based on different LDAP server implementations. For a local KDC use case running a full-featured LDAP server is not required nor intended. However, it would be great if different applications could expose parts of the data needed by the KDB interfaces and cooperate together. Then a single KDB driver implementation could be used to streamline and provide uniform implementation of Kerberos-specific details in a local KDC.

One of the promising interfaces to achieve that is the User/Group record lookup API via varlink from systemd. Varlink allows applications to register themselves and listen on UNIX domain sockets for communication similar to D-Bus but with much less implementation overhead. The User/Group API technically also allows to merge data coming from different sources when an application inquires the information. “Technically”, because io.systemd.Multiplexer API endpoint currently does not support merging non-overlapping data representing the same account from multiple sources. Once it would become possible, we could combine the data dynamically and may interact with users on demand when corresponding requsts come in. Or we can implement our own blending service.

Blending data requests from multiple sources within MIT KDC needs a specialized KDB driver. We certainly don’t want this driver to duplicate the code from other drivers, so making these drivers stackable would be a good option. Support for one level of stacking has been merged to MIT Kerberos through a quickly processed pull request and will be available in the next MIT Kerberos release. This allows us to have a single KDB driver that loads other drivers specialized in storing Kerberos principals and processing additional information like MS-PAC structure or applying additional authorization details.

Establishing trusts

If Alice and Bob are in the same network and want to exchange some files, they could do this using SMB and Samba. But that Alice can authenticate on Bob’s machine, they would need to establish a Kerberos cross realm trust. With the current tooling this is a complex task. For users we need to make this more accessible. We want to allow users to request trust on demand and validate these requests interactively. We also want to allow trust to be present for a limited timeframe, automatically expiring or manually removed.

If we have a Kerberos principal lookup on demand through a curated varlink API endpoint, we also can have a user-facing service to initiate establishing the trust between two machines on demand. Imagine a user trying to access SMB share on one desktop system that triggers a pop-up to establish trust relationship with a corresponding local KDC on the remote desktop system. Both owners of the systems would be able to communicate out of band that provided information is correct and can be trusted. Once it is done, we can return back the details of the specific Kerberos principal that represents this trust relationship. We can limit lifetime of this agreement so that it would disappear automatically in one hour or a day, or a week.

Current state of local authentication hub

We started with two individual implementation paths early in 2024:

• Support IAKerb in MIT Kerberos and Samba
• Enable MIT Kerberos to be used locally without network exposure

MIT Kerberos did have support for IAKerb protocol extension for more than a decade but since Microsoft introduced some changes to the protocol, those changes needed to be integrated as well. This was completed during summer 2024, though no upstream release is available yet. MIT Kerberos typically releases new versions yearly in January so we hope to get some updates early 2025.

Samba integration with IAKerb is currently under implementation. Originally, Microsoft was planning to release Windows 11 and Windows Server 2025 with IAKerb support enabled during autumn 2024. However, the Windows engineering team faced some issues and IAKerb is still not enabled in the Windows Server 2025 and Windows 11 releases. We are looking forward to getting access to Windows builds that enable IAKerb support to ensure interoperability before merging Samba changes upstream. We also need to complete the Samba implementation to properly support locally-issued Kerberos tickets and not only do acquisition of the ticket based on the password.

Meanwhile, our cooperation with MIT Kerberos development team led to advancements in the local KDC support. The MIT Kerberos KDC can now be run over a UNIX domain socket. Also on systemd-enabled systems we allow socket activation, transforming local KDC into an on-demand service. We will continue our work on a dynamic database for a local KDC, to allow on-demand combination of resources from multiple authoritative local sources (Samba, FreeIPA, SSSD, local KDC, future dynamic trust application).

For experiments and ease of deployments, a new configuration tool was developed, localkdc. The tool is available at localkdc and COPR repository can be used to try the whole solution on Fedora.

If you want to get that test tried in a simple setup, you might be interested in a tool that we developed initially for FreeIPA: FreeIPA local tests. This tool allows to provision and run a complex test environment in podman containers. The video of the local KDC usage was actually generated automatically by the scripts from here.

Model/View Drag and Drop in Qt - Part 1

Moving a row in a tableview, step 1

Moving a row in a tableview, step 2

Moving a row in a tableview, step 3

Reorderable ListView

Reorderable TableView

class CountryModel : public QAbstractTableModel
{
    ~~~
    Qt::ItemFlags flags(const QModelIndex &index) const override
    {
        if (!index.isValid())
            return Qt::ItemIsDropEnabled; // allow dropping between items
        return Qt::ItemIsEnabled | Qt::ItemIsSelectable | Qt::ItemIsDragEnabled;
    }

    // the default is "copy only", change it
    Qt::DropActions supportedDropActions() const override { return Qt::MoveAction; }

    // the default is "return supportedDropActions()", let's be explicit
    Qt::DropActions supportedDragActions() const override { return Qt::MoveAction; }

    QStringList mimeTypes() const override { return {QString::fromLatin1(s_mimeType)}; }

    bool moveRows(const QModelIndex &sourceParent, int sourceRow, int count, const QModelIndex &destinationParent, int destinationChild) override; // see below
};

bool CountryModel::moveRows(const QModelIndex &sourceParent, int sourceRow, int count, const QModelIndex &destinationParent, int destinationChild)
{
    if (!beginMoveRows(sourceParent, sourceRow, sourceRow + count - 1, destinationParent, destinationChild))
        return false; // invalid move, e.g. no-op (move row 2 to row 2 or to row 3)

    for (int i = 0; i < count; ++i) {
        m_data.move(sourceRow + i, destinationChild + (sourceRow > destinationChild ? 0 : -1));
    }

    endMoveRows();
    return true;
}

Reorderable treeview

Reorderable treeview with a tree model

Reorderable QListWidget

Reorderable QTableWidget

Reorderable QTreeWidget

The post Model/View Drag and Drop in Qt - Part 1 appeared first on KDAB.