Skip to content

Saturday, 13 July 2024

See you next week what what will probably be an extra large installment!

Friday, 12 July 2024

Pre-orders are now open for the Akademy 2024 T-shirt, this is only for those who will be attending Akademy, in person, in Würzburg

Pre-orders will close on 31st July

We will  be opening in a few weeks separate online orders for t-shirts for those who aren't attending in person

Let’s go for my web review for the week 2024-28.


Take action to stop chat control now!

Tags: tech, politics, law, privacy

It’s time to push European governments to abandon this nonsense.

https://www.patrick-breyer.de/en/take-action-to-stop-chat-control-now/


VLMs are Blind

Tags: tech, ai, machine-learning, gpt, computer-vision

Those brand new models keep failing at surprisingly simple tasks.

https://vlmsareblind.github.io/


Universal Code Execution by Chaining Messages in Browser Extensions

Tags: tech, browser, security

This is a concerning finding. One can escape from the browser to the system with such chaining.

https://spaceraccoon.dev/universal-code-execution-browser-extensions/


Let’s talk about supply chain attacks and backdoored dependencies

Tags: tech, security, supply-chain, dependencies

Good tour of all the way dependencies might get compromised in your supply chain. Getting this easy to detect is needed.

https://kerkour.com/supply-chain-attacks-and-backdoored-dependencies


Ubuntu Security Updates Are a Confusing Mess | Grey Lumpy Dinosaur

Tags: tech, ubuntu, security

The title is a bit pushing it. Still, I didn’t realize some of the fine prints of the Ubuntu support schemes.

https://gld.mcphail.uk/posts/ubuntu-security-updates-are-a-confusing-mess/


An Empirical Study of Rust-for-Linux: The Success, Dissatisfaction, and Compromise | USENIX

Tags: tech, linux, kernel, rust

It’s nice to have a balanced view on the matter. It’s not just roses and rainbows. This gives a good overview of the current limitations and where Rust can give most benefits in the kernel.

https://www.usenix.org/conference/atc24/presentation/li-hongyu


A new way to develop on Linux

Tags: tech, linux, system, systemd

Interesting approach to test system changes. Especially welcome on immutable systems.

https://www.codethink.co.uk/articles/2024/A-new-way-to-develop-on-Linux/


nmbl: we don’t need a bootloader

Tags: tech, linux, uefi, boot, system

Looks like GRUB days as the standard bootloader are counted. Booting straight using the Linux kernel could bring interesting benefits. Probably not doable on every hardware platform though.

https://fizuxchyk.wordpress.com/2024/06/13/nmbl-we-dont-need-a-bootloader/


Towards Idempotent Rebuilds?

Tags: tech, linux, packaging

An interesting puzzle to pursue. Is it possible to rebuild exactly the same binary distribution packages?

https://blog.josefsson.org/2024/07/10/towards-idempotent-rebuilds/


Binary Search Tree with SIMD

Tags: tech, simd, performance

Another interesting algorithm to handle using SIMD.

https://clement-jean.github.io/simd_binary_search_tree/


Making Python Less Random — Andrew Healey

Tags: tech, linux, system

It’s really a good reminder of how powerful ptrace is. You can nicely intercept and change the behavior of syscalls with it.

https://healeycodes.com/making-python-less-random


Cryptomator: end-to-end encrypt files in any cloud

Tags: tech, cloud, storage, cryptography, security, tools

Looks like a nice tool indeed. Might be handy.

https://www.andreagrandi.it/posts/cryptomator-end-to-end-encrypt-files-in-cloud/


Dirty writes – Surfing Complexity

Tags: tech, databases

A simple explanation about dirty writes during database transactions.

https://surfingcomplexity.blog/2024/07/05/dirty-writes/


PostgreSQL and UUID as primary key

Tags: tech, databases, uuid, performance

Forced to use UUID as primary key in a table? Then make sure to use them properly to not kill the performance more than necessary. Ideally use something else though.

https://maciejwalkowiak.com/blog/postgres-uuid-primary-key/


Unravelling attribute access in Python

Tags: tech, programming, python

Ever wondered how attributes work in Python under the hood? Here is how.

https://snarky.ca/unravelling-attribute-access-in-python/


Making an iterator out of a function | mathspp

Tags: tech, programming, python

An interesting Python construct to make iterators based on a simple function.

https://mathspp.com/blog/til/making-an-iterator-out-of-a-function


State of Text Rendering 2024

Tags: tech, gui, fonts

Very long read but will be an essential resource to have a fine understanding of text rendering in its current form.

https://behdad.org/text2024/


the software crisis

Tags: tech, quality, craftsmanship, engineering, complexity, history

Interesting musing about the “software crisis” which was declared in the late 60s. We’re coping with it by piling levels of abstractions but clearly we’re still not out of it. Our craft still needs to grow.

https://wryl.tech/log/2024/the-software-crisis.html


Standups: Individual → Teammate - by Kent Beck

Tags: tech, team, organization, agile

Good reminder that teams are made out of people. It’s good to look at the daily standups less as a technical management tool and more as a need to get into the work.

https://tidyfirst.substack.com/p/standups-individual-teammate



Bye for now!

Friday, 12 July 2024

KDE today announces the release of KDE Frameworks 6.4.0.

KDE Frameworks are 72 addon libraries to Qt which provide a wide variety of commonly needed functionality in mature, peer reviewed and well tested libraries with friendly licensing terms. For an introduction see the KDE Frameworks release announcement.

This release is part of a series of planned monthly releases making improvements available to developers in a quick and predictable manner.

New in this version

Attica
  • Gitignore: add VS Code dir. Commit.
Baloo
  • Revert "[Extractor] Change to QCoreApplication". Commit. Fixes bug #487628
  • Port QML modules to declarative type registration. Commit.
  • CI: update clang-format image. Commit.
Bluez Qt
  • Add core/primary services for LE Audio: CSIS, MCS, BASS, PACS, CAS. Commit.
  • Add comments to improve readability and maintainability. Commit.
  • Manager: Add list property for connectedDevices. Commit.
  • Manager: Sort out property shorthand in tests. Commit.
Breeze Icons
  • Use newly introduced ColorScheme-Accent. Commit. Fixes bug #446468
  • Improve excalamation point placement in data-warning/dialog-warning icon. Commit. Fixes bug #487626
  • Add generic translate icon, symlink crow-translate-tray. Commit.
  • Add non-symbolic base version of network-wireless-bluetooth icons. Commit.
  • Mark as non gui executables to have output on Windows. Commit.
Extra CMake Modules
  • Avoid spurious message about appstreamtest. Commit.
KArchive
  • Don't accept tar files with sizes < 0. Commit.
  • Fix crash with malformed files. Commit.
KAuth
  • Move dbus dep to backend & tests. Commit.
  • Gitignore: add VS Code dir. Commit.
KBookmarks
  • Gitignore: add VS Code dir. Commit.
KCalendarCore
  • Add missing QtCore dependency to QML module. Commit.
  • Add QML bindings for CalendarListModel and CalendarPluginLoader. Commit.
  • Add CalendarListModel. Commit.
KCMUtils
  • Plugins: Use help-about icons instead of dialog-information. Commit.
  • SettingHighlighterPrivate: Optimize connections. Commit.
  • Plugins: Use symbolic versions of the icons. Commit.
  • Declare missing QML module dependency. Commit.
  • Properly declare KQuickConfigModule as having attached properties. Commit.
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • Gitignore: add VS Code dir. Commit.
  • Ensure KCM load function is called only once. Commit. Fixes bug #487388
KCodecs
  • Remove unused static tables. Commit.
KCompletion
  • Gitignore: add VS Code dir. Commit.
KConfig
  • Fix two cmakelint types of issue. Commit.
  • Fix KCONFIG_USE_QML=OFF by moving ECMQmlModule behind the conditional. Commit.
  • Kconfig_compiler: Don't generate .moc include. Commit.
  • Kconfig_compiler: Allow generating QML type registration macros. Commit.
  • Kconfig_compiler: Use automoc for GENERATE_MOC. Commit.
  • Avoid unused variable definition. Commit.
  • Allow desktop files to be a symbolic link. Commit.
  • Fix compile. Commit.
  • KStandardActions: Use C++11 for loop. Commit.
  • Fix compile dependencies for windows. Commit.
  • Register KCoreConfigSkeleton to QML. Commit.
  • Provide a generic named USE_DBUS option to allow to choose if to use it. Commit.
  • Don't warn if no dbus enabled. Commit.
KConfigWidgets
  • Support USE_DBUS option to enable/disable dbus use. Commit.
KCoreAddons
  • Rename deprecated argument CLASSNAME to CLASS_NAME in ecm_add_qml_module() for KCoreAddonsPlugin. Commit.
  • Kurlmimedata: warn about problems while retrieving files from portal. Commit.
  • Decode os-release as UTF-8. Commit.
  • Use REQUIRED_QT_VERSION for dbus, too. Commit.
  • Provide USE_DBUS option to allow to turn on/off dbus parts. Commit.
KCrash
  • Add missing QtOpenGL dependency. Commit.
KDBusAddons
  • Gitignore: add VS Code dir. Commit.
KDeclarative
  • Add missing copyright to KeySequenceItem. Commit.
KDocTools
  • Remove silly goose from template. Commit.
  • Gitignore: add VS Code dir. Commit.
KGlobalAccel
KGuiAddons
  • KColorSchemeWatcherMac: update name in copyright. Commit.
  • Clipboard: Add roundtrip when setting clipboard. Commit. Fixes bug #466414
  • Support USE_DBUS option to enable/disable dbus use. Commit.
KHolidays
  • Add holidays for Germany, Bremen and Germany, Hamburg. Commit.
  • Add public holidays for Uzbekistan. Commit.
  • Holiday_pe_es - add Peru holidays per "Decreto Legislativo 713". Commit. Fixes bug #488513
  • Update russian holidays substitution for 2021, 2022, 2023, 2024. Commit.
  • Update holiday_ph_en - fix syntax error "of august" should be "in august". Commit.
  • Change optional name to Philippines. Commit.
  • Introduce a holidays file for Philippines. Commit.
KI18n
  • Make country name detection substring matching more strict. Commit.
KIconThemes
  • Flip logic when to use Accent/HighlightedText. Commit.
  • Support USE_DBUS option to enable/disable dbus use. Commit.
KIdletime
  • Port towards QNativeInterface::QWaylandApplication. Commit.
  • Gitignore: add VS Code dir. Commit.
KImageformats
  • Fixed wrong plugin options behaviour. Commit.
  • EXR: Full support for gray image/colorspace. Commit.
  • Disable JXR plugin due to crashes in JXRLIB. Commit.
  • PFM: Portable FloatMap read only support. Commit.
  • PXR: Pixar raster read only support. Commit.
  • Sanity checks (fuzzer). Commit.
  • XCF: fixed wrong composite on Grayscale images. Commit. See bug #476755
  • Fix compilation. Commit.
  • Use of metadata macro definitions. Commit.
  • JXR: added CMYK image to read test. Commit.
  • XCF: Increased maximum property size. Commit. See bug #426222
  • JXR support. Commit. See bug #451584
  • PSD: support native CMYK introduced by Qt 6.8. Commit.
KIO
  • Core: use QString to format number. Commit.
  • Remove unnecessary space character in empty text file template. Commit. Fixes bug #487768
  • Port DropJob to KJobWidgets::windowHandle(). Commit. See bug #488515
  • Use resolved executable in KProcessRunner::fromExecutable. Commit.
  • Handle using parent environment correctly in SystemdProcessRunner. Commit.
  • Prepare environment for systemd in SystemdProcessRunner. Commit.
  • Enable CommandLauncherJob tests (modulo _KDE_APPLICATIONS_AS_SERVICE on CI). Commit.
  • Kmountpoint: use advised call params when possible for libmount. Commit.
  • Remove unused includes. Commit.
  • Drop unused modifiers support from user agent code. Commit.
  • [kprotocolmanager] Remove unused http_config. Commit.
  • Remove unused includes. Commit.
  • Don't pass proxy config to workers. Commit.
  • Drop protocol proxying. Commit.
  • Deprecate KProtocolInfo::proxiedBy. Commit.
  • [ftp] Drop ProxiedBy=http. Commit.
  • Kfileitem.cpp: If fileMode is not unknown, set InitCalled to true in readUDSEntry. Commit. Fixes bug #485771
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • [webdav] emit ERR_UNSUPPORTED_ACTION when performing fileSystemFreeSpace on a file. Commit.
  • Add desktop file for kiod. Commit. Fixes bug #459054
  • Gitignore: add VS Code dir. Commit.
  • Kfileitemtest: testBasicDirectory relax size assertion. Commit.
  • Commandlauncherjob: emit error when no executable. Commit.
Kirigami
  • Fix: install KirigamiPrivate soversion. Commit.
  • Add Kirigmai.Action unit tests. Commit.
  • Fix static build. Commit.
  • Allow initializing a Kirigami.Action from a QAction. Commit.
  • NavigationTabButton: improve text legibility. Commit. Fixes bug #489573
  • Dialog: Add horizontal padding to prevent content from leaking onto borders. Commit.
  • ActionTextField: de-duplicate action icon code. Commit.
  • Dialogs/Dialog.qml: removed colorSet. Commit.
  • ActionTextField: fix QML errors. Commit.
  • Update build.gradle in application template to Qt 6. Commit.
  • Remove unused KItemModels import. Commit.
  • Fix: The issue of abnormal touch events. Commit.
  • Relax property types to QtQuick.Templates counterparts. Commit. See bug #487904
  • Units: decouple grid unit from font metrics. Commit.
  • Fix typo. Commit.
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • PlaceholderMessage: Announce the text and description to a11y. Commit. Fixes bug #482753
KItemModels
  • Kdescendantsproxymodel: fix nested signals on source model reset. Commit.
  • Kdescendantsproxymodeltest: use model tester in the tests where it passes. Commit.
  • Port kdescendantsproxymodeltest to std::unique_ptr. Commit.
KItemViews
  • Relicense files from LGPL-2.0-only to LGPL-2.0-or-later. Commit.
  • Gitignore: add VS Code dir. Commit.
  • KWidgetItemDelegateEventListener: remove useless destroy event sending. Commit.
KJobWidgets
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • Gitignore: add VS Code dir. Commit.
KNewStuff
  • Remove Icons view. Commit.
  • Remove user-facing view switcher. Commit.
  • Use Tiles view by default. Commit.
  • Add punctuation to warning. Commit.
  • BigPreviewDelegate: Fix "Update" action. Commit.
  • Action: Remove dead code. Commit.
  • Gitignore: add VS Code dir. Commit.
KNotifications
  • Fix missing HAVE_DBUS usages. Commit.
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • Adapt Android default notifications and documentations to KF6 search path. Commit.
KNotifyConfig
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • Port to KStandardActions. Commit.
KPackage
  • Remove stray lib. Commit.
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • Gitignore: add VS Code dir. Commit.
KParts
  • PartLoader: Allow reading of InitialPreference from KParts object. Commit.
  • PartLoader: Add mechanism to provide capabilities standardized in JSON metadata. Commit.
KPeople
  • Add missing send. Commit.
  • Remove no longer used var. Commit.
  • Fix non-dbus path. Commit.
  • Support USE_DBUS option to enable/disable dbus use. Commit.
KPlotting
KPTY
  • Fix "Connecting from COMPAT signal" warning. Commit.
KQuickCharts
  • Controls: Don't use visible for hiding text in LegendDelegate. Commit.
  • Controls/legenddelegate: Always hide name if its width is 0. Commit.
KRunner
  • Fix build with QT_NO_CAST_FROM_ASCII. Commit.
KService
KStatusNotifieritem
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • Gitignore: add VS Code dir. Commit.
KSVG
  • Fix masks of completely black mask frames. Commit.
  • Restore proper devicepixelratio when extracting from cache. Commit. Fixes bug #488295
  • Fix description in metainfo. Commit.
  • Remove explicit maintainer from metainfo. Commit.
KTextEditor
  • Add a tool to test javascript scripts. Commit.
  • Fix JS API result of view.executeCommand(). Commit.
  • Kateregexpsearch: fix FAST_DEBUG. Commit.
  • API documentation: fix typo on kte_design page. Commit.
  • Add action to popy the current file name and line. Commit. Fixes bug #488027
  • Prefer even indent sizes. Commit. Fixes bug #474505
  • Try different way to read file for digest. Commit. See bug #482800
  • Remove gradient, looks ugly. Commit.
  • Add missing :, are there for all other entries. Commit.
  • Don't remember encoding if known to be broken. Commit. Fixes bug #445015
  • Ensure we don't keep old session config keys. Commit. Fixes bug #486648
KTextWidgets
  • Gitignore: add VS Code dir. Commit.
KUserFeedback
  • Gitignore: add VS Code dir. Commit.
KWallet
  • Gitignore: add VS Code dir. Commit.
KWidgetsAddons
  • Fix the since version of KJobWidgets::windowHandle,setWindowHandle. Commit.
  • Kjobwidgets: Force setWindow() creating a window handle. Commit. See bug #488515
  • Kjobwidgets: Add functions to associate QWindow with a KJob. Commit. See bug #488515
  • KTitleWidget: restore size policies to Preferred & vertical centering. Commit. Fixes bug #488319
  • KTitleWidget: do not autoFillBackground by default, drop StyledPanel frame. Commit. Fixes bug #475898
  • Make compile fine without qt6.7 deprecated methods. Commit.
KWindowSystem
  • Fix compiler error when memfd is absent. Commit.
  • Gitignore: add VS Code dir. Commit.
KXMLGUI
  • Remove old dead code from KDE4. Commit.
  • Use static constexpr for global int value definitions instead of enums. Commit.
  • Revert "Avoid a crash in broken Konsole code for now, waiting for fix release". Commit.
  • Port away from deprecated signal QCheckBox::stateChanged. Commit.
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • Gitignore: add VS Code dir. Commit.
Network Manager Qt
  • Add missing find_dependency call on Qt6Network. Commit. Fixes bug #489094
  • Fixed occasional lag issue with ActiveConnection. Commit.
  • Recognize OWE in AP capabilities and connection security. Commit. See bug #464615
  • Add VRF and loopback device types. Commit.
  • Recognize the "loopback" connection type. Commit.
  • Fix check for device and AP cipher compatibility. Commit.
Prison
  • Adjust scanner-qt6.qml for newer QtMultimedia. Commit.
Purpose
  • JobView: Fix switch-case: add default branch. Commit.
  • Plugins/barcode: Fix up QML. Commit.
  • Widgets: Rework type registration and property initialization, improve QML. Commit.
  • Don't let the Heading overflow. Commit.
  • AlternativesModel: Use logging category. Commit.
  • AlternativesModel: Drop unused header. Commit.
  • Tests: Rework testing share tool. Commit.
  • Tests: Set initial properties instead of introspecting objects later. Commit.
  • Tests: Correctly check for additional positional arguments. Commit.
  • Tests: Remove duplicating calls to add help and version CLI options. Commit.
  • Tests: Add missing translation domain. Commit.
  • Drop QML import versions. Commit.
  • Fix some QML and C++ code style. Commit.
  • Use ellipses instead of triple period. Commit.
  • AlternativesModel: Use correct role for a tooltip. Commit.
  • Kdeconnect: Don't let the Heading overflow. Commit.
  • Namespace embedded resources. Commit.
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • Gitignore: add VS Code dir. Commit.
QQC2 Desktop Style
  • Combobox: Set focus on text field within combobox. Commit.
  • Delegates: Use inset to control spacing around hover/selection highlight. Commit.
  • Limit focus workaround to affected versions. Commit.
  • Itembranchindicators: Store QPersistentModelIndex instead of QModelIndex. Commit.
  • Itemindicators: Don't access thread-unsafe data from the render thread. Commit. Fixes bug #487850
  • Implement baselineOffset for more controls. Commit.
  • KQuickStyleItem: Mark baselineOffset method as const. Commit.
  • Implement SplitView with splitter handles. Commit.
  • Support USE_DBUS option to enable/disable dbus use. Commit.
Solid
  • Fix "Connecting from COMPAT signal" warning. Commit.
  • Support USE_DBUS option to enable/disable dbus use. Commit.
  • Narrow down types of backend objects to Ifaces::DeviceManager. Commit.
  • Doc: Port 0 to nullptr. Commit.
  • Drop a reference to non-existent interface class. Commit.
  • Add missing forward declaration. Commit.
  • Gitignore: add VS Code dir. Commit.
Syndication
  • Dublincore.h "organisation" -> "organization" (prefer en_US). Commit.
  • Gitignore: add VS Code dir. Commit.
Syntax Highlighting
  • Add CustomType highlighting and improve import highlighting. Commit.
  • Corrected Gleam syntax. Commit.
  • Add Gleam syntax. Commit.
  • OrgMode: add header-item folding, properties folding, more todo keywords. Commit.
  • Avoid that we try to open files without name. Commit.
  • [dracula.theme] Fix "ISO/Delphi Extended" for Pascal. Commit.
  • Nginx: update for new directives and variables. Commit.
  • Systemd unit: update to systemd v256. Commit.
  • Add basic syntax highlighting rules for opsi-script. Commit.
  • [dracula.theme] Fix "Special Variable" for PHP. Commit.
  • Add Typst highlighter. Commit.

Thursday, 11 July 2024

OPC UA: Programming against Type Descriptions

OPC UA client code that relies on hardcoded NodeIds is brittle and often only works with a specific OPC UA server instance. This article shows the proper way to write robust and portable OPC UA client code.

Continue reading OPC UA: Programming against Type Descriptions at basysKom GmbH.

We are happy to announce the release of Qt Creator 14 RC!

Qt for Android usually supports a wide range of Android versions, some very old. To keep the supported versions to a level that’s maintainable by Qt, especially for LTS releases which are expected to live for three years, Qt for Android is adopting new guidelines for selecting the supported versions for a given Qt release in the hope that this effort would make the selection clear and transparent, and help shape proper expectations of support for each Qt for Android release.

Building system images for embedded devices from the ground up is a very complex process, that involves many different kinds of requirements for the build tooling around it. Traditionally, the most popular build systems used in this context are the Yocto project and buildroot.

These build systems make it easy to set up toolchains for cross-compilation to the embedded target architecture, so that the lengthy compilation process can be offloaded to a more beefy machine, but they also help with all the little details that come up when building an image for what effectively amounts to building an entire custom Linux distribution.

More often than not, an embedded Linux kernel image requires changing the kernel config, as well as setting compiler flags and patching files in the source tree directly. A good build system can take a lot of pain away for these steps, by simplifying this with a declarative interface.

Finally, there is a lot of value in deterministic and reproducible builds, as this allows one to get the very same output image regardless of the context and circumstances where and when the compilation is performed.

Introducing Nix

Today we take a look at Nix as an alternative to Yocto and buildroot. Nix is a purely functional language that fits all of the above criteria perfectly. The project started as a PhD thesis for purely functional software deployment and has been around for over 20 years already.  In the last few years, it has gained a lot of popularity in the Server and Desktop Linux scene, due to its ability to configure an entire system and solve complex packaging-related use cases in a declarative fashion.

In the embedded scene, Nix is not yet as popular, but there have already been success stories of Nix being used as an alternative to Yocto. And with the vast collection of over 80,000 officially maintained packages in the nixpkgs repo (this is more than all official packages of Debian and Arch Linux combined), Nix certainly has an edge over the competition, as most software stacks are already packaged. For most common hardware you will also find an overlay of hardware-specific quirks in the nixos-hardware repository. However, since the user demographic of Nix is slightly different at the moment, for more obscure embedded platforms you are still much more likely to find an OpenEmbedded layer.

For the Nix syntax it is best to refer to the official documentation, but being a declarative language, the following parts should be easy to comprehend even without further introduction. The only uncommon caveat is the syntax for lambdas, which essentially boils down to { x, y }: x + y being a lambda, that takes a set with two attributes x and y and returns their sum.

Cross-Compilation

On Nix there are two possible approaches to do cross-compilation. The first one would be to just pull in the packages for the target architecture (in this case aarch64) and compile it on a x86_64 system by configuring qemu-user as a binfmt_misc handler for user space emulation. While this effectively cheats around the actual cross-compilation by emulating the target instruction set, it has some advantages such as simplifying the build process for all packages, but most importantly it allows to reuse the official package cache, which has already built binaries for most aarch64 packages. While most of the build process can be shortcut with this, packages that need to be actually built will build extremely slow due to the emulation.

For that reason we use the second approach instead, which is actual cross-compilation: Instead of pulling in packages normally, we can use the special pkgsCross.${targetArch} attribute to cross-compile our packages to whatever we pass as ${targetArch}. The majority of the packages will just work, but rarely some packages need extra configuration to cross-compile. For example, for Qt we need to set QT_HOST_PATH to point to a Qt installation on the build host, as it needs to run some tools such as moc during the actual build on the build host. The disadvantage of this approach is that for most packages the official Nix cache does not provide binaries, so we have to build everything ourselves.

Of course builds are already cached locally by default (because they end up as a derivation in /nix/store), but it is also possible to set up a custom self-hosted Nix cache server, so that binaries have to be built only once even across multiple machines.

Building an image with Nix

As an example we will be looking into building an entire system image for the Raspberry Pi 3 Model B from the ground up. This includes compiling a custom Linux kernel from source, building the entire graphics stack including Mesa and the whole rest of the software stack. This also means we will build Qt 6 from source.

As example application we will deploy GammaRay, which is already packaged in the official Nix repository. This is to illustrate the advantage of having the large collection of nixpkgs at our disposal. Building a custom Qt application would not be much more involved, for reference take a look at how GammaRay itself is packaged.

Then at the end of the build pipeline, we will have an actual image that can be flashed onto the Raspberry Pi to boot a custom NixOS image with all the settings we have configured.

To build a system image, nixpkgs provides a lot of utility helper functions. For example, to build a normal bootable ISO that can install NixOS like the official installer, the isoImage module can be used. Even more helper functions are available in the nixos-generators repository. However, we do not want to create an “installer” image, instead we are interested in creating an image that can be booted straight away and already has all of the correct software installed. And because the Raspberry Pi uses an SD card, we can make use of the sd-card/sd-image.nix module for that. This module already does a lot of the extra work for us, i.e. it creates a MBR partitioned SD card image, that contains a FAT boot partition and an ext4 partitioned root partition. Of course it is possible to customize all these settings, for example, to add other partitions, we could simply append elements to the config.fileSystems attribute set.

Leaving out some slight Nix flakes boilerplate (we will get to this at the end), with the following two snippets we would already create a bootable image:

nixosConfigurations.pi = system.nixos {
  imports = [
    "${nixpkgs}/nixos/modules/installer/sd-card/sd-image-aarch64.nix"
     nixosModules.pi
  ];
  images.pi = nixosConfigurations.pi.config.system.build.sdImage;
};

This first snippet imports the sd-image module mentioned above and links to a further nixosModules.pi configuration, that we define in the following snippet and that we can use to configure the entire system to our liking. This includes installed packages, setup of users, boot flags and more.

nixosModules.pi = ({ lib, config, pkgs, ... }: {
	environment.systemPackages = with pkgs; [
		gammaray
	];
	users.groups = {
		pi = {
			gid = 1000;
			name = "pi";
		};
	};
	users.users = {
		pi = {
			uid = 1000;
			password = "pi";
			isSystemUser = true;
			group = "pi";
			extraGroups = ["wheel" "video"];
			shell = pkgs.bash;
		};
	};
	services = {
		getty.autologinUser = "pi";
	};

	time.timeZone = "Europe/Berlin";

	boot = {
		kernelPackages = lib.mkForce pkgs.linuxKernel.packages.linux_rpi3;
		kernelParams = ["earlyprintk" "loglevel=8" "console=ttyAMA0,115200" "cma=256M"];
	};
	networking.hostName = "pi";
});

Thus, with this configuration we install GammaRay, set up a user that is logged in by default and uses a custom Raspberry Linux kernel as the default kernel to boot. Most of these configuration options should be self-explanatory and they only show a glimpse of what is possible to configure. In total, there are over 10,000 official NixOS configuration options, that can be searched with the official web interface.

The line where we add GammaRay to the system packages, also automatically adds Qt 6 as a dependency. However, as mentioned previously, this does not work quite well with cross-compilation. In order for it to build we need to patch a dependency of GammaRay to add the QT_HOST_PATH variable to the cmake flags. What would involve bizarre gymnastics with most other build systems, becomes incredibly simple with Nix: we just add an overlay that overrides the Qt6 package, there is no need to touch the definition of GammaRay at all:

nixpkgs.overlays = [
	(final: super: {
		qt6 = super.qt6.overrideScope' (qf: qp: {
			qtbase = qp.qtbase.overrideAttrs (p: {
				cmakeFlags = ["-DQT_HOST_PATH=${nixpkgs.legacyPackages.${hostSystem}.qt6.qtbase}"];
			});
		});
	})
];

And note, how we pass the path to Qt built on the host system to QT_HOST_PATH. Due to lazy evaluation, this will build Qt (or rather download it from the Nix binary cache) for the host architecture and pass the resulting derivation as string at evaluation time.

In order to quickly test an image, we can write a support script to test the output directly in qemu instead of having to flash it on real hardware:

qemuTest = pkgs.writeScript "qemuTest" ''
	zstd --decompress ${images.pi.outPath}/sd-image/*.img.zst -o qemu.img
	chmod +w qemu.img
	qemu-img resize -f raw qemu.img 4G
	qemu-system-aarch64 -machine raspi3b -kernel "${uboot}/u-boot.bin" -cpu cortex-a53 -m 1G -smp 4 -drive file=qemu.img,format=raw -device usb-net,netdev=net0 -netdev type=user,id=net0 -usb -device usb-mouse -device usb-kbd -serial stdio
'';

Here we decompress the output image, resize it so that qemu can start it and then use the uboot boot loader to finally boot it.

Taking a final look at our config, we now have the following flake.nix file:

{
	description = "Cross compile for Raspberry";
	inputs = {
		nixpkgs.url = "nixpkgs/nixos-23.11";
	};
	outputs = { self, nixpkgs, nixos-hardware }: let
		hostSystem = "x86_64-linux";
		system = nixpkgs.legacyPackages.${hostSystem}.pkgsCross.aarch64-multiplatform;
		pkgs = system.pkgs;
		uboot = pkgs.ubootRaspberryPi3_64bit;
	in rec {
		nixosConfigurations.pi = system.nixos {
			imports = [
				"${nixpkgs}/nixos/modules/installer/sd-card/sd-image-aarch64.nix"
				nixosModules.pi
			];
		};
		images.pi = nixosConfigurations.pi.config.system.build.sdImage;
		qemuTest = pkgs.writeScript "qemuTest" ''
			zstd --decompress ${images.pi.outPath}/sd-image/*.img.zst -o qemu.img
			chmod +w qemu.img
			qemu-img resize -f raw qemu.img 4G
			qemu-system-aarch64 -machine raspi3b -kernel "${uboot}/u-boot.bin" -cpu cortex-a53 -m 1G -smp 4 -drive file=qemu.img,format=raw -device usb-net,netdev=net0 -netdev type=user,id=net0 -usb -device usb-mouse -device usb-kbd -serial stdio
		'';

		nixosModules.pi = ({ lib, config, pkgs, ... }: {
			environment.systemPackages = with pkgs; [
				uboot
				gammaray
			];
			services = {
				getty.autologinUser = "pi";
			};

			users.groups = {
				pi = {
					gid = 1000;
					name = "pi";
				};
			};
			users.users = {
				pi = {
					uid = 1000;
					password = "pi";
					isSystemUser = true;
					group = "pi";
					extraGroups = ["wheel" "video"];
					shell = pkgs.bash;
				};
			};
			time.timeZone = "Europe/Berlin";

			boot = {
				kernelParams = ["earlyprintk" "loglevel=8" "console=ttyAMA0,115200" "cma=256M"];
				kernelPackages = lib.mkForce pkgs.linuxKernel.packages.linux_rpi3;
			};
			networking.hostName = "pi";
			nixpkgs.overlays = [
				(final: super: {
					makeModulesClosure = x: super.makeModulesClosure (x // { allowMissing = true; }); # workaround for https://github.com/NixOS/nixpkgs/issues/154163
					qt6 = super.qt6.overrideScope' (qf: qp: {
						qtbase = qp.qtbase.overrideAttrs (p: {
							cmakeFlags = ["-DQT_HOST_PATH=${nixpkgs.legacyPackages.${hostSystem}.qt6.qtbase}"];
						});
					});
					unixODBCDrivers = super.unixODBCDrivers // {
						psql = super.unixODBCDrivers.psql.overrideAttrs (p: {
							nativeBuildInputs = with nixpkgs.legacyPackages.${hostSystem}.pkgsCross.aarch64-multiplatform.pkgs; [unixODBC postgresql];
						});
					}; # workaround for odbc not building
				})
			];
			system.stateVersion = "23.11";
		});
	};
}

And that’s it, now we can build and test the entire image with:

nix build .#images.pi --print-build-logs
nix build .#qemuTest
./result

Note that this will take quite a while to build, as everything is compiled from source. This will also create a flake.lock file pinning all the inputs to a specific version, so that subsequent runs will be reproducible.

Conclusion

Nix has been growing a lot in recent years, and not without reason. The Nix language allows to solve some otherwise very complicated packaging tasks in a very concise way. The fully hermetic and reproducible builds are a perfect fit for building embedded Linux images, and the vast collection of packages and configuration options allow to perform most tasks without ever having to leave the declarative world.

However, there are also some downsides when compared to the Yocto project. Due to the less frequent use of Nix on embedded, it is harder to find answers and support for embedded-related questions and you are quickly on your own, especially when using more obscure embedded platforms.

And while the Nix syntax in and of itself is very simple, it should not go unmentioned that there is a lot of complexity around the language constructs such as derivations and how everything interacts with each other. Thus, there is definitely a steep learning curve involved, though most of this comes with the territory and is also true for the Yocto project.

Hence overall, Nix is a suitable alternative for building embedded system images (keeping in mind that some extra work is involved for more obscure embedded platforms), and its purely functional language makes it possible to solve most tasks in a very elegant way.

About KDAB

If you like this article and want to read similar material, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

The post Using Nix as a Yocto Alternative appeared first on KDAB.

Wednesday, 10 July 2024

Hi there! The past few weeks have been really busy with my final exams, so I had to slow down my work. Here’s a brief status report on my progress over the past 4 weeks:

I created a SubtitleEvent class to help us better manage subtitle event information, which can replace the original SubtitledTime class. To distinguish subtitles from different layers, I also added basic display support for subtitle layers as multiple subtitle tracks.

Currently, I’m focused on refining these features. There are still some minor tasks to complete and bugs to fix. You can find more information at this MR.

Stay tuned!

Tuesday, 9 July 2024

Every once in a while people ask me about my email routine, so I thought I’d write about it here.

Everything I do starts with the philosophy that work and project email is a task queue. Therefore an email is a to-do list item someone else has assigned to me.

Ugh, how horrible! Better get that stuff done or rejected as soon as possible so I can move on to the stuff I want to do.

This means my target is inbox zero; achieving it means I got all my tasks done. Like everyone, I don’t always achieve it, but zero is the goal. How do I work towards it?

#0: Separate KDE and non-KDE emails

When I’m not in KDE mode, I want to be able to turn that stuff off in my own brain. To accomplish this, I have a home email account and a KDE email account. I adjust all my KDE accounts to only send email to my KDE address.

#1: Use an email client app

To manage multiple email accounts without going insane, I avoid webmail. In addition to not supporting multi-account workflows, it’s usually slow, lacking useful features, and has poor keyboard navigation.

I currently use Thunderbird, but I’m investigating moving to KDE’s KMail. Regardless, it has to be a desktop email client that offers mail rules.

#2: Automatic categorization (0 minutes)

I configure my email client with mail rules to automatically tag emails with colored labels according to what they are, and then mark them as read:

This results in almost all newly-arrived emails becoming colored and marked as read:

When I get a new kind of automated email that didn’t automatically receive a color label, I adjust the rules to match that new email so it gets categorized in the future, too.

#3: Manual categorization (1-3 minutes)

When I first open my email client in the morning, everything will be categorized except 5-15 emails sent by actual people. To see just these, I’ll filter the inbox by unread status, since all the auto-categorized colored emails got automatically marked as read.

Then it’s time to figure out what to do with them. For anything that needs a response or action today, I mark it as urgent by hitting the “1” key. For anything that needs a personal response in the next few days, I hit “9” to tag it as personal and it becomes green. And so on.

Any emails that don’t need a response get immediately deleted. I never miss them. It’s fast and painless. Put those emails out of their misery.

#4: Action all the urgent emails (5-15 minutes)

Urgent means urgent; first I’ll go through these one at a time, and action them somehow. This means one of the following:

  1. If it’s from a person, write a reply and then delete the email.
  2. If it’s from an automated system, open the link to the thing it’s about in a web browser and then delete the email.

The email always ends up deleted! For people like us emails are not historical records, they’re tasks. Do you need to remember what tasks you performed 8 years ago on Tuesday, May 11th? Of course not. Don’t be a digital hoarder; delete your emails. You won’t miss them.

At this point I may realize that I was overzealous in tagging something as urgent. That’s fine; I just re-tag it as something else, and then I’ll get to it later.

#5: Action all the merge request emails (5-10 minutes)

Since my day job is “quality assurance manager”, these are important. I’ll go through every automated email from invent.kde.org about merge requests for repos I’m subscribed to and action them somehow:

  1. Open the link to the merge request in my web browser, and then delete the email.
  2. Decide I don’t need to review this particular merge request, and just delete the email.

More deletion! I never keep these emails around; they’re temporal notifications of other people’s work. Nothing worth preserving.

#6: Action all the bug report emails (5-15 minutes)

My web browser is now filled up with tabs for merge requests to review. Now it’s time to do that for relevant bug reports. I follow the same process here: open the bug report in my web browser because it needs a comment or other action from me, and then delete the email — or else immediately delete the email because it’s not directly actionable. Delete, delete, delete. It’s the happiest word when it comes to email. Everyone hates emails; delete them! Show them you mean business.

#7: Do actual work

At this point I’ve spent between 15 and 40 minutes just on email, ugh. Time to do some actual work! So now I’ll spend the next several hours going through those tabs in my web browser, from left to right. First reviewing merge requests, then handling the relevant bug reports (closing, re-opening, replying to comments, changing metadata, marking as duplicate, CCing others, etc). During this step, I’ll also triage the day’s new bug reports.

Sometimes I’ll check email again while doing these, since more will be coming in. It’s easy to delete or action them individually.

After all these tabs are closed, hooray! I have some time to be proactive instead of reactive! Usually this amounts to 0-120 minutes a day during working hours. I try to spend this time on fixing small bugs I found throughout the day, opening and participating in discussion topics about important matters, working on the KDE HIG, and sometimes helping people out on http://www.discuss.kde.org or http://www.reddit.com/r/KDE.

#8: Action all the rest of the emails (10-25 minutes)

Towards the end of the day I’ll look at the emails marked as “Personal” and “KDE e.V./Akademy” and try to knock a few out. It’s okay if I’m too tired; these aren’t urgent and can wait until tomorrow. After a few days of sitting there, I’ll mark them as urgent.


And that’s pretty much it! This is just my workflow; it doesn’t need to be yours. But in case you want to try it, here are answers to some anticipated objections:

Ugh, that sounds like it takes forever!

It really doesn’t.

On a Monday maybe it takes more like 35 or 40 minutes since there are emails from the weekend to process. But on Tuesday through Friday, it’s closer to 15-20 minutes. Often 10 on Friday. Thanks to the automatic categorization, all of this is much faster than manually looking at every email one by one, and much more effective than getting depressed by hundreds of unread emails in the morning and ignoring them.

Deleting emails is too scary, what if I need them in the future?

You won’t.

But if that’s too scary or painful, set up your email account or client app to archive “deleted” messages in permanent storage rather than truly deleting them. Just keep in mind that you’ll eventually run out of storage space and have to deal with that problem in the future. Once it happens, consider it an opportunity to reconsider, asking yourself how many emails you actually did need to dig out of cold storage. I’m guessing the number will be very low, maybe even 0.

This might work for your workflow, but I get different types of emails!

Maybe so, but the general principle of automatically tagging (but not moving) emails applies to anyone. I firmly believe that anyone can benefit from this part. Make the software do the grunt work for you!

What do I do about all of those the old emails in my inbox? There are too many, I’ll never get through them!

If you’re one of those people who has 50,000 emails in your inbox, select all and delete. You won’t miss any of them.

Seriously. All of them. Every single one. Right now. Just do it.

How do I know this is fine?

  • Old notifications about things like bug reports or merge requests are worthless because they already happened. Delete.
  • Old mailing list conversations long since dried up or got actioned without your input. Delete.
  • Old at-the-time urgent emails from important people are no longer relevant, because the people who sent them long ago concluded that you’re unreliable and decided to not contact you again. Because that’s what happens when you let emails pile up: you’re being rude to all the people whose messages you’ve ignored. Feel sad, resolve to do better, then delete.

The good news is that you can get better at this anytime, but it’s almost impossible without making a clean break with a messy past. You’ll be looking at old stuff forever and won’t have time for new stuff.

I just get too much email, it’s impossible to keep up no matter what I do!

You need to unsubscribe from some things. Maybe a lot of things. Longtime contributors to any project will have accumulated years worth of subscriptions to sources of emails that are no longer relevant. Prune them!

This may trigger Fear Of Missing Out. Recognize that and fight against it. You can almost always reduce your email load by unsubscribing from this stuff:

  • Activity in Git repos for projects you no longer contribute to.
  • Bug reports for products you aren’t involved in or responsible for anymore.
  • Medium to high traffic mailing lists that are mostly or entirely irrelevant to your present interests and activities.
  • Almost all the spam from LinkedIn.
  • All the spam from online stores, newspapers, political campaigns. The “unsubscribe” button will work, don’t give up!

Resist the temptation to filter these emails into folders that you tell yourself you’ll remember to look at once in a while. You probably won’t, and by the time you do, everything in them won’t be actionable anymore — if it ever was in the first place. Unsubscribe and delete!