tl;dr : There is nothing wrong with any of the mechanisms in the subject, you just have to be careful when combining them all.

Long title, but the combination is important. Recently, I had an embedded device on my desk, which drove me to claiming quite strongly that “this is not possible what is happening in front of me!!!” If you are familiar with setups of all the points above, this article might be interesting to you.

Starting from the very strange effects I had. The device itself in question has a read-only root file system, as it is common for embedded devices. On this device a QtQuick application is running and, because I have not the most efficient processor, QML cache is enabled. Instead of going the build-time cache generation route, I have a persistent writable partition on the device, on which the QML cache is generated and stored at first start of the QtQuick application after first boot. Note that the cache needs to be persistently stored since otherwise the whole performance improvement on startup is moot.

So far so good, everything works well… until we are starting to update the software or more precisely the root file system. For this example, and this will be important later, the update of the root file system just updates my QtQuick application and its QML file but not the Qt version. What I then see after the update and the following boot is a system where the QML application still looks like before the update. Looking deeper at the file system, everything seems fine, files are updated, even QML files are updated, but the application just ignores them. But even worse, the application now randomly crashes because the executable and the shared libraries apparently do not match to the QML code being executed. — I will shorten this up, because with the intro the problem is quite obvious: the QML cache is not being invalidated even if it should and old versions of the QML files are used to run the applications. But how can this be?!

How a File in QML Cache is Invalided

All the magic that decides if the cache file is up to date or not essentially is located in qv4executablecompilationunit.cpp. Reading that code, we find the following checks:

1. Check for the right magic key: Ie. here is a basic sanity check that tells if the cache was generated by the right tooling.
2. Check for the cache file having been created by using the same Qt version as is used to executed the application.
3. Check if the cache file was being created with the exact same QML_COMPILE_HASH: This value essentially is the git revision of the QtDeclarative module and thus forces a cache invalidation whenever the QtDeclarative module changes (see here for the generation process with Qt 6, in Qt5 it is similar but just with QMake). As I see this, this check is mostly a Qt developer use case with often changing QtDeclarative versions.
4. Check if the cache file fits to the QML source file: Since all the previous checks are about the Qt versions, there is a final check that checks if the last modified date of the QML file under question is the same or a different one as the one for which the data is in the cache.
5. Note that there is no further check for e.g. the file’s hash value, which is obviously a performance optimization topic because we are doing the whole QML cache process essentially to speed up startup.

I do not think that much further explanations are required that tell why this can break fundamentally when we are building a root file system image in some environment like Yocto that fixes all timestamps in order to make builds reproducible (Many thanks to the NixOS folks for their initial analysis! Apparently we independently hit this issue at nearly the same time.)

The Origin of the Modified Timestamp

Ok, we now know that the modified timestamp of the QML file (this is what eg. “stat -c%Y MyFile.qml” gives you as result) is the key ingredient for making the cache working correctly in our setting. For this, we have to differentiate between two ways how QML files might land on our root file system:

1. As ordinary files, which most probably are placed somewhere in /usr/lib/qml/…
2. As baked-in resource files inside the deployed binaries via the Qt Resource System.

The first case is fairly simple. Here, we have to look into the process on how the root file system image is created (in case of package based distros this is different and you have to check how the packaging system is handling this!). In my case, the root file system is generated by Yocto and there is a global BitBake value called REPRODUCIBLE_TIMESTAMP_ROOTFS, which forces all files inside the root file system to have the same modified time stamp during image creation.

The second case is more interesting though. Here the SOURCE_DATE_EPOCH environment variable is used to set the modified date of the source files to a certain value. Note that one needs such a mechanism in order to make the build really reproducible because one cannot rely on the file date extracted or checkout out sources, which also may further change due to patches being applied during the build process. Rather, we want to use the timestamp of the last commit or a timestamp that is baked into a release tarball.
Yocto, as most modern meta build systems, does exactly this and sets this value for the build from the source data (for details look into Poky). Further, rcc (Qt’s resource compiler) picks this value up and sets the modified timestamps of the QML files correctly while baking the files into the binaries.

Solving the Issue (for Yocto Builds)

Since Yocto already handles SOURCE_DATA_EPOCH correctly, just use a fixed REPRODUCIBLE_TIMESTAMP_ROOTFS and be happy And here, I urge you to not try workarounds like setting REPRODUCIBLE_TIMESTAMP_ROOTFS=””, because this triggers fallbacks at different places of the code. Eg. you will find /etc/timestamp being the actual time of the build but then you will note later that the modified time stamp of the files in the root file system is now the date of the latest Poky revision. Funny, heh ;D So, never rely of fallbacks, because the tend to be inconsistent.

A much better way is to couple the rootfs timestamp in your project to something that is regularly updated, at least once for every release. I am in the lucky position to usually have a meta-layer/submodule with release notes where I know for sure that there is a commit before every release. Thus, I can simply add the following line in my conf/local.conf:

REPRODUCIBLE_TIMESTAMP_ROOTFS:="${@os.popen('git -C "$BBPATH/../sources/my-layer" log -1 --pretty=%ct 2>/dev/null').read().rstrip()}"

It is not that important what exactly to use to initialize the timestamp, as long as it changes often enough, as long as it still makes your build reproducible.

And again, you only have to really worry about all of the above if you have a QML cache on a persistent partition while updating the root files system.

(German version: https://wordsmith.social/felixernst/dolphin-treffen-bei-barcelona)

Among my contributions towards KDE I am probably best known for becoming a dolphin maintainer recently. “Maintainers” are the people most responsible for something, and I share this responsibility with Méven Car these days.

The plan was to have a dolphin meeting at Barcelona, so I set off.

I think the most important topic for a self-organising and -acting international group like us, that swims in all seven seas, is that we meet from time to time at a coast to talk about common goals. Otherwise, everyone surfs and browses wherever the streams pulls them. Everyone has an idea about what is currently the most important thing to get to and then simply swims there as directly as they can. However sometimes it makes sense to discuss what really is the most important area to focus on. And that is exactly the topic I have chosen for our dolphin meeting.

It was new to me to act in a leading organisational role within KDE. Not only concerning Dolphin but also more generally within the ecosystem of penguin fans.

Okay, I think I have to drop the Doublespeak here because technical terms become necessary. To avoid any further confusion, I'll clarify now: Dolphin is an application. To be more precise: It is the default file manager of KDE. Méven Car and I share the responsibility of maintaining it. The text above wasn't really about animals at all!

We did meet in Barcelona and tried to figure out where the biggest areas of improvement for Dolphin might be.

Time Travel for Files and Folders?

Neal Grompa, who brings KDE's software to Fedora and other distributions, had the idea that Dolphin should have a feature that allows users to restore older states of files and folders. The practical application of this would for example be that a user — after ruining a report or article — could simply bring back an older version of that file. Or that someone — after accidentally deleting a file — could restore an older version of the folder containing that file and therefore restore the file itself.

Does that sound like magic to you? Is it even possible? Wouldn't it be necessary to have some sort of backup for this to work?

The answer to all these questions is “Yes”. You might not yet be aware that some modern file systems, which you might already be using while you are reading this, are already keeping old data around. They do this so that you can bring your computer back into a working condition if your system ever becomes dysfunctional. Popular file systems which have this as an integrated feature are BTRFS and ZFS.

While exploring the sights of Barcelona with Luca Weiss and Arjen Hiemstra, we discussed how such a time travel feature could be implemented in Dolphin, and I also researched the topic a bit on my own later: The problem I am currently seeing is that it is difficult to pinpoint where in the file system the older versions of files and folders are located. It turns out that at least for BTRFS there is no mandatory folder structure. While it is normally reasonably easy for users to find the old versions (e.g. in openSuse in “/.snapshot/SNAPSHOTNUMBER/snapshot”) the names could be anything, the age of the backup is not necessarily stored in an easily accessible manner either, and figuring out which files belong together isn't trivial either. What do we do if the file that should be restored has been moved to a different folder in the meantime?

Maybe I am missing something, but I am having difficulty inventing a reliable approach even if I ignore all the technical details for the moment. That doesn't mean that the project is impossible to implement. Not at all! Worst case one could simply search the full disk. However, I have to admit that this is too big of a project to programme on the side. If you are interested or capable to implement this nicely you would be the hero of some users who would otherwise not know how to recover the data. I am certain of that.

One would probably implement it as a KAbstractFileItemActionPlugin which opens a window in which the user could choose the version of the file or folder they want to restore.

Dolphin could be better at dealing with slow storage

In a way the most important thing for a file manager is that it quickly displays the data on storage media. We don't want the name Dolphin to only be for show: A dolphin is fast (fastest mammal in water at 64 km/h). Nothing blocks it in the oceans.

Disks, on the other hand, can be blocking which can be a problem for the other Dolphin. It might take a while for them to retrieve data especially when they are accessed over the internet or another “slow” network. But the access being slow isn't really a good excuse for Dolphin to stutter as well. Loading of data can obviously take some time but Dolphin should stay responsive and quick regardless of that.

More details on this topic can be found in our meeting notes: https://invent.kde.org/system/dolphin/-/issues/35#note_535555

Copying files and folders is wrongfully reported as completed

With some regularity users complain about data loss because they removed a storage device after the data transfer has been reported as completed.

In KDE we report a file transfer as complete when the Linux-kernel reports to us that the transfer is complete. Unfortunately, that isn't the full truth because the Linux kernel is a bit hasty in that regard. It considers a transfer as complete as soon as the data is “accessible from the new location.” However, that is a while before the data has actually been physically transferred.

I see three solutions to this: 1. Linux gains a new option so the reported transfer progress is based on physical data transfer progress. 2. We make sure ourselves that the transfer is complete before we report that it is. 3. (In the words of Kai Uwe Broulik:) “We show a very very angry message to the user when they unplug a device that is still mounted, so they will learn not to do that!”

Dolphin should allow users to manage root-owned files and folders

As you might know, the Linux security model can deal with multiple user accounts that act on the same data. Depending on the specific file or folder, different users can have different permissions when it comes to accessing, changing or executing the data.

The account that has full access to all data on the computer is typically called “root”. It is the administrator account on pretty much every computer.

If a user tries to run Dolphin with all the permissions of an administrator by using the programme “sudo”, which would allow them to edit any data on the file system, Dolphin denies this. The reason for this is that this action could potentially allow a hacker to seize full control over the computer. This blockade was put in place before I was part of KDE but users are still annoyed by it because it makes certain tasks more difficult for them. How can we improve this situation without introducing new security hazards for companies and users alike?

Harald Sitter created an alternative, more secure method (https://apachelog.wordpress.com/2022/08/04/kio-admin/) which allows users to manipulate all data. We might integrate it better into Dolphin eventually.

I also recently discussed a different more immediate solution with Nate Graham: We know that in the past years various methods have emerged to bypass the blockade. These methods, that reduce security, are quite popular. The blockade in Dolphin doesn't really stop users who want to do an insecure thing. So instead of trying to hinder users more effectively, we should take the chance and inform them about the dangers of what they are trying to do. If they still want to continue after that, we can't and shouldn't stop them. It could be a good idea to transform the block into more of a lock with an easy way to open it legitimately.

So much for what was discussed at the Dolphin Meeting. The rest of this article is about other topics that are relevant to me.

Dolphin for Phones?

In Barcelona I talked a lot with the young developers who strive to make an adapted version of KDE Plasma a success on mobile phones. I hope this software will soon become a viable alternative to Google's Android for the average user. There is some interest to have the file manager that people know and love from their computer also available on their phone.

They didn't know — and neither do you probably — that Dolphin is already so flexible and touch-friendly that not too much work should be necessary to bring Dolphin in a state that is nice to use on phones:

We would probably need a separate configuration for phones, so users can easily install Dolphin in such a fitted state. Are you interested in making Dolphin shine in the mobile space? Contributions are always welcome!

Dolphin and the “Blue Angel”

I talked with Joseph P. De Veaugh-Geiss, who supports the eco-friendly “Blauer Engel For FOSS” project, about the possibility of also certifying Dolphin with the “Blauer Engel”. Question is: What are the direct benefits we expect from such a move? Eventually, it could push governmental institutions towards using Dolphin but Joseph explained that they probably wouldn't switch to Linux for this alone. To his surprise — and maybe yours too — Dolphin already works on Microsoft Windows and to my knowledge even on macOS. It has some rough edges on Windows and nobody really takes care of those at the moment. Would that be worthwhile? If we were to popularise the Windows version, would that lead to a lot more free and eco-friendly computing? I am not sure if we shouldn't rather use our limited resources on other things.

You might notice by now that there is way more meaningful work to do in Dolphin alone than we can realistically undertake with our small group of volunteer developers. It would be great if even more friendly contributors would join the project all of a sudden. :)

Documentation in the Application

Another topic that is dear to my heart is that our software should also be usable by computer-illiterate users. We accomplish this with guides and help texts among other things. Some of my efforts to have more help available were successful. One example are the little help buttons which have made their way to various parts of KDE by now after I introduced them on the “Fonts” settings page only two years ago (https://invent.kde.org/plasma/plasma-desktop/-/merge_requests/51). In a similar vein I added a feature to many applications that allows users to invoke exhaustive help directly from an application's user interface. You might have noticed the little “Press Shift for more.”-notes which for example show up when you hover your mouse cursor over a button in Dolphin. In my opinion every KDE application should provide more help this way.

I was in the meeting about writing guides and documentation for the web page https://userbase.kde.org/Welcome_to_KDE_UserBase and tried to promote the idea there that it might make more sense in many cases to have help available directly where it is needed: in the application. Unfortunately, I didn't get the impression that I was able to convince the attendees of that. So instead, I'll repeat here that every step we put between the user and the available help leads to less users actually using that help. When a user wants to know what a button does, the help should be available right there either next to the button or invokable directly from the button.

On a positive note I noticed that some KDE contributors have already figured out the benefits of this feature. Kai is a fan for example. I hope it is only a matter of time until this new way of providing help is used as naturally as the two-year-old little help buttons are by now in system settings.

So much about my endeavours. If you read with interest until now, you might also be interested in my videos about KDE development: https://tube.tchncs.de/c/felix_ernst/videos

Thanks to KDE e.V. and Its Donors

Meeting the unique group that traveled to Akademy 2022 in Barcelona this year held many benefits for our future collaborations and therefore ultimately for our software. The text above is already way too long even though I nearly exclusively talked about Dolphin. So many other topics would be worth mentioning. Above all else how great it was for me to meet all these friendly KDE contributors in person for the first time.

I would like to thank KDE e.V. and the many donors to that organisation for paying the majority of my travel expenses. After personally meeting the people who manage the donation money, I can say with full confidence that donation to the KDE e.V. are in good hands and spent with diligence and strategy to ensure the long-term existence and growth of the wider KDE community. If you would like to donate to this non-profit entity, visit https://kde.org/community/donations/.

Let’s go for my web review for the week 2022-44.

The bird, the billionaire and the precipice. – affordance.info

Tags: tech, twitter, attention-economy

Spot on analysis. What could Musk do with Twitter? Why? Several theories but clearly this will impact and distort even more the information landscape.

https://affordance.framasoft.org/2022/10/the-bird-the-billionaire-and-the-precipice/

The SAFe Delusion – Information for decision-makers considering the SAFe framework

Tags: tech, agile, criticism, safe

Finally out of Google Docs it seems. Better version for sharing around. Still an interesting list of case studies and opinions around SAFe. I learned a few things, I didn’t realize it’s creation was so disconnected from the pre-existing agile community. It all seems to confirm my opinion that it’s better to stay away from it though. The few organizations I know of which use it are clearly very much in a command and control mode. This is going backwards.

https://safedelusion.com/

Phylum Discovers Dozens More PyPI Packages Attempting to Deliver W4SP Stealer in Ongoing Supply-Chain Attack

Tags: tech, python, security, supply-chain

OK, this is a weird but real supply chain attack on going in the Python ecosystem.

https://blog.phylum.io/phylum-discovers-dozens-more-pypi-packages-attempting-to-deliver-w4sp-stealer-in-ongoing-supply-chain-attack

Plasma 5.26 review - Pretty reasonable

Tags: tech, kde

Always pleasant to see a nice and positive review!

https://www.dedoimedo.com/computers/plasma-5-26-review.html

The type system is a programmer’s best friend

Tags: tech, programming, safety, type-systems

Definitely this! It’s important to model properly your domain and leverage smart value types everywhere it makes sense. This can prevent quite a few different types of bugs.

https://dusted.codes/the-type-system-is-a-programmers-best-friend

Early speed optimizations aren’t premature

Tags: tech, programming, optimization, performance

Good reminder that “premature” doesn’t mean “early”. Poor Knuth is so often badly quoted in the context of optimization that it’s really sad. The number of times I see “early pessimisation” on the pretense of avoiding “premature optimization”. Such a waste…

https://pythonspeed.com/articles/premature-optimization/

Good old-fashioned code optimization never goes out of style

Tags: tech, programming, optimization

What the title said, there’s nothing fancy about optimizations. It’s mostly well known knowledge, doesn’t change much over time or on different stacks… still it’s very satisfying.

https://pythonspeed.com/articles/old-fashioned-software-optimized/

How to communicate effectively as a developer

Tags: tech, communication, writing

If you like remote work, then you need to make sure your written communication is good. There’s a handful of proper guidelines in this paper. Good reminders.

https://www.karlsutt.com/articles/communicating-effectively-as-a-developer/

Bye for now!

Once again, I got the honor of the Halloween special for the News from KDE PIM post. Let’s see what happened during the last two months.

Akademy 2022

Akademy 2022 happened in early October. Since it was in Barcelona it was almost next door for me and I could easily attend by train. Of course, a PIM BoF occurred there. Here are a few notes of what was discussed there:

• we might move to a less aggressive dependency bumping strategy, supporting one or two KDE Frameworks past versions would help people working from distro packages to contribute
• the idea of merging the kontact and kalendar chat channels is floating around
• plans are being made for a PIM sprint (involving Plasma Mobile PIM and Desktop PIM) aiming at holding it in Toulouse somewhen during Spring 2023
• Kalendar will hopefully be splitted into reusable modules for other apps consumption (like Raven)
• the docker container for PIM development might be decommissioned as it’s apparently unused
• there’s growing interest into switching Akonadi to SQLite again, would help with the robustness, clearly will be quite some work though
• branching for Qt 6 will occur around the same time as Plasma

KMail

• Several bugs got fixed including Saving all attachments being broken (which was a fairly old bug)
• Now SMTP passwords are also stored using QtKeychain which makes it independent from KWallet (bug 441214), unfortunately more work is required on QtKeychain end to completely get rid of KWallet
• The crypto support in the message composer has also been improved which has been covered in more details in another blog post with images and videos, some more improvements are on the way.
• It is also now possible to resend an email you opened via KMail (courtesy of Christian Schoenebeck), definitely useful to send emails generated by another program without access to SMTP
• Language support has also been improved
  • More translators are available (Google Translate, Libre Translate, Yandex, Bing, Lingva you name it)
  • We can directly use LibreOffice auto-correction list now
  • Lots of bug have been fixed in the grammar plugin support

KOrganizer

KOrganizer handling of recurring to-dos has been improved. Since version 22.08.3 the “Toggle To-do Completed” action properly acts on past or future occurrence as you would expect. Indeed, when toggling an instance if it gets marked as done then all previous occurrences are considered done as well, if it gets marked as not done then all future occurrences are considered not done as well.

KDE Itinerary

KDE Itinerary gained support for boat and ferry tickets, ICAO VDS health certificates and many improvements to the travel data extractor which also benefit the KMail itinerary plugin. More details can be found in this blog post.

Kalendar

• Views now have a new “basic” mode useful for low performance or battery powered devices. In this mode the views are not swipe based and more static but significantly faster and less resource intensive.

• A refactoring occurred in the data models used in the backend. This led to overall performance improvements especially when editing events and tasks.
• The sidebar is now responsive and will adapt to still be useful when the window gets narrower. It will collapse in a strip or expand depending on the available width.

• Lots of bug fixes ranging from UI glitches to major crashes. Make sure to update to the latest version!

Kleopatra

• For OpenPGP smart cards supporting ECC Kleopatra now allows generating keys for the curves supported by the cards. Moreover, the keys stored on an OpenPGP smart card can now be (re-)generated separately. (T4429)

• The notepad now has a dedicated Import button. Previously, the Sign/Encrypt button changed to Import when certificate data was detected in the notepad. This behavior made encrypting certificate data impossible and wasn’t good for usability and accessibility. (T6188)
• Secret key backup of S/MIME certificates was fixed. (T6189)
• The Certificate Dump window now opens in the foreground when clicking “More Details” in the Certificate Details window for an S/MIME certificate opened from the Group Edit dialog. (T6180)
• Canceling an encrypt operation triggered from the notepad doesn’t show an error message anymore. (T6196)
• If one enters a bad passphrase when importing an S/MIME certificate in PKCS#12 format, then a proper error message is shown. (T5713)
• Fixed potential invalidation of keys referenced by commands which could cause those commands to fail unexpectedly. (T6157)
• Unusable actions are now disabled when the details of a remote key (i.e. the result of a lookup) are displayed. (T6201)
• The Certify dialog no longer offers expired user IDs for certification. (T6155)
• The keyboard focus indicator for labels is only drawn if the label received focus by a keyboard command. (T6111)
• Errors during Web Key Directory (WKD) lookups (which happen in the background when doing a key lookup) are now ignored. (T6202)
• Nicolas Fella fixed raising the main window on Wayland. (https://invent.kde.org/pim/kleopatra/-/merge_requests/28)
• The Qt binding of GpgME (used by Kleopatra, KMail, and other KDE applications) can now be built for Qt 6.

Misc

Now dialog box size in PIM applications are correct on HiDpi displays.

Community Analytics, What Else?

Of course, I couldn’t conclude such a post without taking a look at the community data analytics during the period. Let’s see how it looks for the past two months.

A big shout out to all those who participated in KDE PIM work the past couple of months. Thanks a lot! It’s an important job you’re all doing. You know who you are. Still, here is the activity graph of the past two months, so now everyone know who you are (offer them drinks if you get the chance!):

In the past two months, we had 30 different developers on KDE PIM (that seems rather stable compared to my last post two years ago). And of course, since my data is coming solely from git, this unfortunately misses people who helped with bug reports, docs, design etc. Whatever your contribution you can be all proud!

Now, how does the code contributor network looks like?

The three most influential ones are Laurent, Heiko and Volker. You can also see three extra persons pretty central to the network: Carl Schwan, Claudio Cambra and Sandro Knauß. Clearly all the work on Kalendar and crypto support is showing up.

How You Can Help

Next time, your name could show up in visualizations like the ones above!

Check out the KDE PIM Development wiki and find out how to take part into something that matters. If you want your share of this or need help to get started, feel free to contact us via the #kontact and #akonadi IRC channels on Libera Chat or via #kontact:kde.org and #akonadi:kde.org on Matrix.

Also, consider making a tax-deductible donation to the KDE e.V. foundation

Time is running and already a couple of weeks passed since I have been at this year’s Akademy in Barcelona. It was great to (finally again!!!) meet people in person and talk about free software projects, while eating tapas our having nice beer.

One of the topics on my agenda was the next iteration of our Yocto layers. At the moment we have two layers provided by KDE for downstream usage, “meta-kf5” and “meta-kde“. The first provides a simple integration of KDE Frameworks into Yocto projects and the second one is a set of KDE Plasma (Desktop, Mobile & Bigscreen) and KDE Gears applications, which is mostly focused on providing nice show cases of KDE software.

Using the opportunity to have most people in Barcelona for a live meeting and the missing ones at least via remote, we had a quite intense BoF session. For the details just have a look at the notes. For me, the main points and results are these:

• We finally have a wiki landing page for Yocto in KDE: https://community.kde.org/Yocto. This page is supposed to document the main workflows for our Yocto layers, give a first starting point for on-boarding and also explain to downstream users what we want to achieve. Most points are not there yet in all the details they are supposed to, but work is ongoing.
• Our next important goal is to finally get the Yocto layers running in the KDE CI. As an important preparation step, we now introduce Repo manifests that help us to organize the different meta-layers and their revisions we depend on. This is actually what most projects in the Yocto ecosystem are doing, so it’s also part of our housekeeping. As already some time passed since Akademy, the first manifest files are in place and are ready to use, just follow the README. As an important side-note, they not only help with CI but very much speed up bootstrapping your system in case you want to test our layers.
• And finally, I took over the Yocto layer maintainer role from Volker. To celebrating this, I directly ordered yet another 2 TB SSD, which now is already 41% full with Yocto artifacts

If you want to have a look at the KDE Yocto layers, best first have a short glance at the wiki page and then just try it out by using the default.xml manifest.

Backstory

The All-American High School Film Festival (hereafter, A-AHSFF) is a week-long event hosted in New York City. Students can submit their short films in advance, and at the event are exhibitions and rewards for the best-judged films. There is also a tech fair, this year on the 22nd of October. KDE community member Nataniel Farzan is one of this year’s selected entrants, and negotiated a booth for KDE at the tech fair!

There aren’t very many members in the KDE community in the US, but a few of us live in the area. Philip Cohn-Cort and I heeded the call (with the invaluable remote support of Paul Brown and Aniqa Khokhar).

Paul and Aniqa did most of the heavy lifting of preparing the event for success. They requested the budget, and ordered the stickers, tablecloth, and banner. Big thank you to both of you! All Philip and I did was prepare the computers and, of course, show up.

Arrival

For me, the trip is about an hour and a half by train. Since the event started early on Saturday and we needed to set up, my best option was to leave Friday and spent the night. Philip similarly has a bit of a journey into the city. Since the banner was running late, he stayed later at his place in the hope it might arrive.

I got to the hotel first and checked in. A cute and efficient way to spend a night! The wifi was useless so it’s a good thing everything was already downloaded.

Festival

Booth Setup

Philip and I arrived in plenty of time, an hour before the festival started. This turned out to be good, because the venue had an extreme shortage of power outlets and none close to our booth! There was only one outlet on the whole area. I had a long extension cord, but not long enough to run all the way across the room. Happily, we didn’t need much power to run three laptops, so the people setting up the booth in front of the outlet let us plug into their power strip.

With that out of the way, we plugged my power strip into the extension cord, covered the extension cord with mats also kindly offered by the Fotocare booth to prevent a tripping hazard, and got everything set up.

In order from left to right, Philip’s older Surface Book running KDE Neon, Appimage kdenlive, and Krita. My new laptop running Windows with kdenlive. And my old touchscreen laptop running Fedora, krita, and kdenlive.

Krita turned out to be a great addition to the demo. Students loved doodling with the pen touchscreens, and several attendees were interested in it for the ability to do animation.

Power strip might be an American English word? For those not familiar, here is a picture.

Student Engagement

Going into the festival, one key note from Paul kept ringing in my head:

At places like FOSDEM or Scale, KDE is relatively well-known… This is not the case for A-AHSFF. At A-AHSFF, booth staff will have to be pro-active and engage attendees as they pass.

This turned out to be completely true. I spoke with exactly one person (a student) who knew what kdenlive was. So I needed a pro-active strategy.

Across from our booth, the huge lights of the Fotocare booth could not be missed. So I used this to our advantage. As groups or individual students came to admire the lights, I would intercept them and ask simply, “Can I give you a sticker?”

After all, who doesn’t want a sticker? I gave away a lot of stickers this way.

Most visitors would say “Yes”, after which point I had their attention and could present my quick pitch. “This is for kdenlive, free video editing software written by volunteers.”

Student Reception

To be honest, I am surprised at how well my short pitch worked. Some (say, 15%) students were just not interested, and responded with some form of “Ok, thanks for the sticker” and wander off.

But most were at least intrigued, at which point I would launch into my second phase: “It’s totally free, it runs on Mac, Windows, and Linux, and it supports Adobe file formats. It’s running on these laptops, please feel free to have a seat and try it out.”

(The second phrase evolved a lot over the course of the day, trying to head off the questions I had already received, like “What do you mean, free? Is there a subscription?” and “Are you a startup?”)

Nearly every attendee who heard this part of my pitch would look at the laptops and examine them for a few seconds, maybe prod the computer, and most (say, 80%) would leave with some form of “Cool! I’ll check it out.” I imagine by the time their long day is over, most students will have forgotten. But hopefully a few will find their stickers tomorrow, and remember to install it.

Finally, there was the rarer group of students who would actually sit down, and were really interested. This only happened 5 or 10 times through the event, but those students were really engaged. I’m pretty confident they’ll look for their sticker tomorrow and install kdenlive.

Educator Engagement

At the same time as we were trying to engage students, I was trying to pick out and engage adults who were not just parents wandering around after their darting children, but educators interested in putting kdenlive in their classroom. Before the event got fully under way, I started an online form to collect data from interested educators. This form also evolved over the course of the event, but I intentionally kept it short and simple. Just an email entry, a few checkboxes, and a field for “anything else” (largely for our usage, so we could keep notes about them!)

We chatted with quite a few educators and everyone I chatted with put their information into my form. The overwhelming response was “Wow, this is so cool, I can send this home with my students so they can work when their school laptop (and thus, Adobe subscription) isn’t available.” I remember one educator who primarily worked for a large, wealthy university, but in the summers taught at a poorer school where he was excited to bring kdenlive. Another educator said his school district paid for Adobe subscriptions, but that would end as soon as his students graduated, and he was excited to be able to offer them kdenlive as a replacement.

Postmortem

Overall, I am very happy with how the festival turned out. We reached lots of excited students, I suspect many of them will try kdenlive in the near future. Moreover, we reached many excited educators, who thought kdenlive would fit in their classroom in one way or another.

At the start of the event, I commented that I wished the stickers had a QR code. Since our banner hadn’t arrived, we only had the smaller QR codes on pieces of paper on the table, and I doubt my shouts of “kdenlive.org” have stuck in any memories. I usually scan QR codes of things I am interested in during a busy event. That way, the browser tabs act as automatic reminders, long after I’ve forgotten the sticker in the bottom of my bag. However, I think our stickers did an appropriate job of allowing me to start conversations, as well as let students who were really excited remember what it was they were excited about. We had intended to pair it with a big banner with a QR code, which would have filled that hole. Several of the most excited students scanned the printed QR codes we had on the table.

With the relative sudden appearance of the event, the banner was unfortunately not able to arrive in time. I am mostly cross because that means that we now have a banner that we will not be using! However, with the laptops, tablecloth, and stickers, I think our booth was snazzy, and we were able to get plenty of attention.

Personally I haven’t ever been a big user of tiling windowmanagers such as i3, awesome and what not, is not much the workflow style I want 24/7 out of my desktop, but there is definitely something something to say about that kind of multitasking when it makes sense, when is important to see the status of multiple windows at once for some particular task.

Plasma’s KWin has since a long time a basic support for tiles via the quick tiling support, by either dragging a window at edges or corners, or via keyboard shortcuts. This feature is very good, but very basic, and while there are 3rd party tiling extensions such as Bismuth which is a very nice thing, but window geometry managing outside the core always can bring you only so far.

Over the last month I have been working to expand a bit the basic tiling capabilities, both the quick tiling with the current behavior and a more advanced UI and mechanism which lets the user to have a custom tiling layout. Here it is a very short screencast about it.

Tiling is done by a full screen editor done as a fullscreen effect where is possible to add /remove/resize tiles. When moving a window if the Shift modifier is pressed, the window will be dropped in the nearest tile. Resizing the tile in the editor will resize all the windows connected, and on the converse, resizing tiled windows will resize the tiles as well. (also the traditional split screen quick tile will gain this feature, so if you have 2 windows tiled to get half of the screen each, resizing one will resize both)

Another use case we thought about for this tiling feature is to address people which have those fancy new ultra wide monitors.

There are very few applications which have an UI that actually make sense maximized on such proportions (i could imagine perhaps KDEnlive, Krita and very few other productivity apps). Most applications just look wrong, but with custom tiling areas, those can become sub “maximization” zones.

It is not intended to be a full fledged replacement for I3 or Bismuth, but rather an hopefully robust mechanism in the core with a pretty minimal and unobtrusive default ui, then the mechanism will have scripting bindings which should allow for 3rd party extensions to use it and implement interesting variations of the tiling WM out of it.

For now the feature is quite basic, a notable thing missing which i still hope to get in time for 5.27 is having separate tiling layouts per virtual desktop and activity (for now they’re per-phisical screen), but hopefully the full thing should land on a Plasma 5.27 release near you.

Window outlines! Yet another KDE contribution by yours truly! This was fun. Not easy at all, but fun. I'm pretty happy how they turned out.

Breeze Dark

Breeze Light

I hope Nate you don't mind me taking the screenshots from your blog post, I'm just.. Lazy. I have no excuse. Lol.

For those who just want to see how it's made, here's link to the merge request: https://invent.kde.org/plasma/breeze/-/merge_requests/241

Also I am probably gonna make couple LOTR references due to talking about binding and light and dark and I'm sorry about that beforehand.

But why make them?

I have big problem making distinctions between windows if they have dark theme and on top of each other. Window shadows are often dark as well, so the windows just kind of blend into each other and the shadows do not help much. Disable shadows altogether and you got windows that just disappear into each other. This can even happen with light themes, sometimes the shadows just are not enough.

They also look a bit better with the outlines, in my opinion. They give some kind of "constraint" for the window and they ease my weird brain.

To sum up, it just makes the windows feel like they're their own entities which calms my brain, they look nice and they stop windows from blending into each other.

Where are they drawn?

First I had to figure out where the hecc I draw these outlines?

Well, I wanted to make them their own thing, separated completely from other elements. But this would've meant I would need to actually make more in-depth modifications to KDecoration (I think?) and make the outlines their own draw calls and whatnot.

So instead, I made them play nice with shadows. Basically, the outlines are just part of the shadow drawcall.

But what happens if shadows are disabled?

Something cheeky happens! I actually draw the shadows with 0% alpha channel!

    CompositeShadowParams params = lookupShadowParams(m_internalSettings->shadowSize());
    if (params.isNone()) {
        // If shadows are disabled, set shadow opacity to 0.
        // This allows the outline effect to show up without the shadow effect.
        params = CompositeShadowParams(QPoint(0, 4), ShadowParams(QPoint(0, 0), 16, 0), ShadowParams(QPoint(0, -2), 8, 0));
    }

So the shadows are always there, they're just invisible. But the outlines will show up.

Hold on, you could've just skipped drawing shadows entirely!

Yes, but actually no.

Before shadows are even drawn, they make some very very useful calculations for me. And on top of that, remember that outlines are part of the shadow drawcall. Outlines are just one more colored part of the shadow, basically.

No shadows, no outlines. In darkness together, I've bound them.

Bordering madness

I get it all look nice and fancy with borders, they look all nice and rounded on bottom border... And then I disable borders.

And everything looks off.

The bottom border disappears completely and leaves a sharp window edge. The outline then rounds behind the window. It looks bad!!!

So, if borders are disabled, we do a magic trick and draw the outline path on our own.

    // Draw window outline
    const qreal outlineWidth = 1.001;
    const qreal penOffset = outlineWidth / 2;

    // Titlebar already has an outline, so move the top of the outline on the same level to avoid 2px width on top outline.
    QRectF outlineRect = innerRect + QMarginsF(penOffset, -penOffset, penOffset, penOffset);
    qreal cornerSize = m_scaledCornerRadius * 2;
    QRectF cornerRect(outlineRect.x(), outlineRect.y(), cornerSize, cornerSize);
    QPainterPath outlinePath;

    outlinePath.arcMoveTo(cornerRect, 180);
    outlinePath.arcTo(cornerRect, 180, -90);
    cornerRect.moveTopRight(outlineRect.topRight());
    outlinePath.arcTo(cornerRect, 90, -90);

    // Check if border size is "no borders"
    if (borderSize(true) == 0) {
        outlinePath.lineTo(outlineRect.bottomRight());
        outlinePath.lineTo(outlineRect.bottomLeft());
    } else {
        cornerRect.moveBottomRight(outlineRect.bottomRight());
        outlinePath.arcTo(cornerRect, 0, -90);
        cornerRect.moveBottomLeft(outlineRect.bottomLeft());
        outlinePath.arcTo(cornerRect, 270, -90);
    }
    outlinePath.closeSubpath();

This part was actually fixed by Noah Davis: https://invent.kde.org/plasma/breeze/-/merge_requests/241#note_541478

So, uh, I'm not 100% sure what's going on here but it seems that:

• Draw a rectangle with top left and top right with an arc, since they're always rounded
• Check if we have border on or off
• Draw bottom corners with an arc if borders are on, or draw sharp lines if they're off

I think I got it right..? But I can say, my solution was more messy and I'm glad it's not there. It involved basically blending two rectangles together. This is much better!

Outline colors!

The biggest puzzle of this was actually how to get the color for the outlines. We could have just gone with basic black and white coloring here, but it would be too much contrast in some situations and look jarring to some.

So I started with a simple idea: Take the background color of the window, then light or dim it based on its lightness value (HSL). If lightness is equal or over 50%, then dim the color. Otherwise lighten it.

At first I used HSV, which caused a bit weird situations. In HSL, lightness is basically how much the color is lightened or darkened, but in HSV, the value dictates how the color acts under light. For this situation HSL was better since we're not playing with lighting values, but just want to know if the color is "dark" or "light" to our eyes.

Anyhow, here's some copy-pasta from the source files:

    auto s = settings();
    auto c = client().toStrongRef();
    auto outlineColor = c->color(c->isActive() ? ColorGroup::Active : ColorGroup::Inactive, ColorRole::TitleBar);

    // Bind lightness between 0.1 and 1.0 so it can never be completely black.
    // Outlines only have transparency if alpha channel is supported
    outlineColor.setHslF(outlineColor.hslHueF(),
                         outlineColor.hslSaturationF(),
                         qBound(0.1, outlineColor.lightnessF(), 1.0),
                         s->isAlphaChannelSupported() ? 0.9 : 1.0);

Hold on, you said background color of the window, but you're using titlebar color?

When using the background color ColorRole::Frame the problem is that when having colored titlebar, the outline color feels very out of place. It just doesn't look good.

But using ColorRole::TitleBar we get a fun colored outline effect when having a separately colored titlebar!

Also there's the whole qBound(0.1, outlineColor.lightnessF(), 1.0) thing. Like the comment says, if the lightness value is black or very dark, the outline will also be black or very dark. Binding the lightness avoids situations where the outline blends into the dark background color.

And of course, finally we check if transparency is enabled and set it accordingly to avoid awkward situations.

But colors are tricky. Some like this solution, some don't. I am probably going to make a drop-down setting where people can select does the outline use background or titlebar color, or are they just off.

Continuous Iteration

My part was done and the outlines got merged! It was long long long ride and I learned a lot. But I'm super happy I managed to do this part, now it just needs to be iterated on so we can get something that looks The Best.

Nate Graham already made the colors pop a bit more by fixing the lighten/darken values: https://invent.kde.org/plasma/breeze/-/merge_requests/263

Also there was initially a feature that if the window color was super light, we just didn't show outlines at all due to the outlines being so dim it created a weird blur effect as the outlines were gray, the shadow black and the window white. But with this tweak that doesn't need to be done anymore.

There's still probably work to be done on this, but the biggest hurdle seems to be over! Knocks on wood

As I said, I may make the settings section for this, but we'll see. I need a short break from KDE contributions, I got my own projects to work on too lol.

There's also been talk that we could use the separator (the | thing that separates buttons in menus etc etc) color as the outline color, and make that separator color modifiable. I think that would make A Lot Of Sense... But I would still keep the outline color as the titlebar color if the titlebar is colored. :)

Thanks!

Thanks for reading this ramble. I really enjoyed making this contribution and once again I learned A LOT. Open source contributions are great way to learn new stuff and I'm happy I get to make stuff like this for KDE. Also I've learned I kinda enjoy writing C++!

And thanks for all the comments and feedback and code reviews and everything anyone did to help me get this outline thing through. Based on the feedback I've read on Da Internets, this is something many people didn't realise they needed!

The past few months I’ve been rewriting the text layout engine used by Krita’s text tool. This is not the same as the text tool itself, which is still a super small rich text editor, but it is a prerequisite to getting any kind of new features into the text shape. We haven’t done any real improvements to text since the work for the last fundraiser we had for it, and that is because this needed to be done, it is a lot of work, an we had vowed to take care of resource management first, which, uh, took us so long and was so intensive that it covered the whole development cycle from 4.0 to 5.0, or a span of 5~ years. I’m not the only developer who can finally tackle a sore point, there’s work being done on audio, lots of file format updates, work on assistants, technology upgrades and more… But this blog is about text.

Some history:

The purpose of text in Krita is to provide artists an easy way to add text to images. One of the primary cases being to add text to comics, but other uses, such as adding small paragraphs of text, captions and creating headers are also accepted. It does not have to be able to show pages of text, or be able to fill the whole canvas, but it is expected that artists can convert a text to a path so they can make fine grained adjustments. The latter is usually used for sound effects in comics and graphical titles.

We previously used QTextLayout for our text shape layout. QTextLayout is used for Qt’s own text, in its labels and text edits. We initially had hoped it would be sufficient for putting text on an image, but sadly we kept coming across issues, partially ones caused by SVG having more complex needs, but also ones that were caused by QTextLayout no being designed for anything but Latin text. This includes things like no support for vertical text layout, but also, more painfully, certain kinds of technical choices which lead to decorative properties like underlines breaking joined scripts. We had to disable accelerators for our menus because of this; they made text that uses Arabic look glitchy. We also had an interesting bug caused by Qt setting the scaling on a text object in a way that we can’t access when only using QTextLayout, meaning all our font-sizes were wrong.

XML based markup like SVG in addition has some features which can only be computed as long as you understand the markup to be applied on a tree of nodes, while a lot of text layout engines like to think of them as ranges of text that have a format applied to them. This and other SVG specific features mean that we need to be able to customize how certain features are done significantly, and when using QTextLayout that generally resulted in a lot of workarounds, which in turn made the code quite intimidating.

Pango is another commonly used text layout engine, specifically the one used by all GTK projects. Inkscape uses it for its text layout. However, studying Inkscape I found that they too had workarounds for Pango, which, after the hell that was dealing with QTextLayout was something I had no interest in. Far more troublesome is that I couldn’t figure out how to use Pango without Cairo. Cairo is a painting engine, specifically one that is used by the likes of Scribus and Inkscape for its vector drawing capabilities. But the thing is that we already have two painting engines inside Krita, the one used for text and vector being QPainter, and our own KisPainter. KisPainter is the one that is fully colour managed and also the one we optimize the most, and I would like it if we could at some distant point switch our vector shape drawing code to KisPainter as well, primarily because it’s the one that can handle floating point depths and also the one we do all the optimizations on. So adding Cairo just to draw text seems to go into the exact opposite direction I’d like to go in.

Scribus is interesting here because while it uses Cairo to draw, it doesn’t use Pango and has its own text layout code. The biggest difference between Scribus and Krita in this regard is that being a Desktop Publishing program, text layout is one of the core features of Scribus, so I imagine for them it was a case of “we absolutely need full control of the code”, where with Krita we would’ve been fine with a halfway solution as long as it didn’t result in bug reports that we couldn’t fix because we were using a library in a way it was never expected to be used.

At the same time, you absolutely don’t want to start from scratch. So our text layout doesn’t implement its own Unicode functions and algorithms, using libraries like Fribidi, libunibreak and Qt’s own offerings where it can, as well as using fontconfig for locating fonts and Freetype for retrieving glyphs from the font. The library that has made everything possible however has been Raqm which handles itemization (breaking text up in runs of similar font and script), calling Fribidi for the bidirectional algorithm and finally calling Harfbuzz for shaping (selecting the correct glyphs for a given string of text). We were able to submit patches for UTF 16 support, as well as some other small things, and it has greatly simplified laying out text, to the point I was able to get the basics of SVG 1.1 text up and running in about… A week or so. The only downside for us as KDE project is that it requires Meson+pkgConfig, while KDE projects generally use CMake. Harfbuzz too seems to be going this route, so it can’t be helped.

So, having laid down which dependencies we’re using I am now going over the peculiarities and problems I encountered in… The order I am making/updating the relevant automated tests.

Going over the tests.

Now, you might be thinking: “Shouldn’t you start with tests when dealing with some so reliant on an official spec?” And you’d be right, and I did always have tests that I checked against. They just weren’t automated:

The benefit of a big sheet like this is that, aside from looking cool, was something I could load into Inkscape and Firefox from time to time to see how compatibility was looking.

The real reason I avoided tests was because we already had a ton of tests, and a lot of them were broken to begin with, which probably never had much to do with whether they actually worked but rather that the fonts weren’t available. Which brings us to the first topic:

Font Selection.

So, typically font selection in Qt goes via QFontDatabase, which is both its own database as well as a front end to the various platforms preferred font selection mechanisms. I had not wanted to touch this, as it is perfectly decent at its job, save for the fact it behaves weird when you try to learn the font-stretch of a given QFont (it always reports 0). Sometimes you come across applications which keep hanging on the “font loading” stage, or which lag when you open up the font combo box, and this is because they don’t cache the fonts. QFontDatabase does, which makes it a huge pity that there’s no way to get ones hands on the filename of a given font. And we need that, because we need those to load the relevant Freetype faces. So one of the first things I had to implement was using Font Config to get the fonts.

The way FontConfig is typically used is that you create a pattern to which you add the things you want to search for. This is stuff like family names, but also font-weight (in combination with the FcWeightFromOpenType function), italics, and most usefully language. Fontconfig apparently keeps a list of characters for each language, allowing you to specify a language and it will then give preference to fonts that have characters for that language. This is kind of neat, given that a lot of font families need to be split up into smaller files because font files are (currently) not able to encode a glyph for each Unicode code point. I’m telling you all this because this isn’t actually written in the documentation, I had to read a whole lot of existing code and issues on trackers to learn this myself.

On the flip side, we can now implement some CSS features QFontDatabase wouldn’t make possible, like a list of font families for fallback. For this I am first creating the pattern, then calling the sort function. Then I take the text I am matching for, and split that up into graphemes using libunibreak. Then for each grapheme I try to find the best match for the whole grapheme and mark those as needing that font.

You want to do this for the whole grapheme because emoji sequences for example are joined by zero width joiners, and if you do per codepoint matching instead of thee whole grapheme, this might get matched to a different font, which means that during the itemization process (where Raqm takes the text and splits it up into runs of text with the same font and script to pass it to Harfbuzz) the parts of the sequence will end up in different runs, meaning Harfbuzz cannot select the appropriate glyph for that sequence. If you have the problem where an emoji sequence looks like a sequence instead of a single emoji despite the font supporting it, this is the reason. Emoji aren’t the only place were this happens, though it is the easiest to test. Variation selectors and combination marks are also best used with the same font, but this is much harder to test for folks who don’t speak languages that use either. Anyway, the matched graphemes are then merged into bigger runs of the same font, and then passed to Raqm.

I still have no idea whether this is going to work on on all platforms, given fontconfig is very Linux specific.

The first tests I created in this area were a test for the grapheme breaking (given we rely on libunibreak for this, it is more a test for our code that splits a string into a string list based on libunibreak’s guidance), and a test for loading fonts that are part of the unit tests. Technically, CSS does have mechanism for adding outside font files with @font-face, but our parser doesn’t support that feature, and implementing it could get quite complicated, with additional questions like ‘should we then support embedding fonts in .KRA files?’, so for now it’s a much simpler method that is only available to the tests.

Now able to ensure we have the test fonts available, the next two tests were ones adapted from the web-test-platform: A test for selecting bold, and one for selecting fonts of the correct font-weight on a font with OpenType variations (not to be confused with CSS font-variant, or with Unicode Variation Selectors). The later broke because we tried to cache the fonts to speed up layout, which meant that that if you configure the same font for multiple variations (or font-sizes), it would only use the last configuration.

More font tests followed.

Better font fall backs for unicode variation selectors.

The Unicode variation selectors needed custom test fonts which contained a glyph, and then another with both a glyph and a variation selector, because what we want in the end is that if no font has a given glyph and a variation selector, it should take the next best font (the one with only the glyph). Unicode graphemes start with the most important codepoint first, so this is a case of keeping around partial matches if no complete match was found.

Bitmap fonts.

Most fonts these days are vector based instead of raster based, and while we need to render to bitmap to display the glyphs eventually, Krita instead takes the glyph outline from Freetype and renders that by itself later. This allows us to support things like color gradient and pattern fills as well as SVG strokes for the text outline. Still, there are fonts out there without outlines, often quite old.

Now, Krita’s support for rendering these isn’t great, because the way our vector layer coordinate system works is that everything is in Points (1/72th an Inch), and we cannot figure out how many pixels there are per inch (PPI) inside the vector shapes. This is both kind of a pain for bitmap fonts, but also for glyph outlines, as font-hinting will result in Freetype returning adjusted outlines depending on what it thinks the PPI is. This is very much a legacy thing that dates back to the time of KOffice, but no one has ever had the time to look into it…

… And neither did I, right now there’s a hack in place that surmises the PPI during the painting code and based on changes there it runs the text layout algorithm. It isn’t great, but it works. For the test I made a quick bitmap font using FontForge with several different sizes for a single glyph. The code is then checked on which size it will select depending on which size is requested. This is important for color fonts as one of the types of color font (which are used for Emoji) is a bitmap font. There’s one big difference between old fashioned pixel fonts and the new color pixel fonts, and that is that the latter should get resized to the desired size if the selected size is too big. I haven’t put in a test for that, because I have no color bitmap font (specifically of the CBDT type) to test with yet.

After those tests, it was finally time for the main event.

SVG 1.1 tests.

Text support in SVG 1.1 allowed for laying out a single line of rich text, and then applying transformations to it. So we can move around and rotate specific glyphs, make them follow a path, or use text length to squeeze or stretch them into a specific shape, and finally, you can select whether a text is anchored at the start, middle or end.

The most common misconception I’ve seen regarding SVG 1.1 text is the notion of a text chunk. Some people think they are what tspans define, that is, a styled chunk of text, but they’re not. They are an absolutely positioned pieces of text, which can be anchored in different ways. The official SVG 2.0 algorithm even calls them ‘Anchored Chunks’, which is probably the better name. This is important, because SVG 2.0 introduces a big change with regard to text chunks. Before, each text chunk had to be shaped separately, now, all text chunk are laid out in one go meaning that shaping (which is necessary for joined scripts like Arabic) doesn’t break across boundaries as long as the font object doesn’t change. There’s some discussion in issue 631 about whether this is undefined behaviour and thus can differ between implementations, but the SVG 2.0 text layout algorithm makes this implicit, as it says to layout SVG 1.1 text as a single line of unbounded length with a CSS-based text renderer. CSS-based text renderers don’t know about SVG text chunks, so shaping (and bidirectional algorithm) behaviour is defined. This has consequences, but more on that later.

This is the point where in test land I started fixing the old tests. To my pleasure, most of the tests had merely some anti-aliasing differences between QTextLayout and my implementation. So I mostly spend time ensuring that the necessary font is loaded, and then took the string of SVG that was being tested and made it into an external file so I could occasionally check in other programs how they handled the test.

Some of the tests actually got extended a little. For the transforms I had to add test for vertical, as well as test for per-glyph-rotation, as we had neither before. For the test where we test whether different fills can be applied to different part of the text, I added ligatures and a combining mark to see how it would handle that. What is supposed to happen is that the first color assigned to a grapheme is used for the rest of the grapheme, even if the rest of the code points are assigned a different color, as this gives the most consistent result. Interestingly, this only works with font-caching enabled, as that ensures spans that have otherwise the same font will also use the same font object, meaning it won’t get split up during itemization (and Harfbuzz, which handles the ligatures cannot create them when they are in different runs of text). We generally don’t want that to happen (because it messes up joined scripts), so we’re going to need to find a way to solve that.

Among the tests that broke were attribute tests, because in the shift to SVG 2.0, glyph-orientation-vertical which takes an angle, needs to be switched to text-orientation, which takes keywords, and more such small conversion problems. Texts orientation allows for controlling whether glyphs for horizontal scripts get rotated when being laid out in vertical text. Krita doesn’t do much with this attribute, as we still need to implement support for it in Raqm.

As an aside, you might be wondering how we’re dealing with some of the more intricate features of CSS, like padding, and stuff like display:table-block. And the answer is that we don’t have to: all child nodes of a text in SVG can only be inline, and SVG doesn’t use the CSS box model. This is because it would otherwise truly become too complicated.

The old test that gave me the most trouble was the right to left text. Now, here is were it starts to become ‘interesting’. The actual bug I had was fixed because I had forgotten to set the first absolute xy position to 0,0 if it was not set otherwise, but I still kept getting issues. You see, when I open it up in different browsers, I get different results:

My result in Krita was the same as that of Firefox, and it looked really wrong. Chromium looks the most correct here, right. However, after much contemplation, Firefox is correct. This is a side effect of SVG 1.1 text being laid out in a single line, as it means bidi-reordering is done over the whole paragraph. So a right-to-left text with two text chunks, which respectively end and start with left-to-right text, will have those end parts flipped around.

And then when we start positioning the text, a gap appears because of the flip of the two sets of glyphs. Arguably you could fix this by repositioning those glyphs so they’re snug together, or maybe something can be done with the bidi algorithm control points. I’m kind of hesitant however, because we’re not laying out a line (which would warrant the bidi algorithm to be only applied on the characters in the line, after line breaking), we’re positioning a logical chunk of text… So I’m unsure what to do here, and had to conclude Firefox is correct. I’ve submitted an issue to the SVG working group tracker.

Anyway, after this adventure, you can imagine why all the new tests are in triplicate: One for left-to-right, one for right-to-left and one for vertical.

TextLength

TextLength is an SVG feature where text is stretched or squashed to fit a particular length, with an option to only transform the spaces in between letters or the glyphs themselves. It can be nested, meaning that a tspan with this feature enabled can have a child node that also has a textLength.

The SVG 2.0 algorithm for this is mostly correct. Because it is possible to have nested text length, it needs a recursive function: you go down the tree, first handling the text length of child nodes, and then that of the parents.

There is two things it misses though: glyphs that follow a stretched piece of text need to be adjusted too, until the end of the anchored chunk. Furthermore, all glyphs need to be adjusted in the ‘visual order’, that is, the order after the bidirectional algorithm is done with it, not the ‘logical’ (pre-bidi) order. This means that if you’re done with a node, you need to afterwards look forwards (or backwards for right-to-left) until the next anchored chunk begins or the text ends, note down the visual index, and then adjust these glyphs in the visual order, with the total amount of shift that the text length causes, as long as their visual index is higher than the one you adjusted last.

Also, the SVG 2.0 algorithm is setup for adjusting spacing only, meaning that the last character is not taken into account for the shift-delta nor adjusted. If you are transforming both spacing and glyphs you will want to include the last character so everything stretches nicely.

After all these adjustments, text length starts working as it says it should work, and you’ll have a perfectly working SVG 1.1 feature no one really uses. So, from there we’ll now go to the SVG 1.1 feature that everyone wants to use…

Text on Path

Being able to arrange text so it follows a path is a pretty common use case for the kind of typesetting that tries to mimic calligraphy and lettering (whether these are two separate disciplines depends on how you approach calligraphy). So greeting cards, poster titles, etc.

The SVG 2.0 algorithm is clear here, and you should follow it. However, there’s a few caveats unique to features in SVG 2.0.

First up is that the first absolute xy transform on the first glyph in a textPath element needs to be set to 0,0, because otherwise you get problems with multiple textPath elements in one single text element.

Secondly, SVG 2.0 allows for trailing spans of text that is outside the path, but not in a new text chunk, and the spec says, “hang these at the end of the path”. This mostly works, except, of course, with right-to-left text.

Chromium looks correct, but it’s kind of wrong in a sneaky way, because this is the text flow:

This is probably what is expected, but algorithm-wise that is the start of the path:

Double checking, it seems that right to left text-on-path was just never really considered, and every implementation I have tried will not show right-to-left text unless the start offset is at 100%, which is on some level weird. So I have made an issue out of that.

Before I go on to discuss text-wrapping, there’s two more side things to discuss:

Baseline alignment

Baseline alignment was part of SVG 1.1, and it allows text to be adjusted based on metadata inside the font, but in SVG 2.0 it’s supported through CSS-inline, here it’s folded into vertical-align. So, technically speaking if you get your text out of a CSS-based text renderer, like the algorithm suggest you do, you shouldn’t have to worry about this. We’re not doing that, so I had to implement this myself.

Initially I had wanted to make it part of Raqm, given it seemed something that everyone could use, but I quickly failed at doing so, as baseline alignment is applied tree-wise. Once I figured that out, it was relatively easy to implement, as SVG 1.1’s description is fairly straightforward:

Make a recursive function that first makes a table or map of baseline meta-data that is inside the first font of the given text span (with fallbacks as defined by CSS inline), then go over each child node and call this function again, passing them this table. Finally, use the table and the table the function got from the parent node to adjust the glyphs.

Fonts that carry baseline meta data are kind of rare though, so it’s best to have defaults to fall back to, and in particular to use Harfbuzz 4.0, which has such defaults built-in. For the tests, I ended up using a test-font from here.

Text Decorations

This comprises of underlines and strike-throughs. Even if this should be something that is handled by the suggested CSS-based text-renderer, you will probably want to do your own implementation of this, if you want to have good looking underlines while text is on path. I had some trouble with this as no one had really figured this out, but eventually I found something out that made me happy:

You will need to make a recursive function, that first calculates the child-nodes, and passes the textPath path down to them if there is one. Then, you will wan to calculate the text-decoration boxes by joining the bounding boxes of the glyphs in the span in a way so that each text-chunk inside the span has it’s own decoration box. These are then used as the source for generating over lines, underline, strike-throughs, etc.

Now, for text-on-path, if you’re smart and know how to offset bezier curves, you should proly offset a part of the original path that corresponds to the width of each decoration box. I’m not that smart, so instead I create, per decoration box, a polyline which is as wide as the decoration box, but has a node every 4*underline stroke width (in the case of the “wavy” decoration-line-style, every other node is moved down 4*underline width as well, so it creates a nice even zig-zag). Then I use the same method as used for positioning text on path to adjust each node, and finally I use QPainterPathStroker to turn the path into a proper shape. These are then cached in the object that represents the text span, so that I can draw them before and after drawing the glyphs as CSS3-text-decoration wants, with the correct span colour and everything.

You will want to do this before doing the text-on-path alignment, so the glyphs have not been adjusted yet.

You can use per node offsets to stretch glyphs on path as well, with the caveat that straight parallel sections would be best off to be replaced by an offset bezier curve section, and this is especially necessary for Devanagari (as the connector line in fonts is usually a straight line), despite not doing that, I am keeping this around as a proof of concept.

So after all this, there’s the final important feature…

Inline-size

SVG 2.0’s biggest feature is the several text-wrapping options it introduces, some being simpler than others, and these are really necessary as doing paragraphs of text with simple chunk positioning is a headache. Inline-size is the simplest of these wrapping options, as it only says “wrap at this width”, with nothing really special going on there.

While my initial idea was to focus on SVG 1.1 features, I did want to get this in, as it requires a line-breaking library, so if we included this now, we wouldn’t need to add new dependencies for a while (at the least, until we want to hyphenate I guess). I later discovered that having libunibreak in was really useful because grapheme breaking helps a lot with font-selection. Anyway, right now we don’t have support for things like wrapping inside a shape yet, as I tried to focus on inline-size and text wrapping features.

There’s noting exciting here, a typical example of the greedy line-wrapping algorithm: Get libunibreak to find wrapping points, count the logical characters until you find a wrapping point, check if the total advance is higher than inline size, if not, add “word” to line (and adjust characters, etc.), else start new line and move word there, adjust previous line for line height reasons.

As you may guess from the word “line”, there was bidirectional algorithm things here too. officially, the bidirectional algorithm need to be applied after line breaking, but for us it happens before line breaking, because Raqm (via FreeBidi) handles it. And indeed, after I had implemented all CSS-text-3 things like overflow-wrap and line-height, I noticed that my bidirectional text was wrong, and that was the point at which my morale broke.

For a week. I managed to get a sort of fix by switching from counting in the visual order to the logical order and then recalculating the advance, which works fine for implicit bidi-reordering, but I need to investigate what happens when we introduce bidi-controls, which is fine-tuning for the algorithm that shouldn’t be necessary most to the time, but does exist for special edge-cases. In the worst case we’ll have to get wrapping to happen in Raqm, but if we ever want to have hyphenation, we might need to do that anyway…

Other than that, inline-size is a bit peculiar in that it defines the wrapping box as starting at the anchor, and ending at the width of the inline-size, with text-anchor (not text-align!) defining how the text is distributed. This makes sense from a specification stand point, as it means it is possible for a renderer to implement a simple as possible text-wrapping without immediately committing to CSS-text-3, though it does mean any kind of justification isn’t possible yet.

Wrapping inside a shape does allow for this, but I am going to implement that as a separate patch, partially because this one was becoming too big, and partially because I had no clear plan on how to tackle this when I decided the that no more new features were to be added to the current work. That said, I did go through all of CSS-text-3 to see which features I could implement…

Text-transform

The feature that allows you to set all text in all-caps, or lowercase, as a styling thing, without affecting the text itself. Where the complexity with East-Asian scripts is the sheer number of glyphs, and the complexity with joined scripts is that they have complex shaping rules, the complexity with Latin, Greek and Cyrillic is that at some point clerks in Europe decided that Important Words were going to have their first letter written in the slower formal strokes, while the rest of the letters would be written in the faster less formal strokes. Every language using these “Bicameral” scripts has on top of that their own rules what constitutes Important Words, some have their own rules which lowercase letter corresponds to which uppercase letter, and finally, there’s also different rules on how VERY IMPORTANT TEXT should look.

Thankfully, most of this case-mapping is thoroughly documented by Unicode, and if you have access to a library of Unicode functions it will have uppercase and lowercase functions at minimum. It does mean however that we need to have a language assigned to a text (line breaking and many font features require this too), and this is possible for text now, but I still need to figure out how to take the language set on the Krita document and have it be inherited as the default on text shapes which live in vector layers. Another thing which I am thinking of is whether we might be able to offer spelling check as a way to encourage artists to select the appropriate language for the text so it gets the best possible layout, though there is plenty more to do before I can do that.

That means that in Krita’s case we handle text-transform uppercase and lowercase by using QLocale’s to upper and lower functions. It’s going to be interesting how well this works, because these rely on ICU’s functions first, and then falls back on operating system functions and we don’t build ICU for Krita, by there’s a non-zero chance operating specific functions use ICU themselves.

For capitalization (which CSS simplifies to every first character of a word), you can get pretty far by finding every grapheme that follows a CSS word separator character and then only doing uppercase on those. You’ll have to create some language specific exceptions, like for example for Dutch to check if a J follows an I, because “Ijsbeer” reads like some kind of geographic feature, while “IJsbeer” is a Polar Bear.

Then there are two functions for East-Asian text layout, one to ensure the largest letters are chosen in situations where the text is very small, and the other to ensure that text lays out nicely in vertical situations. Full-size kana is a case of mapping the characters as defined by the table inside the CSS-text-3 specification. Full-width mapping in our case maps the relevant ASCII to the Unicode codepoints in the full-width and half-width forms page. For the rest, I use QChar’s decompositionTag() to see if “narrow” is part of it, and if so, replace with the decomposed form. Under the hood this also uses Unicode functions.

For the tests, I had wanted to adapt the ones for the web-platform-tests, but it seems that one tests all possible letters that have an uppercase, which was a bit too much for me, so I went with a greatest-hits version that tests the Latin alphabet, some Turkish text for the I, and then adapted the tailoring tests.

The downside of not implementing the uppercase function myself is that here is one test for Greek I don’t grasp. Namely, Greek uses diacritics, tonos, for lowercase text, but when the text is set to all-caps these tonos are removed (or reordered?). The uppercase function used by QLocale does all that. However, apparently this is not the case with capitalization. And I am unsure how to check for that, as I am unsure what is expected of these tonos. So I kind of need help here.

Same thing with the kana and half-width tests, with former testing some Katakana and Hiragana, an the latter with a bit of Latin, some half width katakana, some Hangul and a bunch of punctuation.

Line-break, word-break and overflow-wrap.

These are all was to refine the way line-wrapping is handled, with line-break largely handled by libunibreak (it supports strict and normal, we may need to patch it if we want loose at some point). Word-break is missing break-word because I didn’t have the concentration for it, spending it instead on overflow-wrap, which controls what happens when the words are too long to wrap.

Hyphenation technically belongs with them as well, but would require adjusting Raqm as shaping will need to be redone when hyphenation happens inside a ligature, in addition to using a library/hyphenation dictionary for the actual breaking. I’m delaying all that, as hyphenation is uncommon for text on images.

Line-height

This controls how far the lines are apart. The spec says that line-height:normal is up to the renderer, but the percent and ratio-based values are pretty well defined.

I have dyslexia, and while there’s different text-layout adjustments that help different people with dyslexia, mine is best served with bigger spacing between the lines, so I was kinda pleased I managed to get this to work.

White-space

White-space controls what happens with multiple spaces in a sequence, and replaces xml:space property as used by SVG 1.1. The white-space property for SVG 2.0 needs to use the one from CSS-text-4 because it needs to provide a correct fallback for xml:space="preserve". I managed to get some of this implemented, but I am not testing it yet and will have to go back to double check everything, because our parsing code removes duplicate white-spaces by itself, and that needs to be undone carefully.

Text-indent and tab-size

Text indent indents the first line, or all lines but the first. I got both hanging and regular text-indent to work, but can’t test each-line, because the parsing code is still removing hard-breaks as one of the white spaces it removes. When I was doing the tests I had another bug with right-to-left text, but this time it had nothing to do with bidi-reordering for a change. Rather, I had mixed up negative and positive values for right to left meaning that when I intended to subtract the text-indent value, I was instead adding it.

We were already testing tabs, so I extended that old test to also test tab-size.

Hanging-punctuation

Hanging punctuation is when you let punctuation go outside of the wrapping area. This is particularly nice with justified text, and I suspect it may also give less messy word-wrapping if the wrapping algorithm is allowed to consider some punctuation as being outside the boundaries. Hanging punctuation is a potential solution for this, which is why it made sense to me to implement it…

… And then I discovered that no one else has a full implementation of it, and that even Adobe InDesign doesn’t have it, making me very worried. Like, on a rational level I know it is likely because nobody wants to mess with their perfectly functional line wrapping code, but on some level I am worried it is actually because hanging punctuation is known to cause computers to eat puppies or something.

Final tests

After the inline tests I spend some time on adapting the CSS font-variant property tests on the web-platform-tests to SVG. These control whether OpenType features like ligatures and different character styles are applied during shaping. Krita previously only supported small caps, but with more direct access to Harfbuzz (OpenType support being one of its core features), it was easy to get this to work. It was mostly a case of copy-pasting the correct CSS-properties to be parsed, and then assembling a list of features for the given range and let Harfbuzz handle the rest. Right now only the CSS-fonts-3 features are supported, as CSS-fonts-4 requires parsing of @rules, which Krita doesn’t do yet.

The final test I wrote was a render test for ColrV0 fonts, using some OpenType test fonts from here. Krita officially supports ColrV0 and CBDT, but I still need font for the latter. I need to double check what is up with SBIX, and after that, the two big color fonts types are SVG and ColrV1. I didn’t bother with these yet, as I hadn’t figured out how to cache them properly (which is necessary both because of color palettes, as well as allowing the text to be converted to paths). I think I may have an idea how to now, but there’s more important things to be done first, and I suspect I still have a year or so before anyone starts to miss them. As with the font variants, @rules are not supported yet, so @font-palette-values isn’t either.

Future work

I am currently trying to clean stuff up, with my colleagues helping me with getting everything to build and helping me speed things up. Font-caching is a thing that absolutely needs to be done properly (maybe I should make sure that we can reuse it for the @rules later?). Text on path is a bit slow right now, and we may be able to speed that up. I am still kind of worried about the speed of the rendering, but we may need to make benchmarks before I can communicate my worries more clearly.

After that, I want to finish up the text-wrapping work, so we have proper white-space and text-in-shape handling. Probably fix bugs here and there too.

Then… doing preparation work so we have an interface to change text programmatically without having to write and parse SVG XML strings. And doing research, because there’s some common focus issues with on-canvas text tools that I’d really like to avoid.

After or intermediary to that I’d also like to get more improvements to East-Asian text layout, like text orientation, emphasis marks, and maybe even ruby (may seem unnecessarily ambitious, given there’s no other implementation doing this, but ruby annotations are an accessibility feature that is really common is East-Asian comics).

And there’s rendering things like better .nodef boxes for missing glyphs like the ones Firefox has. Or implementing paint-order for joined scripts, or the more advanced colour font formats. I don’t know yet in which order they will go…

Overall, text is complicated, sure, but it was also kinda fun to do? Like, it has a lot of research, and requires a lot of thinking about edge cases, and I’ll freely admit I was very upset whenever I found another new way for the bidirectional algorithm to make life difficult (though, it seems my code became simpler every time I fixed a right-to-left bug), but at the same time, I have seen a lot of programmers trying to do what I did, and at the research stage they give up, panicking at the size and fiddliness of it all (further fueled by articles like these making the rounds, said article is fully correct btw. We thankfully don’t have to worry about some of the elements, Krita being a painting program and not needing the ability to layout and render thousands of words like a web browser). I am still not sure why I was able to do all this, but if you are in my situation my tips would be:

• Try to figure out what has priority for your usecase. Krita as a painting program doesn’t need to do subpixel anti-aliasing, because more or less no other graphics program does that, which in turn is because type-setting of text in graphics programs is usually at a larger size/resolution. So I am not going to do that.
• Limit your scope. There’s a whole bunch of things in the parser or elsewhere that I am flat out ignoring for this particular project. If I had to implement a on-canvas tool in this same patch, or some of the obscurer ways of handling a CSS-value, I too, would go nuts.
• Do try to test multiple scripts, even if you can’t read them.
• Likely, someone out there will have tried to solve what you’re trying to solve, and if not, you do not need to be the first person to solve it. This is largely why I spoke against any suggestion to shift away from SVG for text, even though most painting programs don’t mix text layout and vectors: CSS is a very mature standard, and there is tons of discussion by people of all sorts of backgrounds on how to implement something, even if it also has stuff that only makes sense in an historical context.
• Examples of unsolved problems include, but are not limited to:
  • Using ligatures and kashida to improve the quality of Justification. There’s attempts out there, but the quality is not great, and no one knows what the best possible solution would look like (hell, as far as I can tell there’s even no consensus at which part of the stack it needs to happen).
  • Figuring out breakpoints for languages that have no word-spacers for the computer to recognize as break-points like Thai. There’s some rumours about using dictionaries to recognize the words, but that is too little information: what kind of dictionaries? And do we have to do some processing on those dictionaries as well, like a spelling checker would???
  • Apparently letter-grouping for Devanagari uses different groupings than unicode graphemes, but there’s no documentation on what that entails.
  • Getting the glyphs of joined scripts to merge so it doesn’t give a wrong outline. The big browsers may have their own problems here, but in Krita’s case we rely on qt’s path union operation, which is too crude for text glyphs, so hence why I am going to implement paint-order so artists can specify outlines to be drawn behind the main text, though it will still not solve everything.
  • How are color font glyphs supposed to be stroked, if at all?

And like, it’s very hard, no doubt about that, but it is not impossible.

P.S. Sorry to the Inkscape devs for the bugs I found but did not report: I can’t seem to get my gitlab.com login to work.