“Boiling the Ocean” is a sprint/hack days event series from GNOME with varying subjects,
which I and a few others from KDE have been joining recently to discuss and collaborate on overlapping topics.
I have written about the state of NFC support for Plasma Mobile back in August,
and a lot of this is common infrastructure for all Linux platforms, and it has to be to avoid making things
messy as hardware access tends to be exclusive.
Together with the GNOME team we looked at our options for the Linux NFC stack. Beyond
the ones mentioned in the above linked post there’s an additional one from Tizen.
We agreed however that for now we’d try to make neard work first. My fixes for
neard have meanwhile been integrated, which gives me some hope that might actually be viable.
The next steps would then be:
Expose ATS responses from ISO 14443-4 tags (sometimes also called “historical bytes”),
which is needed for identifying the type of a tag.
Implement exclusive access to tags.
Implement a raw command interface for tags.
With that we should be able to support e.g. the AusweisApp use-case or reading
FIDO2 tokens.
I hit a wall on the first item already unfortunately. There’s an attribute in the kernel API (NFC_ATTR_TARGET_ATS)
to retrieve this information from tag discovery information, which however always returns an empty result, as this
only becomes available later, during tag activation. And at that point there is no way to read it out anymore
as far as I can tell.
All this is just a start though, there’s much more to do:
Implement Host Card Emulation (HCE) support in neard.
Implement NFC pass-through in Waydroid.
Figure out how to align/unify the Linux kernel NFC API and the PC/SC API.
Expose NFC and/or PC/SC APIs via XDG portals.
Make neard work without requiring root permissions.
I don’t have the same level of insight on the implementation on the GNOME side, but
with Glib support merged and
terminal intents implemented
this also looks well under way.
Emancipatory Computing
It’s not all about technical implementation details though. There’s also overlap in more fundamental questions
on the vision of what we actually want to achieve and why we are doing all of this, in particular for those of us
who have different ambitions than just being a random part of a software supply chain.
Digitalzwang
We went through a similar discussion about 10 years ago in KDE, resulting in the vision
of a world in which everyone has control over their digital life and enjoys freedom and privacy.
That’s still valid today I think, but one aspect is starting to show its age as the world has been changing since then.
The distinction between the digital life and the “real” life have been disappearing as more and more aspects
of life involve some kind of digital interaction nowadays, such as mobility and travel, handling money, accessing your health records, or
for setting up a new appliance.
There’s the German term Digitalzwang describing this, which unfortunately doesn’t have a similarly compact
English equivalent, meaning being forced to use (proprietary) digital means to use a service or participate in a certain aspect of society.
Rejecting that and asking for a right to participate in all of that entirely without digital means is a valid
approach, but doesn’t address this entirely IMHO. I can travel with a paper ticket, but I wont be able to
benefit from accurate up-to-date information in case of disruptions then. I don’t want to miss that and I don’t want to be
forced to make a decision between a fully offline approach and using a proprietary app on a proprietary platform.
While there are technical aspects to addressing this, the bigger and harder part is actually political.
We need guaranteed and fair access to the necessary interfaces, standards and procedures to interact
with the “real” life things in order to provide free and open alternatives.
Self-hosting
Related to this is the topic of dependencies on cloud services. The GNOME team is doing interesting work with
peer-to-peer networking, which could address parts of this, e.g. when it comes to collaboration tools
and sharing your own data.
The ability to self-host is another part, and a necessary prerequisite. It’s not a solution in itself though,
as hosting services yourself isn’t really accessible for most people.
There’s also services where wide-spread self-hosting isn’t viable for other reasons, Transitous
is such an example. While that’s not even particularly complicated to set up for what it does, it only
scales in very large increments. This means you can’t start with something small like the stereotypical Raspberry Pi
but need a rather big server, and that’ll then be to handle way more requests than you likely need.
So we also need to figure out how can we run services as a community in a way that’s sustainable in the long run, and how can we
design fair usage rules for those.
Permacomputing
Another aspect is looking at the negative impact our work causes, and minimizing this, like KDE Eco
has done and is doing.
While ideas like permacomputing and hardware longevity were initially
motivated by considering the climate and human rights impact of buying new devices, in the current situation it has
become a simple necessity given the very limited availability of (affordable) hardware.
None of the above is something that anyone of us can solve on our own,
this needs much bigger coalitions than just GNOME and KDE even.
But with the distinction between the “digital life” and the “real life” fading away
and rampant enshitification there’s also many more people affected and interested in these topics now.
Outlook
That’s just picking three topics from three different events, there are many more things
that have been discussed and which benefit from coordination and collaboration,
such as XDG portal APIs for contact and calendaring data, or scalability issues we observe with
the current push notifications design, so there’s more to come.
This week a significant technical milestone was achieved: the XWayland compatibility layer is now loaded only when needed. Previously it had always been running even when no apps were using it, and fixing this required porting various system services away from legacy X11 code paths. This work was done by Vlad Zahorodnii, David Edmundson, and David Redondo, and will ship in Plasma 6.9.
But that wasn’t all, not by a long shot! Check it out here:
Notable new features
Plasma 6.9
You can now quickly forget networks from the Networks widget, not just its associated page in System Settings. (Peder Bergan, KDE Bugzilla #525852)
The System Monitor app and widgets now include a sensor that lists the number of physical CPU cores. (Méven Car, ksystemstats MR #157)
Notable UI improvements
Plasma 6.9
System Settings’s Task Switching page has been ported to QML, with a substantially improved UI and the Overview effect’s settings included. (Oliver Beard and Tobias Fella, kwin MR #7604)
System Settings’ Date & Time page has also been ported to QML, with a simplified single-page design that makes it faster to change the time zone when needed. (James Graham and David Edmundson, plasma-desktop MR #3653 and plasma-workspace MR #6884)
The desktop wallpaper configuration page now reserves more space for wallpapers, thanks to moving the combobox used for selecting the desktop type over to the “Layout” page. (Ramil Nurmanov, plasma-desktop MR #4113 and plasma-workspace MR #7294)
New
Old
The Folder View widget and desktop icons now give you more size options, filling in a huge size gap between the largest size and the size right before it. (Nathan Heller, KDE Bugzilla #521201)
When a grouped Task Manager task is flashing to get your attention, opening its group popup now reveals which specific window wanted attention. (Christoph Wolk, plasma-desktop MR #4225)
The Timer widget now lets you change a timer’s duration in the conventional way, without having to scroll over it. This also fixes some theming-related glitches. (Christoph Wolk, kdeplasma-addons MR #1251 and KDE Bugzilla #522797)
When using the Media Frame widget to show only a single image, it no longer cross-fades between that image and itself. (Christoph Wolk, KDE Bugzilla #518053)
The Comic Strip and Media Frame widgets now use consistent floating toolbars for their “Previous” and “Next” buttons, improving visual consistency and Fitts’ law adherence. (Christoph Wolk, kdeplasma-addons MR #1236 and kdeplasma-addons MR #1238)
System Settings’ Keyboard page no longer lets you toggle custom layouts on or off; it assumes that you want to use the system-wide keyboard settings until you add some custom ones, and now it clearly informs you of this. (Tobias Fella, plasma-desktop MR #3825)
Breeze-styled sliders now have nicer-looking tickmarks when using a fractional scale factor. (Eike Hein, breeze MR #649)
Plasma’s network share setup wizard is now more intelligent about populating its settings fields with information from a pasted URL, including the network protocol and whether the connection should be encrypted or not. (Méven Car, plasma-desktop MR #4216)
Job transfer notifications now show a prettier representation of the destination in cases where the destination is a special URL like desktop:/ or trash:/. (Tobias Fella, plasma-workspace MR #7391)
The Media Controller widget no longer shows tickmarks on its playback slider for media shorter than a minute in length. (Christoph Wolk and Yuma Kakei, plasma-workspace MR #7260 and libplasma MR #1608)
The Wobbly Windows effect (which you should absolutely be using) now includes some fun named presets for different levels of wobbliness. (Antti Savolainen, kwin MR #10308)
Frameworks 6.33
Throughout Plasma and Kirigami-based KDE apps, buttons that show text will no longer ever display a hover tooltip with the same text, because that’s silly. (Kai Uwe Broulik, kirigami MR #2232)
Qt 6.13
Focusing a QtWidgets-based KDE app by clicking on its hamburger menu button now changes all of the window’s colors to their focused versions, rather than only after the menu was closed. (Kai Uwe Broulik, KDE Bugzilla #452212)
Notable bug fixes
Plasma 6.6.8
The Task Manager widget’s textual list popup now looks correct when using a right-to-left language like Arabic or Hebrew. (Christoph Wolk, plasma-desktop MR #4197)
Fixed an issue that made pointers look a bit jagged in screencasts taken on screens using fractional scaling. (Anton Lisovskiy, KDE Bugzilla #528423)
Fixed an issue that made the Color Picker widget sometimes draw a contrast circle around saved colors at inappropriate times. (Christoph Wolk, kdeplasma-addons MR #1253)
Plasma 6.7.7
Fixed an issue in the Color Picker widget that prevented middle-clicking on a preview from copying the color to the clipboard. (Christoph Wolk, kdeplasma-addons MR #1254)
Plasma 6.8.3
Fixed a case that could make the screen locker crash (without unlocking the system, so there was no security issue) in certain cases when a screen was disconnected while the system was sleeping. (Drew, KDE Bugzilla #528438)
Fixed a case where System Settings could crash when the Wallpaper page was open and a screen was disconnected. (Nikolai Nyegaard, KDE Bugzilla #528380)
Fixed a recent regression that caused visual corruption on certain systems with Intel CPUs and NVIDIA GPUs. (Xaver Hugl, KDE Bugzilla #527163)
Fixed a recent regression that made the systemwide color picker fail to include a picked color’s alpha channel. (Jean-Baptiste Mardelle, KDE Bugzilla #528396)
Fixed a visual glitch on System Settings’ Color Scheme page that made some previews visually overflow. (Akseli Lahtinen, KDE Bugzilla #528106)
Fixed a subtle issue that caused inappropriate default scale factors to be chosen for some laptops and handheld gaming consoles. (Xaver Hugl, kwin MR #10327)
Global Themes once again correctly apply non-default font settings. (Alexander Berg, plasma-workspace MR #7340)
Plasma 6.8.4
Hardened KWin against crashing when a game like Elden Ring sent it invalid HDR data. (Xaver Hugl, KDE Bugzilla #528428)
Fixed a case where Discover could crash when you asked it to trash the orphaned data of an uninstalled Flatpak app whose runtime had also been uninstalled. (David Edmundson, KDE Bugzilla #528572)
Fixed a somewhat common way that Discover could crash on operating systems using its RPM-OSTree backend, including Fedora Kinoite and Bazzite. (Aleix Pol Gonzalez, discover MR #1445)
Fixed an issue that would make Plasma freeze if you put the Window List widget into a vertical panel wider than 144 pixels. (Christoph Wolk, KDE Bugzilla #527314)
Fixed a regression that could make certain fullscreen windows visually freeze when screen tearing was allowed and atomic modesetting was disabled. (Xaver Hugl, KDE Bugzilla #528579)
Fixed a regression that made it impossible to drag files and folders on the desktop across screens if their filenames contained URLs. (Akseli Lahtinen, plasma-desktop MR #4205)
Fixed a regression that made the “Add Widgets” sidebar sometimes open on the wrong screen of a multi-screen setup. (Niccolò Venerandi, KDE Bugzilla #528201)
Copying text on Info Center’s “About” page that you selected with the pointer now copies just that text, and the little message that tells you about it is now correct, too. (Tobias Fella, kinfocenter MR #324)
Plasma 6.9
The Plasma Browser Integration plugin no longer breaks the ability to share your screen using Microsoft Teams. (Kanishk Ranjan, KDE Bugzilla #507991)
Using an Alternate Calendar plugin in the Digital Clock widget no longer forces its events panel to appear when it would otherwise remain disabled. (Tobias Fella, KDE Bugzilla #497122)
Notable in performance & technical
Plasma 6.9
Spectacle is now even faster to launch, thanks to further optimizations and parallelizations. (Kai Uwe Broulik, spectacle MR #625 and spectacle MR #626)
Window thumbnails in the Task Manager widget no longer wastefully regenerate themselves when the pointer leaves them and returns, avoiding some unnecessary work and fixing a slight visual flicker. (Christoph Wolk, libplasma MR #1629)
If you’re using Plasma Login Manager as your login manager/greeter and it breaks for whatever reason, now it includes a simple fallback that at least gets you to a terminal, rather than stranding you at a black screen. (Harald Sitter, plasma-login-manager MR #232)
How you can help
KDE has become important in the world, and your time and contributions have helped us get there. As we grow, we need your support to keep KDE sustainable.
Donate to KDE’s current fundraiser! We’ve raised 42% of the funds we need to be in a better position for next year, so every little bit helps!
Beyond that, you can help KDE by directly getting involved in any other projects. Donating time is actually more impactful than donating money. Each contributor makes a huge difference in KDE — you are not a number or a cog in a machine! You don’t have to be a programmer, either; many other opportunities exist.
You can also help out by making a donation! This helps cover operational costs, salaries, travel expenses for contributors, and in general just keeps KDE bringing Free Software to the world.
To get a new Plasma feature or a bug fix mentioned here
Any Sane Person in Computers Right Now: “Are We the Baddies?”
Tags: tech, politics, history
Or the importance of community and studying other fields. The political landscape around tech has led to a turbocharged dystopia but there are other ways. Any bubble of humanism in the margins is precious and has to be kept. It’ll be needed someday.
Interesting essay. It’s very much attuned to the US context but it’s a timely exploration of the current handling of innovation in our societies. Interestingly we’ve forgot that innovation and progress don’t necessarily go hand in hand. This isn’t without bad consequences.
It’s a good illustration of how information channels became very fragile. LLMs based systems reaching for openly available information on the web are very susceptible to being manipulated.
If you ever wonder about what will happen in your body in the end. This gives a good idea of the process and it’s longer than what we usually have in mind.
Kwi has continued to mature at a fast clip over the past week, with many a bug slain and a significant amount of UI refinement.
Exciting news also from our important upstream, the Ghostty project: Earlier than we had anticipated, libghostty-vt was released in version v1.0 on the weekend. This provides Kwi with a stable API/ABI from our headless terminal emulation backend, which will greatly aid distros and packaging efforts in particular.
For the Ghostty project this is the culmination of a solid half-decade of work - congratulations to mitchellh and many other contributors over there!
Sharper, better rendering of geometric symbols at fractional scales
Improved grid size calculations at fractional scales
Improved font baseline placement
Additional performance improvements in text rendering
Configured SSH aliases/hostnames are now used in tab titles instead of resolved IPs; this was filed as a regression defect against Konsole we fixed in Kwi (bug #524673)
Fixed a greyscale glyph placement bug that caused e.g. Noto Sans Mono's i character to act up when LCD filtering was disabled in the text renderer (bug #528550)
Tab bar:
New output causes background tabs to get colorful title text to communicate activity, à la Konsole (feature request #528658)
Dragging files, URLs and similar over tabs now raises them after a brief delay, to make it easy to drop those contents into the tab (feature request #15)
On popular request, the tab bar is now overall significantly more compact (on Matrix and reddit, and in other places)
Tab button contents are laid out more pleasingly
Ditto for tool buttons on the tab bar
As a small cleanup, the typically redundant default tab icon is no longer shown; special status icons remain (by popular request on Matrix)
Icons that are line art (e.g. many symbolic Breeze icons) are now colorized to match the tab title text (which goes nicely with the above activity coloring)
If the window is fullscreen, care is taken that tab buttons are proper click targets at the screen edge (c.f. good old Fitts' law) (bug #528546)
Default tab switching shortcuts now match Konsole's (both primary and alternate; requested on Matrix)
The bottom divider below the tab bar now looks better and more consistent with other KDE apps at fractional scale
A window's terminal grid size is now preserved correctly when the window is moved between screens with different fractional scales (bug #528515)
Other UI:
Kwi now supports the "Find Action" dialog that many other KDE apps offer (similar to a Command Bar in other apps), available via the standard Ctrl+Alt+I shortcut (requested on reddit)
The background opacity slider now uses a less granular tick step to make it look less busy, and space for its label is reserved so the slider doesn't change width during use
All windows now share a single QML engine, slightly improving resource usage
Portability:
Sandboxed Kwi (Flatpak) now works on certain NixOS, and potentially other, systems with unusual PATH/filesystem setups (bug #528754)
Kwi now works on BusyBox-based systems (by using the portable timeout -s instead of timeout --signal=; reported on Matrix)
Dependencies:
Bumped our libghostty-vt dependency first to v1.0, then to v1.0.1
Project bits:
Updated the example screenshot for Kwi's KDE apps listing (note: the CDN updates these only very slowly)
Additionally, because Kwi is built on KDE's latest technologies, we sometimes get to find and fix bugs upstream of the project, or contribute new features needed by Kwi.
Upstream This Week - by Kwi, for everyone:
We fixed a bug in KDE's KDBusAddons library so that running Kwi from a terminal will open the new window in the terminal's current working directory, even if Kwi is already running (MR #75, bug #528661).
We fixed a bug in KDE's Kirigami UI library that made it impossible to configure Alt+Arrows as shortcuts in Kwi (MR #2234, bug #528659).
We worked on components in KDE's Kirigami Application Components UI Library to allow the "Find Action" action dialog to work in apps which can open multiple windows, along with enabling some code cleanups/reductions in Kwi (MR #27).
Further in Kirigami Application Components, we fixed a bug that made it hard to search for certain actions using the search field in the "Find Action" dialog (MR #26).
One more in Kirigami Application Components, we have a contribution remaining in flight that will fix invoking actions in the dialog using the Enter/Return keys (MR #29).
In KDE's Breeze UI theme we fixed the rendering of slider tickmarks at fractional scale (MR #649), as seen in Kwi's Settings dialog.
Update on Release
As mentioned last week, Kwi continues to undergo the "KDE Review" process at the moment, a community quality gate through which it will graduate to become an official KDE project. While we have already completed the required checklist, the minimum mandatory duration is two weeks. Kwi will complete this on November 18th.
Shortly thereafter we plan to tag a v0.1 release and publish to Flathub!
Outlook
Also mentioned last week, we're working on a way to make it easy to switch classic KDE apps such as Yakuake and Dolphin to embed Kwi.
While the Kwi-side bits have been in place since its initial publication, this is mainly about providing a central setting to make this easier for users (currently it requires fiddling in the filesystem). We now have a prototype implementation, and plan to submit a merge request to the relevant KDE project (roughly in the dependency stack of System Settings) soon.
Now the bad news: Expect Kwi development to move a bit slower in coming weeks, and focus mainly on important bug fixes. There may even be a brief interruption in this newsletter! The reason is brief interruption by work on other projects related to things nearly as old as UNIX terminals.
Once we are back to full capacity, we plan to begin work in earnest on the implementation of the much-requested Split Panes.
KDE e.V., the non-profit organisation supporting the KDE community, is
looking to hire a software engineer to help improve the software stack that KDE software relies on.
Please see the call for proposals
for more details about this contract opportunity.
We are looking forward to your application.
This report came from Serg Star who spotted it reviewing this code. It's great that we have people spotting this stuff, though it may be too late for some.
The Malware
The package looks very legit, its an animating wallpaper that works as advertised, they even did later commits on top adding configuration options!
However since the initial commit the code has been shipping hidden malware.
A script (hidden in a base64 string in a base64 string) makes a web request to download an "image", which is another bash script which in turn downloads a libary .so file into ".config/Plasma6Library.so" and launches it via LD_PRELOAD=~/.config/Plasma6Library.so sleep 999999 The payload thus far looks like this: https://github.com/BishopFox/sliver which is designed to allow remote control.
If you have installed this wallpaper, a good test would be checking for .config/Plasma6Library.so and I would suggest starting anew.
What Happens Now
There is only so much we can do with 3rd party content. We have reported it to all relevant places including Pling and OpenDesktop who have taken swift action within hours of us getting the report.
It it a stark reminder to be vigilant and wary when installing external content from other sources especially those that run in unsandboxed contents, and a reminder to those of us working on the Linux desktop to continue working on technologies that allow for safer and better distribution.
New release for plank-player, aura-browser and bigscreen-ui
Wow, new releases! We are releasing applications and a new UI framework this month.
Bigscreen UI is a library of shared Qt Quick components to build applications for TV, built on Kirigami.
Plank Player and Aura Browser were originally released as part of the Plasma 5 release schedule. We have updated them to use Bigscreen UI, and released them independently for now. The eventual goal is to move them to the KDE Gear release schedule.
bigscreen-ui is a new library of UI components to build applications for Plasma Bigscreen, with a focus on making it easy to implement arrow navigation for remote control. The components were extracted out of the ones used in the Plasma Bigscreen shell, now decoupled for application development purposes.
Learn more about it on GitLab, and see the example application. We plan on introducing official documentation as the API stabilizes.
Aura Browser
Aura Browser is a web browser for Plasma Bigscreen, with a focus on making it easy to navigate pages with a remote.
Plank Player
Plank Player is a media player for Plasma Bigscreen. Users can either browse media via the homescreen, or invoke the application directly on the media. It is built on Qt Multimedia, which brings support for other features such as subtitles and audio tracks.
One of the big things I wanted to achieve during this HDR project is to have a decent tone and gamut map filter, something Krita never had all these years, in spite of us having unbounded floating point editing as well as color management via LCMS that works with this unbounded floating point.
The unbounded floating point allows us to have values that might be (0.0, 0.5, 1.0) in linear rec2020, and then when converted to linear sRGB this becomes (-0.37, 0.56, 1.0). Now, the range of each channel here is [0 1], so if we try to represent that as an inbounds sRGB colour (which we need to do if we want to save for web), the default behaviour is to clip the out-of-bounds values, so we get (0.0, 0.56, 1.0). This changes the proportion between the blue and the red and green channels, meaning the hue and saturation will shift. In some cases, it can cause colours to gradually shift to values like (0.0, 1.0, 1.0). This is, for example, why your phone camera won’t shoot that lovely orange sunset just right: The colors are too colourful and get pushed towards yellow due clipping. So to combat this, you do gamut mapping: you shift the colour in bounds by a certain algorithm before or after doing the conversion.
Gamut Mapping
The most common gamut mapping algorithm is chroma-reduction. This assumes that the target space and the source space have their greys at the same location (taking into account white point adaptation). Then, the colors are converted to a third space, a opponent color space that also has its greys in the same location.
Opponent color spaces are spaces where the luminosity are separate from the chromacity and hue, and the chromacity and hue themselves are expressed as a red-green and a yellow-blue contrast, following the opponent theory of human color vision. A typical example of an opponent color space is LAB, but there’s many others, like YCbCr, ICtCp and OkLAB. These opponent colorspaces can then be translated to a Hue, Chroma and Luminosity space by treating the red-green and blue-yellow contrasts as Carthesian coordinates, and converting those to polar coordinates, giving us hue as the angle, and chroma as the length.
Then, for chroma reduction, you do a binary search to find the edge of the gamut, with each iteration, increasing the chroma or decreasing it, until the difference between the desaturated color, and the clipped version of the color is below a significant difference. That significant difference is called the minimum delta E, E being the name given to color difference by itself.
Shown above: a slice of LAB chromacity plane, with a black-to-white gradient diagonally. The first image is plain clipping to sRGB, while the second uses chroma reduction gamut mapping. In the first image, the chroma of the colors bottomleft and topright are so bright that when clipping they end up way with the wrong luminosity than they should have in reality. By doing chroma reduction, we pull these colours so far back in that their luminosity is preserved.
This is because the delta E calculation for LAB is a little off, in particular, LAB is known to give weird results with the blue, with bright blues looking a little purple in LAB. ICtCp and OkLAB are much more modern and are particularly designed to have much better handling of the blue hues, which is visible in the gamut mapping:
ICtCp and OkLAB also had the advantage of being possible to implement in an ad-hoc manner starting from XYZ, so I didn’t need to integrate them directly into our color management system yet.
Besides binary search, there’s the cusp method. Here, you figure out the luminosity and maximum chroma of a given hue (the cusp), and use that to approximate the gamut boundary. There’s two ways to calculate the cusp:
The first is to use 6 polynomials, but that method only works with OkLAB as the opponent space and target RGB spaces that are matrix shapers. This in turns had to do with the simplicity of OkLAB as a colour space compared to other opponent spaces.
The second is to create a look up table of cusps per hue. Here, you take the known cusps in the target colour space and convert them to the opponent space to get their chroma and luminosity. This works for both CMYK and RGB spaces, as we know the likely cusps for those: Red, Yellow, Green, Cyan, Blue and Magenta and the interpolations between those. It also works with LAB, ICtCp and OkLAB equally. I went with this second method for now, but the first method is going to be more precise, and is worth investigating.
With the cusp for a given hue, you can then draw a triangle, going from black, to the cusp, to the reference white of the target space (important to keep in mind when combined with ICtCp-pq!), and back to black. The maximum chroma for a given lightness can then be derived by calculating the intersection between the current chroma and luminosity, and the zero chroma (grey) version of said luminosity.
Both the CSS4 gamut edge seeker algorithm and Ottoson’s algorithm use a curved top (cusp to reference white) section for the triangle here, but I am skeptical that this is strictly necessary. The colour only needs to be pushed into gamut enough so that once you clip it doesn’t cause massive hue and luminosity changes.
The interesting thing about the cusp method is that with the location of the cusp, you can decide to not just intersect with the grey of a given luminosity, but instead draw the line slightly towards the luminosity of the cusp. Bjorn Ottoson has a blogpost with experiments demonstrating this, and while not right now implemented in Krita, I would like to give it a go in the future as it can give more chromatic results.
There’s other methods. Scaling (“chroma compression”) methods, for example, generate a histogram of chromacity and lightness at a given hue-angle in the input image, and then the same for the target gamut. The resulting algorithm then doesn’t need to do any binary search, but can just scale the chromacity of the input by taking the maximum source chroma and scaling it by the maximum target chroma. This is also not yet implemented in Krita, but it might be interesting for CMYK output. Technically this is what ICC’s perceptual and saturation intents are all about, but those suffer from being too generic: a desaturated image that is fully in-gamut for the target space will get treated with the same lookup table as a fully saturated image, leading to odd results. A image-specific filter might give much better results.
Finally, when the target space is greyscale, we don’t bother with either method and just set the saturation to 0.
Tone Mapping
Tone mapping is a subset of gamut mapping, where we only try to get the full range of tones (luminosity) into a target range. This is used to prepare HDR images for SDR output.
This is typically done by applying a so-called “filmic” curve. The name “filmic” derives from these curves mimicking real world chemical film’s tone properties, which when mapped onto a graph somewhat resembles a flattened “S”. The top part of this S-shape is referred to as the “shoulder” and the bottom part the “knee”, or, because it is quite tiny compared to the “shoulder”, the “toe”.
In the following section I’ll be using a picture of a blue orb on a grey background with some sparkles. I drew the original image on an HDR display, and it’s brightest highlights are around 8.0 (8 times higher than the reference white). Because most people don’t own a HDR display, I’m not going to try and share the image. Instead, I’m sharing the above image with two different exposures: The first is 0 exposure, it just clips all the colors. The second is with -2 exposure, showing how the highlights are supposed to look, but now the shadows are too dark. What we are looking for in the following images is that the sparkles are distinct from the highlight on the orb. That the blues look nice and don’t hue shift too much and that the shadows are not too light or too dark.
When looking for curves to implement, I wanted to have curves that were user configurable. So the first curve I implemented was the extended Reinhard curve. This is a pretty well known solution and the curve allows selecting the maximum white that gets mapped to 1.0. While it has a nice shoulder, it doesn’t have a knee or toe of any kind, which means it loses the contrast that is present in the HDR image.
I then briefly looked at the Hermetic Filmic Curve as described by bt 2390-8, which has both a shoulder and knee. However, that one is applied in the PQ transfer instead of linear light, and the PQ transfer requires decisions about the brightness of the reference white, so I wasn’t sure how to do that one properly.
Then, I came across John Hable’s Piecewise Filmic Curve: A very configurable curve which has various parameters for editing the shoulder and toe. It takes a bit to configure, but as you can see, once configured, it preserves the contrast quite well:
I implemented both curves on top of LAB, and xyY. This means that they’re applied to the luminance channel only.
Frequently, you will see the discussion on whether to apply the tone mapping to individual channels, or to apply it to a Luminance channel only. The base argument for using individual channels is that it’ll give chroma reduction as the colour will get lighter. The argument for applying it on a luminance channel is that it is more radiometrically correct. Though, it is correct that the bright colors now become more chromatic.
This is because to get the luminance channel, the colors need to be converted to a opponent space (typically, xyY, for easy conversion). But, when dealing with ‘hdr’ colours, like (3.0, 2.0, 0), it is not just the brightness that is high, but also the chroma. When tone mapping in an opponent space, the chroma is untouched. This leads to overly chromatic colours near the bright highlights, and these chromatic colours differ per opponent color model:
However, we already have a solution to reduce chroma: chroma reduction gamut mapping. So if you apply that after applying the tone mapping, the chromatic highlights get fixed:
Is this better than per-channel tone mapping? I think it depends on the application. Gamut mapping can be quite slow, and isn’t exactly shader friendly, so I can imagine why real-time applications (game engines), would prefer per-channel tone mapping (which more or less does its own form of gamut mapping). Meanwhile, the results of luminance channel tone mapping plus gamut mapping afterwards gives much more chromatic results, which might be more interesting for illustrative works.
Similarly, per-channel mapping only works in a RGB context, and Krita has more models than that to support. By doing our tone mapping in an opponent space, we get the same results regardless of what colour space the image is in, which should make it nice and predictable to work with.
This is one of the reasons that Krita’s tone and gamut mapping filter share the same dialog (they can still be applied separately). The other reason is that we can reduce the amount of colour space conversions if we clump together the two transforms. Furthermore, experience tells me that illustrative artists tend to prefer a single filter here.
OCIO Lut baking
While I was pretty happy with the tone and gamut mapping filter, we were still missing something:
OCIO is the colour management library used by the vfx industry, and it’s strength lies in a big portable configuration that can be much more easily designed that a library of icc profiles. Due this, a well made OCIO configuration often has its own tone mapping solutions, and it would be good if Krita could use these too.
The first thing to decide is what we’re exactly baking. OCIO can be quite flexible, but the way it is implemented in Krita is that we have the standard pipeline provided by OCIO, where we…
start off in the assigned color space
go to the config’s preferred linear color space
Apply the exposure.
Apply the looks.
Go to the display color space, apply the view transform.
Apply the channel swizzle.
Apply the gamma.
This made things a little tricky, as it is impossible to go from the display colour space back to the assigned image colour space. Furthermore, ideally we’d apply all transforms, including the view transform, so that we get an image that looks exactly like the one you see.
Initially I thought of reducing the scope, and creating a baking transform that applies everything in the config’s preferred linear space, except the view transform. This didn’t work out, as the gamma transform being applied linearly was too unintuitive.
So then, the next attempt was to apply the whole pipeline. This in turn means that the result is in the display color space. This too has some issues, as it requires the display space to be the exact same as the one selected in the preferences. Now, this is required anyway on wayland, as we have no way to determine from a given ocio display what we need to tag the wayland output as. But on the flip side, the default OCIO PQ transform is tied to 100 nits, and we do not yet have the ability to configure the nits level of Krita’s rec2100 PQ surfaces, so the look will remain a little off when using HDR output until we do have that kind of support.
For this reason though, we ask whether you want to convert from the display colour space to the image colour space, or keep the display colour space. That way, the baked result can be quickly saved as an untagged image, like png, and then wrangled by some external script.
Above: Image baked with blender’s current ocio config, using the sRGB display device and the ‘aces 2.0’ view transform.
We’ve had this “Baking” of the config on our wishlist since OCIO was first integrated, so I’m pretty pleased we managed to get it in, even though it still has some papercuts.
Further Work
So, right now, Krita has a built-in tone and gamut mapping filter, which works on luminance, and OCIO lut baking, where the tone mapping generally works per channel. Overall, the OCIO lut baking is probably going to be best for film workflows, while the built-in filter is going to be useful for non-ocio workflows, or when the artist prefers the more chromatic results of the built-in filter.
Because the colourfulness is going to be the key quality of the built-in filter, it’ll be interesting to look at the other gamut mapping options, especially the adjustment-towards-the-cusp.
For CMYK workflows, the gamut mapping filter isn’t too useful right now, as most artists work in sRGB, and the clipping errors towards CMYK there are very rare. Chroma compression might be interesting to experiment with here, but that requires generating a histogram of the chroma over hue, and I’ll need to figure out how to do that while keeping the UI performant.
The built-in filter also doesn’t have adjustment layers enabled just yet. This is because once we allow it as an adjustment layer, we’ll need to always support it. And because there’s a ton of tone-mapping curves out there, and I want to have a period of experimentation before we commit to a particular set.
In the future, I might need to seriously look at splitting up the gamut and tone mapping filters. Benefits here are for example allowing per-channel filter anyway (which is now impossible because the filter tries to find a space that can both gamut and tone map). From further experimentation, I get the feeling that the desaturated results I got tie in strongly with the configuration of the curve, so with a user configurable curve it might get better results. Other things to consider is to allow other models for tone mapping besides LAB and xyY, as that leads to different chromatic colors being applied. At the same time, I don’t want to make this initial filter too complicated.
There’s also some image-aware tone mapping techniques out there. I haven’t investigated them enough though, so I have no idea if they’re useful for Krita’s purposes.
As for which of these is the correct one? That depends on the artist. Tone and gamut mapping are generally considered to be artistic operations, because there’s always trade-offs and only the artist can tell which trade-offs and qualities are acceptable. Hence why I am trying to find a good set of tools for artists to work with.
Appendix
Manually tone mapping.
So, you can sort of manually tone map in Krita, because it supports unbounded workflows. It is a little annoying however, with some pitfalls.
Firstly, you will want to take an HDR image, and scale it’s range into a [0 1] range. You do this by first color picking or using the HDR metadata tab to find the brightest pixel.
Then, you go into the select-a-color dialog by clicking the dual color button at the top. There, you type in the brightest channel value into each channel input. With your HDR color in hand, create a new layer, and fill it with this color. Then set the layer’s blending mode to “divide”. Now the whole image is in a [0 1] range. Then, you can either use the color adjustment filter or the levels filter to make everything visible again, but you will need to use the RGB adjustment, and not the lightness one, as the later goes through lab, and somehow that means everything gets desaturated.
This generally desaturates the whole image a little, as it is a form of per-channel tone mapping, so I personally pump up the saturation after my levels corrections.
The other pitfall here is that it becomes really hard to get a good curve the higher the brightest white is, because the curve needs to be fine-tuned more and more to both get nice shadows and nice highlights at the same time. The tone mapping filter is easier to use, in this case.
Tagoross released AI Openness: How The Terms Relate, package 3 of 5. Its document and diagram show how AI openness terms differ, conflict and build on each other. It helps non-experts evaluate vendor lock-in and sovereignty risks of AI models.
@vkrause:kde.org
ngraham
