Let’s go for my web review for the week 2023-03.

AI’s Jurassic Park moment - by Gary Marcus

Tags: tech, ai, gpt, ethics

Very good piece about that dangerous moment in the creation of the latest large language models. We’re about to drown in misinformation, can we get out of it?

https://garymarcus.substack.com/p/ais-jurassic-park-moment

OpenAI Used Kenyan Workers on Less Than $2 Per Hour: Exclusive | Time

Tags: tech, ethics, ai, machine-learning, gpt

The human labor behind AI training is still on going. This is clearly gruesome and sent over to other countries… ignoring the price for a minute this is also a good way to hide its consequences I guess.

https://time.com/6247678/openai-chatgpt-kenya-workers/

U.S. No Fly List Left on Unprotected Airline Server

Tags: tech, airline, security

That’s an “interesting” leak, both for how it happens and what it contains. I shows serious biases in the “no fly list” used by airlines.

https://www.dailydot.com/debug/no-fly-list-us-tsa-unprotected-server-commuteair/

New T-Mobile Breach Affects 37 Million Accounts – Krebs on Security

Tags: tech, security

That’s really a massive leak again! The amount of personal data in the wild… will likely help with identity theft too.

https://krebsonsecurity.com/2023/01/new-t-mobile-breach-affects-37-million-accounts/

Jack’s Blog - Revisiting KDE

Tags: tech, kde

Nice to see kind words out there. A couple of issues are pointed out of course but nothing which seems critical I think.

https://jackevansevo.github.io/revisiting-kde.html

How to improve Python packaging, or why fourteen tools are at least twelve too many | Chris Warrick

Tags: tech, python

Yes… python packaging is a mess. I wonder when it’ll get properly unified and get a proper single tool and workflow.

https://chriswarrick.com/blog/2023/01/15/how-to-improve-python-packaging/#summary

The Elusive Frame Timing | by Alen Ladavac | Medium

Tags: tech, 3d, performance

Excellent analysis and explanation of the stutter problem people experience with game engines. It’s an artifact of the graphics pipeline becoming more asynchronous with no way to know when something is really displayed. Extra graphics APIs will be needed to solve this for real.

https://medium.com/@alen.ladavac/the-elusive-frame-timing-168f899aec92

Use.GPU Goes Trad — Acko.net

Tags: tech, web, webgpu, 3d, frontend

This WebGPU framework is getting interesting. Definitely something to keep an eye on and evaluate for productive uses. Obviously requires WebGPU to be widely available before banking on it.

https://acko.net/blog/use-gpu-goes-trad/

Examples of floating point problems

Tags: tech, programming, mathematics

Nice set of problems encountered when using floating point numbers. Definitely to keep in mind.

https://jvns.ca/blog/2023/01/13/examples-of-floating-point-problems/

Examples of problems with integers

Tags: tech, programming, mathematics

Problems with integers now. Kind of better known usually, still to keep in mind as well.

https://jvns.ca/blog/2023/01/18/examples-of-problems-with-integers/

We invested 10% to pay back tech debt; Here’s what happened

Tags: tech, programming, technical-debt

Excellent piece about technical debt. The approach proposed is definitely the good one, it’s the only thing which I know to work to keep technical debt at bay.

https://blog.alexewerlof.com/p/tech-debt-day

From Story Points to Slam Dunks — Planning for Success

Tags: tech, project-management, estimates, kanban

Interesting approach regarding estimates. Might especially make sense combined with kanban like project management.

https://medium.com/agileinsider/from-story-points-to-slam-dunks-planning-for-success-c3e067354d9b

The CARL framework of reflection | The University of Edinburgh

Tags: tech, project-management, hr, management, interviews

It’s coming from the job interview domain… but I wonder if it could be more largely useful due to how simple it is (but not easy mind you). I guess I’ll experiment with it for my next project postmortem.

https://www.ed.ac.uk/reflection/reflectors-toolkit/reflecting-on-experience/carl

Writing Is Magic - Marc’s Blog

Tags: tech, communication, writing

A reminder for me, I write quite a bit, but I feel that I don’t write nearly enough. It’s very important for plenty of cases though.

https://brooker.co.za/blog/2022/11/08/writing.html

These Gorgeous Photos Capture Life Inside a Drop of Seawater | Science | Smithsonian Magazine

Tags: photography, science, biology

Really amazing pictures! All this life we usually can’t see with our own eye.

https://www.smithsonianmag.com/science-nature/these-gorgeous-photos-capture-life-inside-drop-seawater-180981297/

Bye for now!

Carl Schwan already announced it on discuss.kde.org and at @neochat@fosstodon.org.

In this blog we’ll see how it was done and how you can publish your KDE app in the Microsoft Store.

Reserving a Name and Age Rating Your App

The first step requires some manual work. In Microsoft Partner Center you need to create a new app by reserving a name and complete a first submission. How to do this has been described by Christoph Cullmann in the Windows Store Submission Guide. Don’t hesitate to reserve the name of your app even if you are not yet ready for the first submission to the Microsoft Store. Once a name is reserved nobody else can publish an app with this name.

The first submission needs to be done manually because you will have to answer the age ratings questionnaire. NeoChat was rated 18+ because it allows you to publish all kinds of offensive content on public Matrix rooms. Filling out the questionnaire was quite amusing because I did it together with the NeoChat crowd at #neochat:kde.org.

On the first submission of NeoChat I chose to restrict the visibility to Private audience until it was ready for public consumption. I created a new customer group NeoChat Beta Testers with the email address of my regular Microsoft Store account in Microsoft Partner Center and then selected this group under Private audience. This way I could test installing NeoChat with the Microsoft Store app before anybody else could see it.

Don’t spend too much time filling out things like Description, Screenshots, etc. under Store Listings because some of this information will be added automatically from the AppStream data of your app for all available translations.

Semi-automatic App Submissions

The next submissions of NeoChat were done semi-automatically via the Microsoft Submission API with the submit-to-microsoft-store.py script while writing this Python script and the underlying general Microsoft Store API Python module microstore. The script is based on a Ruby prototype (windows.rb) written by Harald Sitter.

The idea is that the script is run by a (manual) CI job that the app’s release manager can trigger if they want to publish a new version on the Microsoft Store.

To run the script locally you need the credentials for an Azure AD application associated with KDE’s Partner Center account. Anything else you need to know is documented in the script’s README.md.

Making NeoChat Publically Available

The last step of the process to get NeoChat published in the Microsoft Store was another manual submission which just changed the visibility to Private audience. This could also have been done via the Microsoft Submission API (but not with the current version of the script), but I think it’s good to have a last look at the information about the app before it is published. In particular, you may have to fill out the Notes for certification, e.g. if your app cannot be tested without a service or social media account. For NeoChat we had to provide a test account for Matrix.

Moreover, you may want to fill out some details that are currently not available in the AppStream data, e.g. a list of Product features, the Copyright and trademark info, or special screenshots of the Windows version of your app.

What’s Next

On our GitLab instance, we want to provide a full CI/CD pipeline for building and publishing our KDE apps on the Microsoft Store (and many other app stores). A few important things that require special credentials or signing certificates are still missing to complete this pipeline.

And we want to get more KDE apps into the Microsoft Store.

If you need help with getting your KDE app into the Microsoft Store, then come find me in the #kde-windows room.

Updates after publication:

• 2023-01-31: Updated link to script after merge of the MR

New year, new RISC-V Yocto blog post \o/ When I wrote my last post, I did really not expect my brand new VisionFive-2 board to find its way to me so soon… But well, a week ago it was suddenly there. While unpacking I shortly pondered over my made plans to prepare a Plasma Bigscreen RaspberryPi 4 demo board for this year’s FOSDEM.

Obvious conclusion: “Screw it! Let’s do the demo on the VisionFive-2!” — And there we are:

After some initial bumpy steps to boot up a first self-compiled U-boot and Kernel (If you unbox a new board, you need to do a bootloader and firmware update first! Otherwise it will not boot the latest VisionFive Kernel) it was surprisingly easy to prepare Yocto to build a core-image-minimal that really boots the whole way up.

Unfortunately after these first happy hours, the last week was full of handling the horrors of closed-source binary drivers for the GPU. Even though Imagination promised to provide an open source driver at some time, right now there is only the solution to use the closed source PVR driver. After quite a lot of trying, guessing and and comparing the boot and init sequences of the reference image to the dark screen in front of me, I came up with:

• a new visionfive2-graphics Yocto package for the closed source driver blobs
• a fork of Mesa that uses a very heavy patch set for the PVR driver adaptions; all patches are taken from the VisionFive 2 buildroot configurations
• and a couple of configs for making the system start with doing an initial modeset

The result right now:

VisionFive-2 device with Plasma-Bigscreen (KWin running via Wayland), SD card image built via Yocto, KDE software via KDE’s Yocto layers, Kernel and U-Boot being the latest fork versions from StarFive

Actually, the full UI even feels much smoother than on my RPi4, which is quite cool. I am not sure where I will end in about 3 weeks with some more debugging and patching. But I am very confident that you can see a working RISC-V board with onboard GPU and running Plasma Shell, when you visit the KDE stall at FOSDEM in February

For people who are interested in Yocto, here is the WIP patch set: https://github.com/riscv/meta-riscv/pull/382

Let’s go for my web review for the week 2023-02.

A community isn’t a garden, it’s a bar.

Tags: tech, social-media, fediverse, culture

A very interesting metaphor. Indeed on social media we’re not dealing with gardens.

https://powazek.com/posts/3571

Study Finds That Buttons in Cars Are Safer and Quicker to Use Than Touchscreens

Tags: tech, automotive, ux

Not very scientific, but indeed thought provoking and taps into safety considerations.

https://futurism.com/the-byte/study-finds-that-buttons-in-cars-are-safer-and-quicker-to-use-than-touchscreens

I scanned every package on PyPi and found 57 live AWS keys

Tags: tech, security, secrets, python

This is apparently a somewhat common mistake. Something is apparently not easy enough to handle and error prone.

https://tomforb.es/i-scanned-every-package-on-pypi-and-found-57-live-aws-keys/

The yaml document from hell

Tags: tech, yaml

A few examples of why yaml is getting out of control. It is very very error prone at this point.

https://ruudvanasseldonk.com/2023/01/11/the-yaml-document-from-hell

How to store your app’s entire state in the url

Tags: tech, web, browser

Interesting approach although you probably don’t want to do this systematically. For some applications it is a good idea.

https://www.scottantipa.com/store-app-state-in-urls

Introduction - Just Programmer’s Manual

Tags: tech, tools, command-line

Interesting little tool. I usually use make for this kind of things, but it seems to bring some benefits for non build tasks.

https://just.systems/man/en/

Makefiles for Web Work – Ross Zurowski

Tags: tech, command-line, tools, unix

A love letter to Makefiles. A couple of interesting tricks in there.

https://rosszurowski.com/log/2022/makefiles

What Happens When A CPU Starts

Tags: tech, cpu

Nice nugget reminding us the early steps and basic mechanisms of the CPU life at boot.

https://lateblt.tripod.com/bit68.txt

The Power of Prolog

Tags: tech, programming, logic, prolog

Nice resource to get started with Prolog.

https://www.metalevel.at/prolog

Architecture diagrams should be code - BAM Weblog

Tags: tech, architecture, diagrams

I’m more and more tempted by this kind of approach. Managing architecture models using code seems fairly neat. That said I wish we’d have better free software tooling for that, I find it still fairly limited. Maybe I should check out the Haskell library which is mentioned.

https://brianmckenna.org/blog/architecture_code

Shopify: Say no to meetings | UNLEASH

Tags: tech, management, meetings

Always a good idea to seek reduction in time spent in meetings. I’ve seen this being too often a drain. Can get quickly out of control.

https://www.unleash.ai/hr-technology/shopify-encourages-employees-to-say-no-to-meetings/

Epochalypse

Tags: tech, unix, date, funny

It’s comiiing! OK… not quite yet. But if that prevents your sleep here is an easy way to check.

https://www.epochalypse.today/

The science of why you have great ideas in the shower

Tags: science

Finally, we have an idea of why we get ideas in “strange” moments. Fascinating stuff.

https://www.nationalgeographic.co.uk/history-and-civilisation/2022/08/the-science-of-why-you-have-great-ideas-in-the-shower

Bye for now!

I am pleased to announce Linux-Stopmotion release 0.8.6! The last release was three years ago and this is the first release since Stopmotion became a KDE incubator project.

About Stopmotion

Stopmotion is a Free Open Source application to create stop-motion animations. It helps you capture and edit the frames of your animation and export them as a single file.

Technically, it is a C++ / Qt application with optional dependencies to camera capture libraries.

Changes in release 0.8.6

• New build system using CMake. The qmake one is deprecated and will be removed.
• The test executable can be executed as a CMake test target (make test-stopmotion && make test).
• Fixed various warnings from Clang, GCC, and Qt 5.15.
• We have a build pipeline executing automated builds and tests.

Future plans

• We decided to renamed the application to KStopmotion, as Linux is trademarked.
• Transition from Qt 5 to version 6.
• We should integrate better to KDE's tech stack: Internationalization, using KDE libraries, update and reformat documentation.
  

Get involved!

If you are interested, give Stopmotion a try. Reach out to our mailing list kstopmotion@kde.org to share ideas or get involved.

You can also help to improve Stopmotion. For example, we started the transition to Qt 6 and we welcome any helping hand.

Happy new year! Let’s go for my web review for the week 2023-01.

BREAKING: Meta prohibited from use of personal data for advertisment

Tags: tech, attention-economy, surveillance, facebook

This could be huge if properly enforced. This would reduce profits of personalized ads quite a bit.

https://noyb.eu/en/breaking-meta-prohibited-use-personal-data-advertisment

In 2022, web3 went just great - Molly White

Tags: tech, web3, scam

Maybe they’ll finally realize it just make no sense? It’s time for this absurdity to be put to rest.

https://newsletter.mollywhite.net/p/in-2022-web3-went-just-great

I am not a supplier - Software Maxims

Tags: tech, foss, commons, supply-chain

Friendly reminder, if you’re not paying authors of FOSS libraries, they owe you nothing.

https://www.softwaremaxims.com/blog/not-a-supplier

Journalists (And Others) Should Leave Twitter. Here’s How They Can Get Started | Techdirt

Tags: tech, attention-economy, twitter, fediverse

Let’s hope journalists hear that call. It’s indeed sad that so far it’s mostly words and not many actions to move away from Twitter in that profession.

https://www.techdirt.com/2023/01/04/journalists-and-others-should-leave-twitter-heres-how-they-can-get-started/

The internet wants to be fragmented - by Noah Smith

Tags: tech, social-media, internet

Interesting take, let’s see if it’s true and things will decentralize (or at least audiences fragment, the author seems to confuse both) more in the future.

https://noahpinion.substack.com/p/the-internet-wants-to-be-fragmented

Bring back personal blogging - The Verge

Tags: tech, web, blog, culture

Let’s hope it’s one good resolution for 2023 that plenty will go for. We need blogs to be back, massively. It would be better for everyone.

https://www.theverge.com/23513418/bring-back-personal-blogging

The state of HTTP in 2022

Tags: tech, http, web

That’s a lot which happened in this community over the past year. It’s important that is keeps pushing forward and luckily it does.

https://blog.cloudflare.com/the-state-of-http-in-2022/

The Rise of Monolithic Software

Tags: tech, web, protocols, http, vendor-lockin

Interesting piece. It shows quite well what users have lost with the over reliance on HTTP for everything. Moving more and more things in the brother fosters walled gardens indeed. Compound this with branding obsession from most company and you indeed end up with an absurd situation.

https://itnext.io/the-rise-of-monolithic-software-9e538cfec6e4

You Want Modules, Not Microservices

Tags: tech, microservices, architecture, organization

Excellent piece, looking back to history to justify why microservices are mostly a fad. Check what your needs really are and depending on them pick the right way to decompose the problem or organize your teams.

https://blogs.newardassociates.com/blog/2023/you-want-modules-not-microservices.html

Parallelizing C++ using Execution Policies | Azeem Bande-Ali | Engineering Manager

Tags: tech, multithreading, c++

A little reminder that those too often forgotten execution policies for C++ algorithms actually exist and can give interesting results nowadays.

https://azeemba.com/posts/cpp17-execution-policy.html

Performance of WebAssembly runtimes in 2023 | Frank DENIS random thoughts.

Tags: tech, performance, webassembly

Time to look a bit at the maze of WebAssembly runtimes. Good overview on how they currently perform and how well they are documented or easy to use.

https://00f.net/2023/01/04/webassembly-benchmark-2023/

Microfeatures I’d like to see in more languages • Buttondown

Tags: tech, programming

Since I’m also a bit of a nerd of nice programming language features, that’s an interesting list (mostly) coming from less known languages. Some of that syntactic sugar would be welcome in more main stream languages I think.

https://buttondown.email/hillelwayne/archive/microfeatures-id-like-to-see-in-more-languages/

Test Desiderata. Go placidly amid the noise and haste… | by Kent Beck | Medium

Tags: tech, tests, tdd

This what we should strive for with our tests. I like how he keeps it flexible though, again it’s likely a trade-off so you can’t have all the properties fully all the time. Still you need to know what you give up, how much of it and why.

https://medium.com/@kentbeck_7670/test-desiderata-94150638a4b3

Fast Path to a Great UX – Increased Exposure Hours — UX Articles by UIE

Tags: tech, ux

A bit old but interesting finding. Kind of confirms my own view about it: it’s best when everyone (not just designers) can interact with the users of the system you’re building.

https://articles.uie.com/user_exposure_hours/

Sign-up Versus Assignment - by Kent Beck - Geek Incentives

Tags: tech, project-management, management

It’s clearly a choice in management style. For such choices, always keep in mind the trade offs this create, maybe it’ll push you to revise your choice.

https://geekincentives.substack.com/p/sign-up-versus-assignment

Navigating power dynamics as a manager

Tags: management, empathy

Like it or not (I’m part of those who don’t like it) but the role of manager will necessarily create power imbalances. This article is thus a must read to managers at any level to know how to deal with it properly.

https://leaddev.com/communication-relationships/navigating-power-dynamics-manager

Conversation Skills Essentials – Tynan.com

Tags: culture, empathy

Interesting list of tips for better conversations. Some of them are common knowledge at that point, some others less so.

https://tynan.com/letstalk/

Tangram Heightmapper

Tags: tech, 3d

OK, this is really cool for all your realistic height map needs!

https://tangrams.github.io/heightmapper/

Why the super rich are inevitable

Tags: economics, politics

Very interesting model, I didn’t know about this one. As pointed out you can’t really base policy decisions upon it but that’s still powerful since it explains some of the phenomena at play in the real world. In this way it is enough to debunk some of the assumptions taken a bit too much for granted.

https://pudding.cool/2022/12/yard-sale/

Bye for now!

As an Linux application developer, one might not aware that there could be certain effort required to support Input Method (or Input Method Editor, usually referred as IME) under Linux.

What is input method and why should I care about it?

Even if you are not aware, you are probably already using it in daily life. For example, the virtual keyboard on your smart phone is a form of input method. You may noticed that the virtual keyboard allows you to type something, and gives you a list of words based on what you already partially typed. That is a very simple use case of input method. But for CJKV (Chinese, Japanese, Korean, Vietnamese) users, Input method is necessary for them to type their own language properly. Basically imagine this: you only have 26 English key on the keyboard, how could you type thousands of different Chinese characters by a physical keyboard with only limited keys? The answers, using a mapping that maps a sequence of key into certain characters. In order to make it easy to memorize, usually such mapping is similar to what is called Transliteration , or directly use an existing Romanization system.

For example, the most popular way for typing Chinese is Hanyu Pinyin.

What do I need to do to support Input method?

The state of art of input method on Linux are all server-client based frameworks. The client is your application, and the server is the input method server. Usually, there is also a third daemon process that works as a broker to transfer the message between the application and the input method server.

1. Which GUI toolkit to use?

Gtk & Qt

If you are using Gtk, Qt, there is a good news for you. There is usually nothing you need to do to support input method. Those Gtk toolkit provides a generic abstraction and sometimes even an extensible plugin system (Gtk/Qt case) behind to hide all the complexity for the communication between input method server and application.

The built-in widget provided by Gtk or Qt already handles everything need for input method. Unless you are implementing your own fully custom widget, you do not need to use any input method API. If you need your custom widget, which sometimes happens, you can also use the API provided by the toolkit to implement it.

Here are some pointers to the toolkit API:

Gtk: gtk_im_multicontext_new GtkIMContext

Qt: https://doc.qt.io/qt-6/qinputmethod.html https://doc.qt.io/qt-6/qinputmethodevent.html

The best documentation about how to use those API is the built-in widget implementation.

SDL & winit

If you are using SDL, or rust’s winit, which does have some sort of input method support, but lack of built-in widget (There might be third-party library based on them, which I have no knowledge of), you will need to refer to their IME API to do some manual work, or their demos.

Refer to their offical documentation and examples for the reference:

https://wiki.libsdl.org/SDL2/Tutorials-TextInput

https://github.com/libsdl-org/SDL/blob/main/test/testime.c

https://github.com/rust-windowing/winit/blob/master/examples/ime.rs

Xlib & XCB

Xlib has built-in XIM protocol support, which you may access via Xlib APIs. I found a good article about how to add input method support with Xlib at:

https://tedyin.com/posts/a-brief-intro-to-linux-input-method-framework/

As for XCB, you will need to use a third-party library. I wrote one for XCB for both server and client side XIM. If you need a demo of it, you can find one at:

https://github.com/fcitx/xcb-imdkit/blob/master/test/client_demo.c

Someone also wrote a rust binding for it, which is used by wezterm in real world project. Some demo code can be found at:

https://github.com/H-M-H/xcb-imdkit-rs/tree/master/examples

wayland-client

As for writing a native wayland application from scratch with wayland-client, then you will want to pick the client side input method protocol first. The only common well supported (GNOME, KWin, wlroots, etc, but not weston, just FYI) one is:

https://wayland.app/protocols/text-input-unstable-v3

2. How to write one with the APIs above?

If you use a toolkit with widget that can already support input method well, you can skip this and call it a day. But if you need to use low level interaction with input method, or just interested in how this works, you may continue to read. Usually it involves following steps:

1. Create a connection to input method service.
2. Tell input method, you want to communicate with it.
3. Keyboard event being forwarded to input method
4. input method decide how key event is handled.
5. Receives input method event that carries text that you need to show, or commit to the application.
6. Tell input method you are done with text input
7. Close the connection when your application ends, or the relevant widget destructs.

The 1st step sometimes contains two steps, a. create connection. b. create a server side object that represent a micro focus of your application. Usually, this is referred as “Input Context”. The toolkit may hide the these complexity with their own API.

Take Xlib case as an example:

1. Create the connection: XOpenIM
2. Create the input context: XCreateIC
3. Tell input method your application wants to use text input with input method: XSetICFocus
4. Forward keyevent to input method: XFilterEvent
5. Get committed text with XLookupString
6. When your widget/window lost focus, XUnsetICFocus
7. Clean up: XDestroyIC, XCloseIM.

Take wayland-client + text-input-v3 as an example

1. Get global singleton object from registry: zwp_text_input_manager_v3
2. Call zwp_text_input_manager_v3.get_text_input
3. Call zwp_text_input_v3.enable
4. Key event is forward to input method by compositor, nothing related to keyboard event need to be done on client side.
5. Get committed text zwp_text_input_v3.commit_string
6. Call zwp_text_input_v3.disable
7. Destroy relevant wayland proxy object.

And always, read the example provided by the toolkit to get a better idea.

3. Some other concepts except commit the text

Support input method is not only about forwarding key event and get text from input method. There are some more interaction required between application and input method that is important to give better user experience.

Preedit

Preedit is a piece of text that is display by application that represents the composing state. See the screenshot at the beginning of this article, the “underline” text is the “preedit”. Preedit contains the text and optionally some formatting information to show some rich information.

Surrounding Text

Surrounding text is an optional information that application can provide to input method. It contains text around the cursor, where the cursor and user selection is. Input method may use those information to provide better prediction. For example, if your text box has “I love |” ( | is the cursor). With surrounding text, input method will know that there is already “I love ” in the box and may predict your next word as “you” so you don’t need to type “y-o-u” but just select from the prediciton.

Surrounding text is not supported by XIM. Also, not all application can provide valid surrounding text information, for example terminal app.

Reporting cursor position on the window

Many input method engine needs to show a popup window to display some information. In order to allow input method place the window just at the position of the cursor (blinking one), application will need to let input method know where the cursor is.

Notify the state change that happens on the application side

For example, even if user is in the middle of composing something, they may still choose to use mouse click another place in the text box, or the text content is changed programmatically by app’s own logic. When such things happens, application may need to notify that the state need a “reset”. Usually this is also called “reset” in the relevant API.

Commit: https://invent.kde.org/qt/qt/qt5/-/commit/281044e2541c842f8d0b0bc1a199999bf9d9951c

Commercial release announcement: https://www.qt.io/blog/commercial-lts-qt-5.15.8-released

As usual I want to personally extend my gratitude to the Commercial users of Qt for beta testing Qt 5.15.8 for the rest of us.

The Commercial Qt 5.15.8 release introduced two bugs that have later been fixed. Thanks to that, our Patchset Collection has been able to incorporate the the fix for one of the issues [1] and revert for the other [2]  and the Free Software users will never be affected by it! 

P.S: Special shout-out to Andreas Sturmlechner for identifying the fix of the issue, since I usually only pay attention to "Revert XYZ" commits and this one is not a revert but subsequent improvement

Wayland has a unique way to let clients specify the contents of the cursor. After receiving a wl_pointer.enter event, the client must call wl_pointer.set_cursor request

    <request name="set_cursor">
      <arg name="serial" type="uint" summary="serial number of the enter event"/>
      <arg name="surface" type="object" interface="wl_surface" allow-null="true"
	   summary="pointer surface"/>
      <arg name="hotspot_x" type="int" summary="surface-local x coordinate"/>
      <arg name="hotspot_y" type="int" summary="surface-local y coordinate"/>
    </request>

The wl_pointer.set_cursor request takes an optional wl_surface object that represents the actual contents of the cursor. That’s it, the wl_surface interface is used both by windows and cursors! It opens a whole lot of features that you could use with the cursor, for example use the wp_viewport protocol for fractional scaling or create cursor surface trees using the wl_subcompositor interface. On the other hand, many compositors view the cursor as a simple image. So, let’s see how we improved kwin in this regard.

Cursor source

Currently (as of 5.26.x), kwin assumes that the cursor can show only a QImage. It’s okay for simple cases, but it falls apart once we need to show a wl_surface, e.g. we will hit problems with getting a QImage from a linux dmabuf client buffer.

So the first thing that we need to do in order to move anywhere forward is to choose proper abstractions to represent what is actually in the cursor. For example, if the cursor hovers a window, it obviously needs to present what the corresponding wl_surface contains. But sometimes the mouse cursor is not above any window, for example if the pointer is above a server-side decoration. Server-side decoration can be considered part of the window, but we cannot use client’s wl_surface anymore, the compositor may choose to show a different cursor, which is a QImage.

class KWIN_EXPORT CursorSource : public QObject
{
    Q_OBJECT

public:
    explicit CursorSource(QObject *parent = nullptr);

    QImage image() const;
    QSize size() const;
    QPoint hotspot() const;

Q_SIGNALS:
    void changed();
};

The CursorSource class is the base class for all other “source” classes that can be attached to the cursor. It contains generic properties, e.g. the hotspot, and it also contains the CursorSource::changed() signal to tell the world when the image has changed. CursorSource::image() exists for compatibility and to make the transition to new abstractions easier.

Sometimes, we need to show a static image in the cursor, so let’s add an ImageCursorSource to serve that purpose

class ImageCursorSource : public CursorSource
{
public:
    explicit ImageCursorSource(QObject *parent = nullptr);

public Q_SLOTS:
    void update(const QImage &image, const QPoint &hotspot);
};

On the other hand, some cursors are not static. For example, the loading cursor usually contains some animation, e.g. spinning wheel

class ShapeCursorSource : public CursorSource
{
public:
    explicit ShapeCursorSource(QObject *parent = nullptr);

    QByteArray shape() const;
    void setShape(const QByteArray &shape);
    void setShape(Qt::CursorShape shape);

    KXcursorTheme theme() const;
    void setTheme(const KXcursorTheme &theme);

private:
    void refresh();
    void selectNextSprite();
    void selectSprite(int index);

    KXcursorTheme m_theme;
    QByteArray m_shape;
    QVector<KXcursorSprite> m_sprites;
    QTimer m_delayTimer;
    int m_currentSprite = -1;
};

The ShapeCursorSource class represents a cursor shape from an Xcursor theme. It can be used to show both animated and static cursors. If the given cursor shape is animated, i.e. it has more than one sprite, ShapeCursorSource will start a timer with the timeout as indicated by the cursor theme. When the timer expires, ShapeCursorSource will switch to the next sprite and emit the CursorSource::changed() signal. If it’s the last sprite, it will wrap around to the first sprite.

And last but not least, we need something to represent wl_surface attached to the cursor

class SurfaceCursorSource : public CursorSource
{
public:
    explicit SurfaceCursorSource(QObject *parent = nullptr);

    KWaylandServer::SurfaceInterface *surface() const;

public Q_SLOTS:
    void update(KWaylandServer::SurfaceInterface *surface, const QPoint &hotspot);
};

ImageCursorSource, ShapeCursorSource, and SurfaceCursorSource are the main types that indicate what the cursor shows.

Scene

The CursorSource classes act as data sources, they don’t actually paint anything on the screen. It’s the responsibility of the scene. I’ve already written a little bit about the scene abstraction in kwin, I recommend you to read my earlier blog post about it https://blog.vladzahorodnii.com/2021/04/12/scene-items-in-kwin/

But as a quick recap: kwin breaks up a window in smaller building blocks called items. The DecorationItem corresponds to the server-side decoration if there’s one. The ShadowItem is a server-side drop shadow, for example a drop shadow cast by the decoration or the panel. The SurfaceItem represents the actual window contents. That’s the same strategy that we will follow here. The cursor will be broken in smaller pieces.

If we look closely at our cursor sources, the painting code should handle only two cases – paint a QImage and paint an arbitrary wl_surface tree. The wl_surface case is already taken care by SurfaceItem \o/, so we just need a new type of item to present an image in the scene graph, e.g. ImageItem

The CursorItem type ties both cases together. It monitors what source is attached to the cursor and creates either a SurfaceItem or an ImageItem according to the type of the cursor source

• If a SurfaceCursorSource is attached to the cursor, the CursorItem is going to destroy its child ImageItem (if there’s one) and create a SurfaceItem tree
• If an ImageCursorSource or a ShapeCursorSource is attached to the cursor, the CursorItem is going to destroy its child SurfaceItem (if there’s one) and create an ImageItem child item

Note that SurfaceItems are rendered the same way regardless of their role.

With all of this, kwin can easily handle simple cases such as displaying an image in the cursor and more esoteric things that you can do with a wl_surface.

Cursor layer

The last thing that’s needed to make cursor handling perfect is putting the cursor on its own hardware plane. Imagine that you move the mouse cursor. Ideally, you should not repaint any windows below the cursor, only update the position of the cursor. Unless the windows need to repaint their content because of new pointer position, e.g. add or remove button outlines, etc. It can be achieved by painting the cursor in its own plane.

KWin already attempts to put the cursor on hardware planes, but we would like to clean things up and unify cursor and overlay planes. It is still work in progress. TBA.

Summary

The new cursor design is more powerful and should make it easier to add new fancy features. For example, it would be amazing if you could wire the velocity of the cursor to its scale so you could shake the pointer in order to find the cursor more easily. The new design also fixes longstanding limitations that prevented kwin from displaying cursors rendered by OpenGL or Vulkan.