Some more progress has been made on fixing my newest drawing tablet!

Fixing up the patch

So as described in the original post, I have to patch the uclogic HID driver. Let’s start by going through the process of submitting a patch upstream!

Before we even think about sending this patch upstream, I have to first - fix it. While the patch is mostly fine in it’s original form, there is one big issue I would like to tackle - which is the tablet frame buttons. Not even the original author seemed to figure out this issue, so I’m excited to jump in and figure it out.

First I want to set an end goal, which is to be able to set the tablet buttons through the new KDE Plasma 5.26 interface, like so:

Currently if you run my new uclogic patch, for some reason it is not considered a “Pad”:

Looking through the merge request shows.. nothing of interest. That’s because I actually needed to look at this KWin merge request which actually talks to libinput! Oh yeah, libinput - our old friend is here again. This MR mentions an event called LIBINPUT_EVENT_TABLET_PAD_BUTTON which is handled by evdev-tablet-pad.c in libinput. Huh, whats this?

...
static inline void
pad_button_set_down(struct pad_dispatch *pad,
		    uint32_t button,
		    bool is_down)
{
	struct button_state *state = &pad->button_state;

	if (is_down) {
		set_bit(state->bits, button);
		pad_set_status(pad, PAD_BUTTONS_PRESSED);
	} else {
		clear_bit(state->bits, button);
		pad_set_status(pad, PAD_BUTTONS_RELEASED);
	}
}
...

So this file seems to be handling “tablet pad” devices, which I didn’t even know were considered a separate device until now. Looking through the code reveals logic that libinput uses to decide what buttons reported by the device are meant for tablet pad usage:

...

static void
pad_init_buttons_from_kernel(struct pad_dispatch *pad,
			       struct evdev_device *device)
{
	unsigned int code;
	int map = 0;

	/* we match wacom_report_numbered_buttons() from the kernel */
	for (code = BTN_0; code < BTN_0 + 10; code++) {
		if (libevdev_has_event_code(device->evdev, EV_KEY, code))
			map_set_button_map(pad->button_map[code], map++);
	}

	for (code = BTN_BASE; code < BTN_BASE + 2; code++) {
		if (libevdev_has_event_code(device->evdev, EV_KEY, code))
			map_set_button_map(pad->button_map[code], map++);
	}

	for (code = BTN_A; code < BTN_A + 6; code++) {
		if (libevdev_has_event_code(device->evdev, EV_KEY, code))
			map_set_button_map(pad->button_map[code], map++);
	}

	for (code = BTN_LEFT; code < BTN_LEFT + 7; code++) {
		if (libevdev_has_event_code(device->evdev, EV_KEY, code))
			map_set_button_map(pad->button_map[code], map++);
	}

	pad->nbuttons = map;
}
...

So I created a list of valid pad inputs and used those to remap all of the pad buttons to valid inputs. This shouldn’t affect existing tablets (especially considering I don’t think anyone has used a tablet with this driver with more than a couple buttons).

...
/* Buttons considered valid tablet pad inputs. */
const unsigned int uclogic_extra_input_mapping[] = {
	BTN_0,
	BTN_1,
	BTN_2,
	BTN_3,
	BTN_4,
	BTN_5,
	BTN_6,
	BTN_7,
	BTN_8,
	BTN_RIGHT,
	BTN_MIDDLE,
	BTN_SIDE,
	BTN_EXTRA,
	BTN_FORWARD,
	BTN_BACK,
	BTN_B,
	BTN_A,
	BTN_BASE,
	BTN_BASE2,
	BTN_X
};
...
/*
* Remap input buttons to sensible ones that are not invalid.
* This only affects previous behavior for devices with more than ten or so buttons.
*/
const int key = (usage->hid & HID_USAGE) - 1;

if (key > 0 && key < ARRAY_SIZE(uclogic_extra_input_mapping)) {
        hid_map_usage(hi,
                        usage,
                        bit,
                        max,
                        EV_KEY,
                        uclogic_extra_input_mapping[key]);
        return 1;
}

And that’s pretty much the only major change I did to the original patch! You can see the patch in linux-input here.

However that isn’t the end of the story, as there’s one issue - libinput doesn’t think my tablet pad is a… tablet pad?

# libinput record /dev/input/event12
...
  udev:
    properties:
    - ID_INPUT=1
    - ID_INPUT_MOUSE=1
    - LIBINPUT_DEVICE_GROUP=3/28bd/91b:usb-0000:2f:00.3-4.1
...

Okay, that’s interesting - why is this a mouse? Uh oh, don’t tell me…

Fixing udev

So unfortunately before my tablet is fully functional, we have to touch another piece of critical Linux infrastructure - great. I want to dive into how libinput even decides device types, and I promise you it’s harder than it should be.

If you never touched the Linux input stack before (like I did before I started this project) you might think, “whats the big deal? the mouse says it’s a mouse, a keyboard says it’s a keyboard, whatever.” except it’s never that simple. If you’ve been reading my blog posts, you might think that’s just information evdev will hand over, which you’re wrong there too. All evdev tells userspace is what events the device will emit - that’s it². So who figures out what is what?

Unfortunately, it’s a guessing game. No you heard me right, udev guesses what devices are. So how do we tell what devices are which? Luckily systemd ships with a hardware database, which details stuff like id vendors and manual fixes for esoteric devices. This is actually what the original patch author did in order to fix udev classifying it as a mouse:

id-input:*:input:b0003v28BDp091Be0100*
 ID_INPUT_MOUSE=0
 ID_INPUT_TABLET=1
 ID_INPUT_TABLET_PAD=1
...

However I didn’t like this solution, and it would be a pain to find the modalias for every single tablet that ever existed, that udev mistakenly categorizes. Is there a better solution? Instead I changed the logic in src/udev/udev-builtin-input_id.c. Let’s take a look at the original code path:

...
is_pointing_stick = test_bit(INPUT_PROP_POINTING_STICK, bitmask_props);
has_stylus = test_bit(BTN_STYLUS, bitmask_key);
has_pen = test_bit(BTN_TOOL_PEN, bitmask_key);
finger_but_no_pen = test_bit(BTN_TOOL_FINGER, bitmask_key) && !test_bit(BTN_TOOL_PEN, bitmask_key);
for (int button = BTN_MOUSE; button < BTN_JOYSTICK && !has_mouse_button; button++)
        has_mouse_button = test_bit(button, bitmask_key);
has_rel_coordinates = test_bit(EV_REL, bitmask_ev) && test_bit(REL_X, bitmask_rel) && test_bit(REL_Y, bitmask_rel);
has_mt_coordinates = test_bit(ABS_MT_POSITION_X, bitmask_abs) && test_bit(ABS_MT_POSITION_Y, bitmask_abs);

/* unset has_mt_coordinates if devices claims to have all abs axis */
if (has_mt_coordinates && test_bit(ABS_MT_SLOT, bitmask_abs) && test_bit(ABS_MT_SLOT - 1, bitmask_abs))
        has_mt_coordinates = false;
is_direct = test_bit(INPUT_PROP_DIRECT, bitmask_props);
has_touch = test_bit(BTN_TOUCH, bitmask_key);
has_pad_buttons = test_bit(BTN_0, bitmask_key) && has_stylus && !has_pen;
...
if (has_mt_coordinates) {
        if (has_stylus || has_pen)
                is_tablet = true;
        else if (finger_but_no_pen && !is_direct)
                is_touchpad = true;
        else if (has_touch || is_direct)
                is_touchscreen = true;
}

if (is_tablet && has_pad_buttons)
        is_tablet_pad = true;

if (!is_tablet && !is_touchpad && !is_joystick &&
        has_mouse_button &&
        (has_rel_coordinates ||
        !has_abs_coordinates)) /* mouse buttons and no axis */
        is_mouse = true;

To explain a little of context, udev has several “built-in” commands to decide what devices are what, and if any quirks or workarounds need to be applied. Notably, the input_id builtin handles classifying input types for devices if they are not already in the hardware database. If you turn on udev debug logging, you can actually see the built-ins it runs whenever you connect, say a usb device to your system. As you can see, evdev gives udev the supported event types of the device, and then udev has to figure out what the device actually is. Now let’s look at the supported event types by the tablet pad:

# libinput record /dev/input/event12
...
devices:
- node: /dev/input/event12
  evdev:
    # Name: UGTABLET 21.5 inch PenDisplay Pad
    # ID: bus 0x3 vendor 0x28bd product 0x91b version 0x100
    # Supported Events:
    # Event type 0 (EV_SYN)
    # Event type 1 (EV_KEY)
    #   Event code 256 (BTN_0)
    #   Event code 257 (BTN_1)
    #   Event code 258 (BTN_2)
    #   Event code 259 (BTN_3)
    #   Event code 260 (BTN_4)
    #   Event code 261 (BTN_5)
    #   Event code 262 (BTN_6)
    #   Event code 263 (BTN_7)
    #   Event code 264 (BTN_8)
    #   Event code 265 (BTN_9)
    #   Event code 266 ((null))
    #   Event code 267 ((null))
    #   Event code 268 ((null))
    #   Event code 269 ((null))
    #   Event code 270 ((null))
    #   Event code 271 ((null))
    #   Event code 272 (BTN_LEFT)
    #   Event code 273 (BTN_RIGHT)
    #   Event code 274 (BTN_MIDDLE)
    #   Event code 275 (BTN_SIDE)
    # Event type 2 (EV_REL)
    #   Event code 6 (REL_HWHEEL)
    #   Event code 8 (REL_WHEEL)
    #   Event code 11 (REL_WHEEL_HI_RES)
    #   Event code 12 (REL_HWHEEL_HI_RES)
    # Event type 4 (EV_MSC)
    #   Event code 4 (MSC_SCAN)
...

I encourage you to read through the original systemd code, as the bug (I think) is hilariously obvious. The tablet pad never emits any BTN_STYLUS events. In fact, my tablet is not even configured on a firmware level to ever emit any BTN_STYLUS events, because the attached log above is without my patch! Obviously this is bad, as that means possibly other tablets are wrongly accused of being mice, and we should fix that logic. That’s also what I didn’t want to simply shove a new entry into the hardware database, as this turns out to be a udev bug.

So I made a systemd PR fixing the logic, maybe that might fix your tablet too!

Tilt support

One more thing I wanted to tackle was tilt support for the pen, which the driver says it supported but I haven’t tested yet. Simple, right? (you already know the answer to that question)

The first thing to do is just evdev:

# evtest /dev/input/event11
...
Event: time 1672152523.926503, -------------- SYN_REPORT ------------
Event: time 1672152523.938496, type 3 (EV_ABS), code 0 (ABS_X), value 37529
Event: time 1672152523.938496, type 3 (EV_ABS), code 1 (ABS_Y), value 13166
Event: time 1672152523.938496, type 3 (EV_ABS), code 27 (ABS_TILT_Y), value 12
Event: time 1672152523.938496, -------------- SYN_REPORT ------------
Event: time 1672152523.952501, type 3 (EV_ABS), code 27 (ABS_TILT_Y), value 13
Event: time 1672152523.952501, -------------- SYN_REPORT ------------
Event: time 1672152523.956501, type 3 (EV_ABS), code 0 (ABS_X), value 37534
Event: time 1672152523.956501, type 3 (EV_ABS), code 1 (ABS_Y), value 13138
Event: time 1672152523.956501, type 3 (EV_ABS), code 27 (ABS_TILT_Y), value 14
Event: time 1672152523.956501, -------------- SYN_REPORT ------------
Event: time 1672152523.962497, type 3 (EV_ABS), code 27 (ABS_TILT_Y), value 15
...

So it looks like it’s reporting tilt events, that’s good. Let’s go one layer up, and check with libinput. Actually, instead of using debug-events or record, libinput actually has a handy tablet testing function called debug-tablet:

#  libinput debug-tablet
Device: UGTABLET 21.5 inch PenDisplay Pen (event11)
Tool: pen serial 0, id 0
libinput:
tip: up
  x:                 299.38 [-------------------------------------------------|-----------------------------]
  y:                 140.64 [-----------------------------------------|-------------------------------------]
  tilt x:            -54.53 [---------------|---------------------------------------------------------------]
  tilt y:             41.21 [---------------------------------------------------------|---------------------]
  dist:                0.00 [|------------------------------------------------------------------------------]
  pressure:            0.00 [|------------------------------------------------------------------------------]
  rotation:            0.00 [|------------------------------------------------------------------------------]
  slider:              0.00 [---------------------------------------|---------------------------------------]
  buttons:
evdev:
  ABS_X:           29952.00 [-------------------------------------------------|-----------------------------]
  ABS_Y:           14077.00 [-----------------------------------------|-------------------------------------]
  ABS_Z:               0.00 [|------------------------------------------------------------------------------]
  ABS_TILT_X:        -55.00 [----|--------------------------------------------------------------------------]
  ABS_TILT_Y:         41.00 [-----------------------------------------------------------------|-------------]
  ABS_DISTANCE:        0.00 [|------------------------------------------------------------------------------]
  ABS_PRESSURE:        0.00 [|------------------------------------------------------------------------------]
  buttons: BTN_TOOL_PEN

Alright that’s good, it looks like libinput is recording the tilt too. How about inside of Krita, which has tilt support? Uhhh… let’s check the tablet tester:

Wait, there’s no way to check for tilt here! That was a really weird omission, so I fixed that in Krita 5.2. Then I thought there was no way that no developer in Krita somehow didn’t have tilt information debugged somewhere, so after a bit searching I found a tool for Tablet Event Logging activated by CTRL+SHIFT+T. Using that, we can find if Krita has picked up the tilt from our pen:

$ krita
...
krita.tabletlog: "[BLOCKED 2:] MouseMove        btn: 0 btns: 0 pos:  828, 515 gpos:  859,2778 hires:  858.956, 2777.95 Source:2"
krita.tabletlog: "[       ] TabletMove       btn: 0 btns: 0 pos:  828, 516 gpos:  859,2779 hires:  859.281,    2779 prs: 0.000000 Stylus Pen id: 0 xTilt: 0 yTilt: 0 rot: 0 z: 0 tp: 0 "
krita.tabletlog: "[BLOCKED 2:] MouseMove        btn: 0 btns: 0 pos:  828, 516 gpos:  859,2779 hires:  859.281,    2779 Source:2"
krita.tabletlog: "[       ] TabletMove       btn: 0 btns: 0 pos:  828, 517 gpos:  859,2780 hires:  859.441, 2779.72 prs: 0.000000 Stylus Pen id: 0 xTilt: 0 yTilt: 0 rot: 0 z: 0 tp: 0 "
krita.tabletlog: "[BLOCKED 2:] MouseMove        btn: 0 btns: 0 pos:  828, 517 gpos:  859,2780 hires:  859.441, 2779.72 Source:2"
krita.tabletlog: "[       ] TabletMove       btn: 0 btns: 0 pos:  829, 517 gpos:  860,2780 hires:  859.562, 2780.25 prs: 0.000000 Stylus Pen id: 0 xTilt: 0 yTilt: 0 rot: 0 z: 0 tp: 0 "
...

Huh? No tilt information at all? That’s slightly worrying. However, there is a layer in-between Krita and libinput, that being the compositor - in my case KWin. I actually didn’t know until now, but KWin has a handy debug tool available by searching KWin in KRunner:

Using that, we can check for the input events that it records:

Well that’s a nice consolation, it looks like KWin is picking up the tilt information from libinput too, so at least it’s not throwing it out. But why does Krita still not receive anything? For some reason, Krita receives tilt information on X11.

To explain, Krita does not interface with KWin directly. Instead, it uses standard protocols - either X or Wayland to communicate with the server or compositor respectively. So why does tilt work when Krita is running under X11 and not when running under Wayland? The trick is that it doesn’t matter, currently Krita is always running under X since it disables Wayland support for some reason. However, XWayland is still technically a Wayland client, so naturally the next place to look is how KWin sends Wayland events. That takes us to the class KWaylandServer

The protocol we care about here is the unstable “tablet” Wayland protocol, currently on version 2. It supports tilt events, so what’s the problem here? In reality, KWin was never sending the events in the first place:

...
const quint32 MAX_VAL = 65535;
tool->sendPressure(MAX_VAL * event->pressure());
tool->sendFrame(event->timestamp());
return true;
...

I actually came across this function when searching through relevant KWin merge requests, notably !3231 which has the unrealistic goal of complete tablet support in just one merge request! I cleaned up the relevant tilt support part, and submitted a new merge request which means pen tilt/rotation is supported under KDE Wayland now! Yay!

Dials

All is well and good now, right? That’s what I thought, until I realized my dials don’t work! I was so entrenched in everything else, that I completely neglected these little do-dads. To explain, these seem to be XP-PEN specific and are just glorified scrollwheels:

However, these are not Wacom rings. Rings on a Wacom tablet (to my knowledge) are actual rings, which report absolute positioning like a wheel. These dials are not those, but are sending relative “scroll” events. I had considered remapping this to buttons on the evdev layer, which was a bad idea since that meant that complicated things further up. Another option was to instead, emulate a Wacom ring by disguising the relative events and turning those into EV_ABS. I also threw that idea away, because that also seems to hide the problem.

If you’re curious, the dials actually worked without my patches - because remember that udev was classifying the pad as a mouse - so libinput thought no further. But now that it’s considered a real tablet pad, it rejects scroll wheel events.

Honestly the best way to solve this issue is to - unfortunately - touch every single bit of the input stack to accommodate these devices. Again.

• The Wayland tablet protocol must be changed to add a few new properties for dials on pads. Tracking via !122.
• Libinput must be modified to expose these dials and correctly map them. Tracking via !845.
• KWin must be modified to support sending dial events in support of the tablet protocol changes. Currently not tracking.
• These dials (and we should also include rings and strips too) should be exposed in the Tablet Settings KCM too. Currently not tracking either.

As you can see, it’s going to be a complex process to support these stupid little circles, but I’m going all in. Luckily, with the rest of my patches going through - everything except for dials on the XP-PEN Artist 22R Pro device should be well supported now. In the coming months, I hope to make headway in supporting dials in KDE.

So I bought an art tablet this year, the XP-Pen Artist 22R Pro which comes with a pen. This pen does not work under Linux, sometimes.

While almost every part of the tablet works, only one of the two stylus buttons is functional. One button is middle mouse click, and the other one is right mouse click. For some reason, only the first stylus button works! This is obviously troublesome, so where do we begin to fix this issue? Let me take you on a journey, if you have no experience with how input works on Linux - well that makes two of us!

Figuring Out The Problem

I work in Krita, so the first place to check is Krita’s tablet testing tool first. Looking at the log yields us something like this:

...
Stylus move X=120.37 Y=129.63 B=4 P=0.0% S=0.0 (DRAW)
Stylus release X=120.37 Y=129.63 B=0 P=0.0% S=0.0
...

According to the legend, we only care about the section named B which tells us which button is pressed. On my pen it’s:

• 0 is released (not pressing on the screen)
• 1 is drawing (pressed on the screen, it doesn’t count hovering)
• 4 is the “middle mouse” button
• The nonfunctional button does not show up at all of course.

So that at least means Krita isn’t the root cause, also testing it on the desktop or any other application yields the same result - the button is non-functional. In some cases, I’ve found the button to actually turn off the stylus entirely, but I’m not sure why yet.

How does tablet input on Wayland actually function? Let’s move down a layer and look at KWin, the Wayland compositor for KDE Plasma.

On Wayland, input is typically handled using libinput, and now even recent X versions have a libinput driver. Digging through the libinput backend on KWin doesn’t reveal anything particularly exciting, so we’ll assume KWin isn’t at fault either for now. Luckily, libinput has a built-on logging utility we can use to check what it sees!

First we want to check what devices libinput sees:

# libinput list-devices
...
Device:           UGTABLET 21.5 inch PenDisplay Mouse
Kernel:           /dev/input/event11
Group:            5
Seat:             seat0, default
Capabilities:     pointer
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      disabled
Nat.scrolling:    disabled
Middle emulation: disabled
Calibration:      identity matrix
Scroll methods:   button
Click methods:    none
Disable-w-typing: n/a
Disable-w-trackpointing: n/a
Accel profiles:   flat *adaptive
Rotation:         n/a

Device:           UGTABLET 21.5 inch PenDisplay Keyboard
Kernel:           /dev/input/event12
Group:            5
Seat:             seat0, default
Capabilities:     keyboard
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      n/a
Nat.scrolling:    n/a
Middle emulation: n/a
Calibration:      n/a
Scroll methods:   none
Click methods:    none
Disable-w-typing: n/a
Disable-w-trackpointing: n/a
Accel profiles:   n/a
Rotation:         n/a

Device:           UGTABLET 21.5 inch PenDisplay
Kernel:           /dev/input/event13
Group:            5
Seat:             seat0, default
Size:             477x268mm
Capabilities:     tablet
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      n/a
Nat.scrolling:    n/a
Middle emulation: n/a
Calibration:      identity matrix
Scroll methods:   none
Click methods:    none
Disable-w-typing: n/a
Disable-w-trackpointing: n/a
Accel profiles:   none
Rotation:         n/a

So it looks like the device we’re interested in is/dev/input/event13. We can now view the debug event log:

# libinput debug-events --verbose
...
 event2   POINTER_MOTION          +25.044s       -1.66/  0.00 ( -1.00/ +0.00)
 event2   POINTER_MOTION          +25.045s       -1.66/  0.00 ( -1.00/ +0.00)
 event2   POINTER_MOTION          +25.046s       -1.87/  1.87 ( -1.00/ +1.00)
 event2   POINTER_MOTION          +25.047s       -1.87/  0.00 ( -1.00/ +0.00)
 event2   POINTER_MOTION          +25.049s       -1.38/  0.00 ( -1.00/ +0.00)
 event2   POINTER_MOTION          +25.050s       -1.38/  0.00 ( -1.00/ +0.00)
 event2   POINTER_MOTION          +25.051s       -1.66/  0.00 ( -1.00/ +0.00)
 event2   POINTER_MOTION          +25.053s       -1.38/  0.00 ( -1.00/ +0.00)
 event2   POINTER_MOTION          +25.055s       -1.11/  0.00 ( -1.00/ +0.00)
 event2   POINTER_MOTION          +25.057s       -1.11/  0.00 ( -1.00/ +0.00)
...

Oh right, there’s a lot of junk in there. We can slim it down to the only device we care about though:

# libinput debug-events --verbose --device /dev/input/event13
...
event13  TABLET_TOOL_AXIS        +9.455s               371.93*/88.01*  tilt: 8.06/15.12        pressure: 0.00
event13  TABLET_TOOL_AXIS        +9.545s               371.67*/88.07*  tilt: 8.06/15.12        pressure: 0.00
event13  TABLET_TOOL_BUTTON      +9.549s       331 (BTN_STYLUS) pressed, seat count: 1
event13  TABLET_TOOL_AXIS        +9.607s               371.38*/88.37*  tilt: 8.06/15.12        pressure: 0.00
event13  TABLET_TOOL_AXIS        +9.627s               371.38/88.56*   tilt: 8.19*/15.12       pressure: 0.00
...

As you can see, whenever I press the first stylus button it reports BTN_STYLUS, but when I press the second one it instead reports itself as BTN_TOUCH (not shown in the snippet).

Now before we go further, I just wanted to confirm whether or not the pen actually worked at all so I booted up Windows real quick to test. And it does work as expected, darn.

Going Deeper

Now it’s easy to assume libinput is the culprit and the one to be fixed, like I did. At first, I wrote up a quick patch to remap the wrong BTN_TOUCH inputs and inject proper BTN_STYLUS2 events but honestly that’s dirty. So what is beneath libinput? evdev!

So now that we know the (possible) culprit, we need to dig deeper into libinput. Like a good software project, libinput has developer documentation for us to look at.

Let’s check the event device that Linux gives us, using the handy evtest:

# evtest /dev/input/event13
...
Event: time 1671823416.485402, -------------- SYN_REPORT ------------
Event: time 1671823416.517402, type 4 (EV_MSC), code 4 (MSC_SCAN), value d0045
Event: time 1671823416.517402, type 1 (EV_KEY), code 330 (BTN_TOUCH), value 0
Event: time 1671823416.517402, type 1 (EV_KEY), code 320 (BTN_TOOL_PEN), value 0
Event: time 1671823416.517402, type 3 (EV_ABS), code 26 (ABS_TILT_X), value 0
Event: time 1671823416.517402, -------------- SYN_REPORT ------------
...
Event: time 1671823416.825414, -------------- SYN_REPORT ------------
Event: time 1671823416.835640, type 4 (EV_MSC), code 4 (MSC_SCAN), value d0045
Event: time 1671823416.835640, type 1 (EV_KEY), code 330 (BTN_TOUCH), value 1
Event: time 1671823416.835640, type 3 (EV_ABS), code 26 (ABS_TILT_X), value 29
Event: time 1671823416.835640, -------------- SYN_REPORT ------------

Uh oh, the same bug appears here too. So it’s time to sigh look deeper into the stack. Unfortunately, going deeper means stepping into the kernel. What is actually handling the input?

# lsusb -t
...
    |__ Port 4: Dev 5, If 0, Class=Hub, Driver=hub/2p, 480M
        |__ Port 1: Dev 8, If 0, Class=Hub, Driver=hub/4p, 480M
            |__ Port 3: Dev 9, If 0, Class=Human Interface Device, Driver=usbhid, 12M
            |__ Port 3: Dev 9, If 1, Class=Human Interface Device, Driver=usbhid, 12M
            |__ Port 3: Dev 9, If 2, Class=Human Interface Device, Driver=usbhid, 12M
...

It looks like the “usbhid” driver which makes sense, this is a USB HID device after all. However at this point is where I got stuck, because as far as I know there is no way to extract the specific HID driver in use (unless you know the actual module name). This is annoying, so I went digging inside of the kernel source tree for any mention of my device ID (0x091b). Nope. So in turn we can assume it’s using the generic HID driver, which is actually kind of impressive.

So what is the proper HID driver for this device? Well after some research it turns out to be uclogic which handles a bunch of UGTablet, Huion and other misc Asian tablet devices. Unfortunately, they haven’t added any support for this model but someone has.

Digimend Project

Luckily there is a project dedicated to upstreaming device drivers for these tablets, and Aren Villanuvea (also known as “kurikaesu”) known for userspace-tablet-driver-daemon has reverse engineered the tablet already. Better yet, he even implemented most of the work in uclogic already! You can see the PR here.

Now that’s where the story ends, yay - except that my device still doesn’t work. That’s because the PR never made it into Digimend, and it was closed by the author without explanation. And because it never made it into Digimend, it never landed upstream. It looks like Aren has decided it was a better use of his time to implement it into userspace instead.

The daemon is nice and all, but I would prefer kernel support for this device for numerous reasons. And what’s worse is that it’s so close to being complete that it sucks that the work hasn’t continued. Oh well…

Kernel Hacking

So now I’m trying to upstream the uclogic patch! Oh how I wish it was a bug in a higher layer… I’ll be taking over Aren’s work and trying to clean up the patch and reduce it down to just supporting the 22R Pro, since that’s the only tablet I own. Here’s what works:

• Both stylus buttons work!
• Pen pressure works.
• Both dials work, although the right one is registered as horizontal scroll which is odd - that might have to be re-mappable on a higher layer.
• The tablet buttons “work” but some of them are not registered properly as there’s more buttons than HID definitions for buttons. I might have to split this up into two devices, or something.

Now here’s my to-do list:

• Hopefully receive Aren’s blessing to release the patches with his original ownership.
• Figure out a fix with the tablet buttons, and contact Nicolas Fella (who is working on the re-mappable tablet buttons in 5.27) to see what needs to be done so they’re re-mappable in the new GUI.
• Choose whether or not to contribute it directly to upstream (slow?) or back to Digimend.

If anyone is familiar with kernel contributions in the HID subsystem, or know the right people (like the ones mentioned in the article, hi!) I would love to get in contact!

Let’s go for my web review for the week 2022-51.

Charge - Blender Open Movie - Blender Video

Tags: tech, blender, 3d, movie

The latest Blender Open Movie. Again a very good one!

https://video.blender.org/w/04da454b-9893-4184-98f3-248d00625efe

UnifiedPush: a decentralized, open-source push notification protocol | F-Droid - Free and Open Source Android App Repository

Tags: tech, mobile, android, notifications

This is a big deal, it’s time we open up that can of worms.

https://f-droid.org/2022/12/18/unifiedpush.html

MSG defends using facial recognition to kick lawyer out of Rockettes show | Ars Technica

Tags: tech, surveillance

This is really creepy… how can this be even allowed?

https://arstechnica.com/tech-policy/2022/12/facial-recognition-flags-girl-scout-mom-as-security-risk-at-rockettes-show/

Okta’s source code stolen after GitHub repositories hacked

Tags: tech, security

This sounds like a bad series of events for such a company…

https://www.bleepingcomputer.com/news/security/oktas-source-code-stolen-after-github-repositories-hacked/

We are drowning in information while starving for wisdom | Realize Engineering

Tags: research, knowledge, ethics

This is indeed very much true… there’s a clear crisis in research. It turned into a hamster wheel of publishing articles at a constantly faster pace. The incentives are misguided which pushes that behavior to even have a career. Meanwhile, knowledge building suffers.

https://realizeengineering.blog/2021/01/20/we-are-drowning-in-information-while-starving-for-wisdom/

Photographer Fools the Internet With AI-Generated Cameras That Don’t Exist | PetaPixel

Tags: tech, ai, gpt, generator, fake

Don’t worry, so called AI isn’t going to take away your jobs. But do worry though, this marks the end of trusting any pictures or texts you see in the media. Everything needs to be challenged, even more so now.

https://petapixel.com/2022/12/15/photographer-fools-the-internet-with-ai-generated-cameras-that-dont-exist/

Cyberpunk Translate

Tags: tech, ai, neural-networks, translation

Interesting tool to for the automatic transcription and translation of videos using off the shelf components. Seems to work nicely.

https://github.com/elanmart/cbp-translate

Copilot Internals | Hacky repo to see what the Copilot extension sends to the server

Tags: tech, github, copilot, ai, machine-learning

Interesting reverse engineering job of Copilot’s client side to have a better idea at which information it actually feeds to the model. A couple of funny tricks to prepare the prompt are involved. Obviously some telemetry involved as well, again with interesting heuristics to try to figure out if the user kept the suggestion or not.

https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals.html

A not so unfortunate sharp edge in Pipenv :: dade

Tags: tech, python, supply-chain

This is indeed a problem in the way Pipenv handle this kind of cases. Makes for bisecting clearly troublesome.

https://0xda.de/blog/2022/12/a-not-so-unfortunate-sharp-edge-in-pipenv/

An Ode to Unit Tests: In Defense of the Testing Pyramid

Tags: tech, programming, tests, tdd, architecture, craftsmanship

Good piece. I like how it frames the debate, asking the right questions on where might be the assumptions on how testing is done.

https://www.infoq.com/articles/unit-tests-testing-pyramid/

The Cargo Cult of Good Code

Tags: tech, programming, craftsmanship

Good points, this shows quite effectively the problem with blindly following rules without keeping in mind the reasoning behind them.

https://pboyd.io/posts/cargo-cult-of-good-code/

Taming Names in Software Development - Simple Thread

Tags: tech, programming, craftsmanship

Excellent piece about naming things in code. The conclusion is perfect of course: it’s hard but it pays off.

https://www.simplethread.com/taming-names-in-software-development/

The Story of A - by Kent Beck

Tags: tech, architecture, refactoring, craftsmanship, xp

A good set of skills to develop to bring technical changes to a project. It doesn’t need to be overwhelming or a fight. If you doubt, if you have empathy and if you slice the proposals it can become doable and help the people around you.

https://tidyfirst.substack.com/p/the-story-of-a

The topologist’s world map – tafc.space

Tags: tech, map, art

This is really cool work. Very insightful and pretty to look at.

https://tafc.space/qna/the-topologists-world-map/

Bye for now! And Merry Christmas!

Hopefully by now, you know that in KDE we are running an End of Year Fundraising campaign.

If you didn't, now you know :)

The campaign has already raised around 16 thousand euros, but there's still a bit to go to the minimum goal of 20 thousand.

So let's spice things up a little, I will donate 10% of every donation you make, you donate 1€, I will donate 0.1€, you donate 100€ I will donate 10€, etc.

I'm placing my maximum total donation amount at (20000-16211.98)/11 = 344.37€, that is if you all donate 3443.7€ (or more), I will donate 344.37€ and we'll reach the 20K goal.

How is this going to work? 

I will make my donation on the 31st of December (just one donation, to save up on fees).

For your donation to be included in my matching donation you need to send me an email to aacid@kde.org with your name/email and put in copy (CC) the KDE e.V. board kde-ev-board@kde.org so they can confirm your donation.

Only donations between now (23rd of December around 18:00 CET) and 31st of December will be considered.

I’m happy to announce the release of Tokodon 22.11.2 (and 22.11.1 who I released earlier this month and forgot to properly announce). These releases contain mostly bug fixes but also some welcome interface improvements.

First this adds an account switcher (similar to the one Tobias Fella implemented in NeoChat). Very usefully when you need to manage multiple accounts and want to quickly switch between them.

This also change the image preview from appearing in a separate full screen window to be contained inside the window. This follow the similar change from from James Graham in NeoChat.

Joshua Goins improved the loading of media attachment and made it possible to now hide sensitive image by default using the blurhash effect. This is also using the already existing implementation of blurhash from Tobias in NeoChat and you might start to see a pattern in this release. ;)

Finally I added support for custom emojis in many places inside the UI. Perfect if you want to show you true verified checkmark in your profile :)

Aside from the nice new improvements, I improved the spacing in the app and while not perfect yet, I hope this makes Tokodon more enjoyable to use. Joshua Goins has also made various improvements to our internal networking code and this should offer better reliability and less prone to crash code. And I fixed an important crash on start-up that was affecting a lot of users

Finally I started adding unit tests in Tokodon and added the infrastructure to mock a Mastodon server. We now have reached 12% unit tests coverage and I hope this number will grow after each release.

Packager section

You can find the package on download.kde.org and it has been signed with my GPG key.

Let’s go for my web review for the week 2022-50.

Statement on EU Comission adequacy decision on US

Tags: tech, surveillance, politics

The EU Commission clearly doesn’t have the best interest of the people at heart there. Let’s see what it’ll give with the CJEU.

https://noyb.eu/en/statement-eu-comission-adequacy-decision-us

Twitter is blocking links to Mastodon - The Verge

Tags: tech, twitter, social-media, fediverse, censorship

Slippery slope indeed… This is really becoming pathetic.

https://www.theverge.com/2022/12/15/23512113/twitter-blocking-mastodon-links-elon-musk-elonjet

PeerTube v5: the result of 5 years’ handcrafting – Framablog

Tags: tech, peertube, fediverse

You like Peertube? I do love this project. And it’s still mostly a single developer behind it. This needs to be supported.

https://framablog.org/2022/12/13/peertube-v5-the-result-of-5-years-handcrafting/

How to rebuild social media on top of RSS

Tags: tech, social-media, rss

Interesting ideas for a proper RSS renaissance.

https://tfos.co/p/rebuild-social-media/

The problem with overestimating AI — pnw.ai

Tags: tech, ai, machine-learning, gpt, surveillance

A few months old but a good piece to put things in perspective after the recent craze around large language models in general and GPT in particular. Noteworthy is the “wishful mnemonics” phrase mentioned and how it impacts the debate. Let’s have less talks about AIs and more about SALAMIs please?

https://pnw.ai/article/the-problem-with-overestimating-ai/121722775

Laurence Tratt: How Might Generative AI Change Programming?

Tags: tech, ai, machine-learning, programming, gpt

At least a good balanced post about Generative AI and programming. It’s not overestimating abilities of the latest trend in large language models and moves away from the “I’ll loose my job, developers will be replaced with AI” stance.

https://tratt.net/laurie/blog/2022/how_might_generative_ai_change_programming.html

The GPT-3 Architecture, on a Napkin

Tags: tech, ai, machine-learning, gpt

Nice article, gives a few clues to get a grasp on how GPT-3 works.

https://dugas.ch/artificial_curiosity/GPT_architecture.html

Faster hardware is a bad first solution to slow software

Tags: tech, performance, optimization

Don’t bank it all on faster hardware, make sure your software isn’t slow first. Otherwise it’ll bring quite some hidden costs.

https://pythonspeed.com/articles/fixing-performance-with-hardware

mold: A Modern Linker 🦠

Tags: tech, linking

Interesting new linker, should shorten build time quite a bit for large projects.

https://github.com/rui314/mold

Auto-vectorization: How to get beaten by compiler optimization — Java JIT!

Tags: tech, java, optimization, performance

Don’t underestimate performance of the generated code when a JIT is in the picture. Very good example with the JVM just there.

https://itnext.io/auto-vectorization-how-to-get-beaten-by-compiler-optimization-java-jit-vector-api-92c72b97fba3

Challenging the status quo at work?

Tags: work, culture

Wise advices on how to approach disagreements. Obviously requires psychological safety at work though. The culture will matter.

https://workweek.com/2022/11/28/challenging-the-status-quo-at-work/

Kind Engineering: How To Engineer Kindness

Tags: tech, management, team, codereview, kindness, feedback

Interesting article. I especially like how it makes the difference between being kind and nice. That honesty is required if you want to be really kind to others. It nicely shows examples on how to apply this (for instance, but not only, in the context of code reviews).

https://kind.engineering/

Ten Minute Physics

Tags: tech, physics, simulation, 3d

Nice resources to build physically based simulations. Quite a few topics to explore.

https://matthias-research.github.io/pages/tenMinutePhysics/index.html

Bye for now!

Ah, working with more than one screen, nice and convenient, but always has been the scourge of usability, due to broken hardware and software, on every platform you can imagine. And yes, on Plasma we are notorious to have several problems with regard to multi screens, which spurred many bug reports.

TL;DR

For Plasma 5.27 we did a big refactor on how the screens are managed (and how they are mapped to desktops and panels) which hopefully should solve many of those issues, creating a much more predictable experience, both for “fixed” multi screen setups (like a setup at your desk) and “on the go” (like attaching a projector at a conference).

• No more default empty desktops after connecting a screen that you already connected in the past
• No more desktops, wallpapers, widgets and panels lost after restart
• No more different sets of desktops between X11 and Wayland
• Better experience when using USB-C based docks

UI wise on the Systemsettings module nothing will change in the most common case, when you have one or two screens, where you will be presented with the usual “primary” checkbox.

When you have 3 or more screens, instead of s simple “Primary” checkbox, you will be presented with a dialog where you can rearrange the screens in a logical order, so that you can say: “This is my screen number one, this is my screen number two etc.”

This will map exactly to what “screen number one, two etc” are for the Plasma shell, so that whatever desktops, wallpapers, widgets and panels are on the “screen number one” will always be on the “screen number one”, never disappearing again, same for each screen, giving a completely predictable, and stable multiscreen setup.

For 5.27 I worked mostly on the Plasmashell part, but many thanks and shouts to Ivan for his work on KScreen and to Xaver for his work on the KWin and Wayland protocol parts.

Long story: the moving parts

I would like to do a semi-technical but still quite high level description on how our multiscreen infrastructure works, and what did we change.

When you have multiple monitors connected, assuming the kernel and drivers will do the right thing (which… doesn’t always happen), either X11 or Wayland (so in our case the KWin Wayland compositor) will see the screens and have info about them we can enumerate and manage.

KScreen

In Plasma we have a daemon/library to manage the screens, called KScreen. It will save a configuration for each combination of screens it encountered. each screen is identified uniquely by its EDID, and when this fails for the above reasons, the connector name the screen is connected to (under X or libdrm they will have names like HDMI-A-1, DP-2, eDp1 and so forth). connector names are also not optimal, but is the best that can be done in case of a bad monitor (For 5.27 Xaver fixed a quite big issue on this fallback path which likely caused several of the reported bugs).

The user then can chose with the Systemsetting module how the screens are arranged, their resolutions, which is the primary and what not. When a screen is connected or disconnected, KScreen will search for an existing configuration for this set of screens and restore that one.

Plasma

The Plasma shell also has to be aware of multiple screens arrangement, because you want the desktops with their wallpapers, widgets and icons to be on the screen you expect it to, as well all the panels to be always on the “proper” screen you expect it to. And that’s where we had a lot of problems in the past, and various approaches have been attempted.

Current

Currently, up to 5.26 Plasma relied on two concepts for working out which screen a desktop and panel should be. Combining a single primary monitor and using screen connectors names for disambiguate second and third screens. In theory …

There are a couple of things that can go wrong with this: between X11 and Wayland, connector names tend to be different (for instance HDMI-1 vs HDMI-A-1 on my system).

Another problem is related to the the USB-C based docks that have video outputs (such as the Steamdeck one, but is common to any other model and brand). They dynamically create and delete the outputs and connectors when they are plugged and unplugged, but the connector name is not stable between different plugs/unplugs: sometimes after re-plugging the old names are not recycled, but increased (so for instance if those were DP-2 and DP-3, they may become all the sudden DP-4 and DP-5). Since we used to store those names, we have a problem: each time a new connector name is encountered is treated as a completely new thing, giving its new empty default desktop and no panels, causing many wtf moments.

Plasma Containments (technical name of the desktops and panels) always had a screen property associated to them which is just an integer: 0 for the primary screen and then 1,2,etc for the others.

5.27

The new approach builds on a single design, extending the primary monitor concept to multiple monitors, which cleans up the codepaths even for the two monitor case.

So we now dropped this association map between screen number integers and connector names, and we go with this ordered list all the way down, so now Plasma, KWin and KScreen agree exactly who “Screen 1” is.

We have a new Wayland protocol for screen ordering (as well an X11 counterpart) for KScreen and KWin to agree to which one is the real logical ordering of the screens, so Plasmashell will now follow this without attempting to identify monitors by itself, removing a very big failure point.

As said, When your system has only 2 screens, in the Systemsettings module you’ll see exactly the UI it had: just a checkbox to select which one is the “Primary”, so no surprises here. When you have 3 screens or more then you are presented with a dialog where you can visually reorder the screens, to decide which is the first, which is the second etc.

In this way, you’ll always be sure that whichever screen was set as “first” will have your “main” set of panels and desktop widgets, that you will never lose, also whichever you set as “second” will always have the desktop and panels you set it and never change and so on.

At KDE we have multiple levels of quality assurance ranging from various degrees of a humans testing features to fully automated testing. Indeed automated testing is incredibly important for the continued quality of our software. A big corner stone of our testing strategy are so called unit tests, they test a specific piece of our software for its behavior in isolation. But for many aspects of our software we need a much higher level view, testing pieces of Plasma’s application launcher in isolation is all good and well but that won’t tell us if the entire UI can be easily navigated using the keyboard. For this type of test we require a different testing approach altogether. A couple months ago I’ve set set out to create a testing framework for this use case and I’m glad to say that it has matured enough to be used for writing tests. I’d like to walk you through the technical building blocks and a simple example.

Let us start of by looking at the architecture at large. So… there’s Selenium which is an incredibly popular, albeit web-oriented, testing framework. Its main advantages for us are its popularity and that it sports a server-client split. This means we can leverage the existing client tooling available for Selenium without having to write anything ourselves, we only need to grow a server. The server component, called a WebDriver, implements the actual interaction with UI elements and is generic enough to also apply to desktop applications. Indeed so thought others as well: there already exists Appium - it extends Selenium with more app-specific features and behaviors. Something for us to build upon. The clients meanwhile are completely separate and talk to the WebDriver over a well defined JSON REST protocol, meaning we can reuse the existing clients without having to write anything ourselves. They are available in a multitude of programming languages, and who knows maybe we’ll eventually get one for writing Selenium tests in QML ;)

That of course doesn’t explain how GUI testing can work with this on Linux. Enter: AT-SPI. AT-SPI is an accessibility API and pretty much the standard accessibility system for use on Linux. Obviously its primary use is assistive technologies, like the screen reader Orca, but to do its job it essentially offers a toolkit-independent way of introspecting and interacting with GUI applications. This then gives us a way to implement a WebDriver without caring about the toolkit or app specifics. As long as the app supports AT-SPI, which all Qt apps do implicitly, we can test it.

Since all the client tooling is independent of the server all we needed to get GUI testing going was a WebDriver that talks to AT-SPI.

That is what I set out to write and I’m happy to announce that we now have an AT-SPI based WebDriver, and the first tests are popping into existence already. There is also lovely documentation to hold onto.

So, without further ado. Let us write a simple test. Since the documentation already writes one in Python I’ll use Ruby this time around so we have some examples of different languages. A simple candidate is KInfoCenter. We can test its search functionality with a couple of lines of code.

First we need to install selenium-webdriver-at-spi, clone it, cmake build it, and cmake install it. You’ll also need to install the relevant client libraries. For ruby that’s simply running gem install appium_lib.

Then we can start with writing our test. We will need some boilerplate setup logic. This is more or less the same for every test. For more details on the driver setup you may also check the wiki page.

  def setup
    @appium\_driver = Appium::Driver.new(
      {
        'caps' => { app: 'org.kde.kinfocenter.desktop' },
        'appium\_lib' => {
          server\_url: 'http://127.0.0.1:4723',
          wait\_timeout: 10,
          wait\_interval: 0.5
        }
      }, true
    )
    @driver = @appium\_driver.start\_driver
  end

The driver will take care of starting the correct application and make sure that it is actually running correctly. Next we’ll write the actual test. Let’s test the search. The first order of business is using a tool called Accerciser to inspect the AT-SPI presentation of the application. For more information on how to use this tool please refer to the wiki. Using Accerciser I’ve located the search field and learned that it is called ‘Search’. So, let’s locate it and activate it, search for the CPU module:

  def test\_search
    search = driver.find\_element(:name, 'Search')
    search.click
    search.send\_keys('cpu')

Next let us find the CPU list item and activate it:

    cpu = driver.find\_element(:class\_name, '\[list item | CPU\]')
    assert(cpu.displayed?)
    cpu.click

And finally let’s assert that the page was actually activated:

    cpu\_tab = driver.find\_element(:class\_name, '\[page tab | CPU\]')
    assert(cpu\_tab.displayed?)

To run the complete test we can use the run wrapper: selenium-webdriver-at-spi-run ./kinfocentertest.rb (mind that it needs to be +x). If all has gone well we should get a successful test.

Finished in 1.345276s, 0.7433 runs/s, 1.4867 assertions/s.

1 runs, 2 assertions, 0 failures, 0 errors, 0 skips
I, \[2022-12-14T13:13:53.508516 #154338\]  INFO -- : tests done
I, \[2022-12-14T13:13:53.508583 #154338\]  INFO -- : run.rb exiting true

This should get you started with writing a test for your application! I’ll gladly help and review your forthcoming tests. For more detailed documentation check out the writing-tests wiki page as well as the appium command reference.

Of course the work is not done. selenium-webdriver-at-spi is very much still a work in progress and I’d be glad for others to help add features as they become needed. The gitlab project is the place for that. <3

The complete code of the example above:

#!/usr/bin/env ruby
# frozen\_string\_literal: true

# SPDX-License-Identifier: GPL-2.0-only OR GPL-3.0-only OR LicenseRef-KDE-Accepted-GPL
# SPDX-FileCopyrightText: 2022 Harald Sitter <sitter@kde.org>

require 'appium\_lib'
require 'minitest/autorun'

class TestKInfoCenter < Minitest::Test
  attr\_reader :driver

  def setup
    @appium\_driver = Appium::Driver.new(
      {
        'caps' => { app: 'org.kde.kinfocenter.desktop' },
        'appium\_lib' => {
          server\_url: 'http://127.0.0.1:4723',
          wait\_timeout: 10,
          wait\_interval: 0.5
        }
      }, true
    )
    @driver = @appium\_driver.start\_driver
  end

  def teardown
    driver.quit
  end

  def test\_search
    search = driver.find\_element(:name, 'Search')
    search.click
    search.send\_keys('cpu')

    cpu = driver.find\_element(:class\_name, '\[list item | CPU\]')
    assert(cpu.displayed?)
    cpu.click

    cpu\_tab = driver.find\_element(:class\_name, '\[page tab | CPU\]')
    assert(cpu\_tab.displayed?)
  end
end

It’s a Plasma widget that visualizes what’s going on on your system, music-wise that is. I’ve started this project years ago but only recently found the motivation to get it to a somewhat acceptable state. It’s pretty amazing to have bars flying across the screen to Daft Punk’s `Touch`.

https://store.kde.org/p/1953779