How Cursor is Rendered

Wayland has a unique way to let clients specify the contents of the cursor. After receiving a wl_pointer.enter event, the client must call wl_pointer.set_cursor request

    <request name="set_cursor">
      <arg name="serial" type="uint" summary="serial number of the enter event"/>
      <arg name="surface" type="object" interface="wl_surface" allow-null="true"
	   summary="pointer surface"/>
      <arg name="hotspot_x" type="int" summary="surface-local x coordinate"/>
      <arg name="hotspot_y" type="int" summary="surface-local y coordinate"/>
    </request>

The wl_pointer.set_cursor request takes an optional wl_surface object that represents the actual contents of the cursor. That’s it, the wl_surface interface is used both by windows and cursors! It opens a whole lot of features that you could use with the cursor, for example use the wp_viewport protocol for fractional scaling or create cursor surface trees using the wl_subcompositor interface. On the other hand, many compositors view the cursor as a simple image. So, let’s see how we improved kwin in this regard.

Cursor source

Currently (as of 5.26.x), kwin assumes that the cursor can show only a QImage. It’s okay for simple cases, but it falls apart once we need to show a wl_surface, e.g. we will hit problems with getting a QImage from a linux dmabuf client buffer.

So the first thing that we need to do in order to move anywhere forward is to choose proper abstractions to represent what is actually in the cursor. For example, if the cursor hovers a window, it obviously needs to present what the corresponding wl_surface contains. But sometimes the mouse cursor is not above any window, for example if the pointer is above a server-side decoration. Server-side decoration can be considered part of the window, but we cannot use client’s wl_surface anymore, the compositor may choose to show a different cursor, which is a QImage.

class KWIN_EXPORT CursorSource : public QObject
{
    Q_OBJECT

public:
    explicit CursorSource(QObject *parent = nullptr);

    QImage image() const;
    QSize size() const;
    QPoint hotspot() const;

Q_SIGNALS:
    void changed();
};

The CursorSource class is the base class for all other “source” classes that can be attached to the cursor. It contains generic properties, e.g. the hotspot, and it also contains the CursorSource::changed() signal to tell the world when the image has changed. CursorSource::image() exists for compatibility and to make the transition to new abstractions easier.

Sometimes, we need to show a static image in the cursor, so let’s add an ImageCursorSource to serve that purpose

class ImageCursorSource : public CursorSource
{
public:
    explicit ImageCursorSource(QObject *parent = nullptr);

public Q_SLOTS:
    void update(const QImage &image, const QPoint &hotspot);
};

On the other hand, some cursors are not static. For example, the loading cursor usually contains some animation, e.g. spinning wheel

class ShapeCursorSource : public CursorSource
{
public:
    explicit ShapeCursorSource(QObject *parent = nullptr);

    QByteArray shape() const;
    void setShape(const QByteArray &shape);
    void setShape(Qt::CursorShape shape);

    KXcursorTheme theme() const;
    void setTheme(const KXcursorTheme &theme);

private:
    void refresh();
    void selectNextSprite();
    void selectSprite(int index);

    KXcursorTheme m_theme;
    QByteArray m_shape;
    QVector<KXcursorSprite> m_sprites;
    QTimer m_delayTimer;
    int m_currentSprite = -1;
};

The ShapeCursorSource class represents a cursor shape from an Xcursor theme. It can be used to show both animated and static cursors. If the given cursor shape is animated, i.e. it has more than one sprite, ShapeCursorSource will start a timer with the timeout as indicated by the cursor theme. When the timer expires, ShapeCursorSource will switch to the next sprite and emit the CursorSource::changed() signal. If it’s the last sprite, it will wrap around to the first sprite.

And last but not least, we need something to represent wl_surface attached to the cursor

class SurfaceCursorSource : public CursorSource
{
public:
    explicit SurfaceCursorSource(QObject *parent = nullptr);

    KWaylandServer::SurfaceInterface *surface() const;

public Q_SLOTS:
    void update(KWaylandServer::SurfaceInterface *surface, const QPoint &hotspot);
};

ImageCursorSource, ShapeCursorSource, and SurfaceCursorSource are the main types that indicate what the cursor shows.

Scene

The CursorSource classes act as data sources, they don’t actually paint anything on the screen. It’s the responsibility of the scene. I’ve already written a little bit about the scene abstraction in kwin, I recommend you to read my earlier blog post about it https://blog.vladzahorodnii.com/2021/04/12/scene-items-in-kwin/

But as a quick recap: kwin breaks up a window in smaller building blocks called items. The DecorationItem corresponds to the server-side decoration if there’s one. The ShadowItem is a server-side drop shadow, for example a drop shadow cast by the decoration or the panel. The SurfaceItem represents the actual window contents. That’s the same strategy that we will follow here. The cursor will be broken in smaller pieces.

If we look closely at our cursor sources, the painting code should handle only two cases – paint a QImage and paint an arbitrary wl_surface tree. The wl_surface case is already taken care by SurfaceItem \o/, so we just need a new type of item to present an image in the scene graph, e.g. ImageItem

The CursorItem type ties both cases together. It monitors what source is attached to the cursor and creates either a SurfaceItem or an ImageItem according to the type of the cursor source

• If a SurfaceCursorSource is attached to the cursor, the CursorItem is going to destroy its child ImageItem (if there’s one) and create a SurfaceItem tree
• If an ImageCursorSource or a ShapeCursorSource is attached to the cursor, the CursorItem is going to destroy its child SurfaceItem (if there’s one) and create an ImageItem child item

Note that SurfaceItems are rendered the same way regardless of their role.

With all of this, kwin can easily handle simple cases such as displaying an image in the cursor and more esoteric things that you can do with a wl_surface.

Cursor layer

The last thing that’s needed to make cursor handling perfect is putting the cursor on its own hardware plane. Imagine that you move the mouse cursor. Ideally, you should not repaint any windows below the cursor, only update the position of the cursor. Unless the windows need to repaint their content because of new pointer position, e.g. add or remove button outlines, etc. It can be achieved by painting the cursor in its own plane.

KWin already attempts to put the cursor on hardware planes, but we would like to clean things up and unify cursor and overlay planes. It is still work in progress. TBA.

Summary

The new cursor design is more powerful and should make it easier to add new fancy features. For example, it would be amazing if you could wire the velocity of the cursor to its scale so you could shake the pointer in order to find the cursor more easily. The new design also fixes longstanding limitations that prevented kwin from displaying cursors rendered by OpenGL or Vulkan.