Okteta got “Best Application” 2024 Akademy Award

The jury of this year’s KDE Akademy Awards, being by tradition representatives of last year’s winners, has selected the hex editor Okteta in the category “Best Application”. Thanks to them for this appreciation, even more for a niche application

Though, appreciation for what, as there are no details? The last new feature was added in 2019, with the 17th patch release since just done. So, for a reliable program with no need to relearn the UI every year and proudly close to zero open actual bug reports? Then the port to Qt6/KF6, while started in 2022, might be only completed in 2025… if ever. So rather, is this an end-of-life award for an aged 16 years old program?

Looking Back

Triggered by the event some reflection on the past development, if only for the author himself, to update the memories how one got here and what it brings for the future. Which turned into a longer text than anticipated

2003-2006: Years of Initial Need for a Widget

The Okteta project was born in 2003 , the known first code traces date back to May 13th, 2003. The first related code was imported into KDE’s code repository in August 15th, 2003, by the commit message:

Initial import of KHexEdit2, featuring a widget, a kpart and an app.
Most important is the widget...
Hopefully it will be usable enough ready for KDE 3.2...

The name “KHexEdit2” was chosen as the project was a re-implementation of KHexEdit, the hex editor part of KDE since KDE 1.1. Because those times I was trying to create a viewer for executables and libraries (project name Binspekt, stalled soon), for which I wanted a widget for displaying bytes. KHexEdit seemed to have no code that could be cleanly ripped out and be reused, so work started to code such a widget from scratch and respectively also consumers of it, to add more reason & motivation.

The formal request on September 29th that year to then move the project’s code from the “kdenonbeta” area into release-covered areas had this optimistic line in it:

Finally there will be an app, build around the ReadWritePart. In 2004.

Turned out, life did not agree to that plan, thus 2004 passed without any such app. And so did 2005.

Still, back in February 2004 as part of KDE 3.2 the first elements of the still named KHexEdit2 project made a first release. Though with a bit of complexity. ((Note that at this time KDE was still also the name of the released bundle product, composed of the so called modules kdelibs, kdebase, kdeutils, etc. Where kdelibs held all the public libraries, kdebase the basic desktop components, and so on.)) As adding a complete implementation of a hexeditor widget to the official kdelibs for just a few potential users was declared unbalanced, instead just some header files with KHE namespaced abstract interface classes were added, with an inline utility method to dynamically load any plugin implementing them. So not increasing the actual runtime and installation size of kdelibs. And the kdeutils module got to provide such a plugin by the name KBytesEdit. This then was implemented by the hex editor widget library from the KHexEdit2 project, also in the kdeutils module, whose own API and headers were kept private. To confuse everyone this library was yet named libkhexedit, even if the actual KHexEdit program did not use it. Because the spirit of the naming was on the level of widgets and classes, not program names, and there the 2 postfix made no sense. Consumers of this construct became at least the utility app for Palm Pilot handhelds KPilot and the debugger plugin of the IDE KDevelop.

In March 2005 KDE 3.4 then included the KHexEdit2 KPart (read-only) as well, also located in the kdeutils module and also implemented using the private libkhexedit library. Making some people unhappy as it registered (like its Okteta successor still does today) as handler for the MIME type application/octet-stream, so popping up as fallback KPart where no other handler was found,. And seeing raw bytes over nothing has been partially perceived as “broken”

2006-2008: Second Try on a Program, as Sample and with new Dedicated Name

2006 arrived, and with the author there was still some ever developing curiosity about the feasibility of writing viewer & editor programs using some higher-level reusable & exchangeable components. Now a byte array is a pretty simple data structure to use for a sample implementation of such a concept. And here we had a widget implementation for byte array viewing and editing fully under our control. And a certain level of skills with C++ acquired at the time. This was just too tempting to not give it an own try, for fun and experience. So a June blog post “Fun with KHexEdit” also mentioned a program again and introduced the name designed meanwhile:

I am tackling the construction of a successor to KHexEdit again, projectname “Okteta”.

Later that year a first visual snippet was shared, showing how KHexEdit’s UI served as initial template for Okteta, also to potentially help the transfer of existing users:

To avoid duplicated efforts and to increase pressure to deliver, two weeks later on November 27th an optimistic email was sent to KDE’s great eternal to-newer-API porting worker Laurent Montel, as it was the time to port KDE software to Qt4/KDELibs4:

Hi Laurent,

please don't spend too much effort at the old program KHexEdit, I am quite far
on the way to write a successor, called Okteta. Concerning feature
compatibility, so far I implemented around 60 % of the features of KHexEdit,
and hope to do the last 40 % until at least January. Yes, no code yet in SVN
(besides the library), but that will change in three weeks, promised.
[...]

This time life agreed mainly to the plan. Though the promise on the code in SVN was delivered on only with almost a year delay. A first dump of the program code was committed into KDE’s code repository on October 23rd, 2007, by the commit message:

Uploading the Okteta program into KDE's playground,
so the code isn't lost, after growing slowly only on my hdd for more than a year.

Okteta is a planned successor to KHexEdit, yet misses all of it's functionality.
With it's modular architecture, based on the co-developed lib kakao, it should
soon offer more than would could be done with the monolithic KHexEdit. I hope.

As can be read, this first copy also was featuring a first draft for the own before mentioned higher-level component system, initially named “Kakao”, later in 2009 to be renamed to “Kasten”. That first name was made ad-hoc inspired by a drink on the table (in learned safe distance from the keyboard), to soon find it used similarly by some certain bigger IT company, even for a somehow related subject, thus a new name was by the time designed less ad-hoc.

And so some months later in April 2008 Okteta entered the “kdereview” phase, to proceed after two weeks into the kdeutils module. In time for KDE 4.1, so premiering its release as part of that in July 2008. Okteta here also took over to provide the KBytesEdit plugin for the kdelibs KHE interfaces as well as the KPart, which before had resided in subdirectories of the KHexEdit program sources. KHexEdit itself stayed unported to Qt4/KDELibs4, so Okteta as planned did not run into duplicated efforts and rivalry (or, it avoided competition, for good and bad).

2008-2012: Years of Features Flow

With the foundations laid and releases established as part of KDE releases, the next years saw a number of features added, initially even each KDE version:

2010-Present: Sharing Functionality in Rich Public Libraries

From the very begin of the project on the byte array viewing & editing feature was embeddable by 3rd-party software using the abstract KHE interfaces in kdelibs, or then the KPart at least for viewing, Though this allowed only little control & features due to a limited API.

Starting with Okteta 0.4 in February 2010, the two sets of underlying libraries, Kasten and Okteta ones, used to implement the Okteta program, the Okteta KPart and the KHE KBytesEdit plugin, are provided with stable public API.

The lower-Qt-level Okteta GUI library also started to be accompanied by a Qt Designer plugin, to allow easy use of the two provided classes of widgets also in Qt’s UI files.

In February 2010 during a week-long Kate-KDevelop development meeting in Berlin the intended flexibility of the new public libraries proofed itself by enabling to create a plugin for KDevelop to integrate hex viewing & edting and all the Okteta tools in just those few days, for some nice satisfaction. The plugin was officially released first with KDevelop 4.1 in October 2010 and later also ported to the Qt5/KF5 version of KDevelop. For the current Qt6/KF6-based version of KDevelop the plugin is excluded from the build for now, given the current lack of a released Qt6/KF6-based version of the Okteta and Kasten libraries.

2012-Present: Switching from Features to Architecture, from Bundled to Stand-Alone

The port to Qt5/KF5 happened without any issues and was completed for version 0.15, released as part of KDE Applications 14.12. During the transformation of KDELibs4 to KDE Frameworks 5 the KHE interfaces also got dropped there, due to Okteta meanwhile directly providing public libraries. So this ported version of Okteta also no longer provided the KBytesEdit plugin, but otherwise as before the public libraries and the KPart, next to the program itself.

After KDE Applications 17.12, as for a while there was no feature development and only occasional work on the design of the Okteta & Kasten libraries happened, the Okteta project switched to a stand-alone release schedule. A 0.25 version branch was created and patch version releases only done when there were user-relevant changes like bug fixes or bigger translation improvements.

Then 2019 brought the first version and for now also latest to also provide at least one new feature to users, for which a 0.26 version branch was created. This version has meanwhile got 17 patch releases, with bug fixes, translation improvements and other adjustments. And after 5 years of such polishing is the one which now received the “Best Application” 2024 Akademy Award

2022-Present: Preparing for Qt6 & KF6

Okteta’s code base has been mostly quickly updated to any API deprecations, also as part of a “zero build warnings” strategy. So the approach taken for both Qt & KF libraries to strive for source-backward-compatible C++ API in their both new major version 6 made the initial port of Okteta to Qt6 & KF6(-Alpha) a matter of less than a day in May 2022. Now, only if one ignores one of the tools.

The Structures tool, first developed by Alexander Richardson in 2010 for Okteta 0.4, was in 2011 for Okteta 0.7 extended by him to also support dynamic structure definitions, using JavaScript expressions. For this QtScript has been used as engine. Four years later, in July 2015 though Qt 5.5 declared QtScript as deprecated. The officially recommended substitute QJSEngine turned out to not allow the dynamic translation of JavaScript properties and methods as relied on by the Structures tool for the copy-avoiding mapping of the data blob interpretation into the JavaScript scene (beware, for what the author understands so far). So it could not be used as drop-in replacement.

As finding a suitable and more future-proof JavaScript engine or exploring a possible reimplementation using the QJSEngine is a complex task and also needs bigger chunks of time & focussing, it had been postponed. Year after year. And thus now nine years later in 2024 there is still not even a plan. And Qt6 no longer now provides QtScript.

Just dropping the Structures tool is not a real option. It is a great feature, which also got some users. So a plan is needed and work to be done by someone. As of now my own, surely radical idea is to rewrite the whole Structures tool from scratch, still for the Qt5/KF5 version of Okteta. This should lead to fully wrapping the brain around this complex feature, instead of seeing to indirectly explore it by trying to understand all the details of the current elaborated implementation with the risk to misinterpret some intentions. Starting from scratch might also allow to finally share all the code used for the data formats in the separate Decoding Table tool, and perhaps even to introduce a more generic approach on the data formats supported in the main mass display besides currently byte values and 8-bit charset mapping. Idea, Should, Trying, Starting, Might, Perhaps… any words of confidence, please?

For now the initial Qt6/KF6 port is maintained by a single commit containing the complete dump of the “it builds, starts and does not crash on simple usage” changes, in a work branch continuously rebased to the latest current Qt5/KF5-based development branch. This commit would then at the time of the real Qt6/KF6 port be properly split into the different aspects of porting. For now it serves to hold the door open while still on the other side.

Looking Forward, by Looking Back Some More

For sure the initial goal with the Okteta program to do something for fun and experience has been largely achieved The current challenge with the needed replacement of the used JavaScript engine promises more experience, though no fun initially to me at least.

And some feedback as well even a KDE Akademy Award now hint the created and publicly shared program also served other people for their serious and less serious needs. Possibly even some desperate Faust-like persons, “So that they may perceive the bytes which hold their doc[ument] together in its inmost fold.” (and even tweak them for better as needed, owning their world or document). Though no contracts done, and thus no souls here owned.

But as before, Okteta actually is just a sample implementation of the actual interest pursued here, exploring the feasibility of writing programs by higher-level reusable & exchangeable components, ideally also allowing random end users to mesh those components themselves into tailored solutions for certain needs. So if development has stalled as it has, both on the components concept but also the hex editing features, how to increase motivation again to set resources aside, and for which part?

Position in the Hex Editor Solution Space

When it comes to the Free Software solution space for hex editor needs, next to Okteta there is currently coverage starting from simple ones like GHex over wxHexEditor, which serves needs beyond Okteta by support for paged loading of big files and also the working memory, though sadly unmaintained currently, to the newer yet most impressive and very powerful ImHex (by what the web pages show, never tried).

So would people suffer if Okteta is gone for current platforms, at least for a while?

Open Component Systems vs. Closed Monolithic Blobs

Now the author, while being curious, never got around to actually study existing solutions for the concept of higher-level component system or even deploy them in projects, only ever saw some theoretic surfaces. And is fully aware of the own experiment turning into something serious rather being a pipe dream. Even more when after soon two decades the initially created TODO list is not even 10 % done, this won’t work out this single human’ life So the following is more like the wish-wash of a hobbyist bird watcher, while also having some chicken in the backyard to which things are compared. Or alike.

It seems composable systems with complex interfaces are not the dominating species in the Free Software ecosystems. The Linux kernel outpaced any microkernel systems, e.g. Gnu Hurd is yet to be spotted outside OS zoos. The Eclipse Rich Client Platform, whose concepts were one of the original by-headlines inspirations for this project, seems to have maxed out some time ago as well. at least in the mainstream through the author’s bubble space. StarOffice^WOpenOffice^WLibreOffice has the UNO component system, but how many Add-Ons flourish on it? Then GnuStep would have enabled to spread the component concepts of OpenStep, but little has be seen? The later GnuStep-related, indeed thriving for stars impressive project Étoilé seemed to be overloaded with related ideas, but sadly never lift off. Then the GNOME project even had a reference to component systems in its initial full name “GNU Network Object Model Environment”, but its respective Bonobo framework based on CORBA faded away rather soon.

Also KDE started initially with implementations around the idea of components. To quote the KDE 1.0 announcement:

In view of these circumstances the KDE Project has developed a first rate compound document application framework, implementing the latest advances in framework technology and thus positioning itself in direct competition to such popular development frameworks as for example Mircrosoft’s MFC/COM/ActiveX technology. KDE’s KOM/OpenParts compound document technology enables developers to quickly create first rate applications implementing cutting edge technology. KDE’s KOM/OpenParts technology is build upon the open industry standards such as the object request broker standard CORBA 2.0.

KOM/OpenParts was then in KDE 2.0 replaced by KParts. Actually the presence of such technology development was the deciding factor to go for KDE when the author those days got into “Linux” and had to choose between GnuStep, Enlightenment, GNOME and KDE. These days though KDE is run with claims like “All About Apps”. The generic KServices system got destructed for KF6. The possibly latest KPart (a Markdown Viewer) was written years ago by this author, and the once KDE-central KPart-driven program Konqueror is only a shadow of its former self. KOffice & Calligra as component-oriented office suites also died or stalled close to extinction. Generic plugins like the KParts are not even listed on apps.kde.org or elsewhere anymore, also no longer mentioned as concept in KDE Gear release announcements. Similar specific plugins like the Plasma applets, they are also not listed separately, but only as part of the respective, in the example, Plasma products.

Additionally packaging formats like Flatpak or Snap are discussion-less embraced and promoted, which push in the direction of isolated and frozen software programs. Even today does Flatpak’s metadata system appstream. also otherwise discussion-less embraced by KDE, not have a concept of generic plugins, so KParts cannot be described properly.

In such an environment a component system would be limited to predefined fixed component sets in libraries, from which applications would then provide a setup and offer that to users. A bit like being able to shop as consumer at the kitchen equipment store only preset exhibition rooms, instead of meshing up items from different providers into ones’ own tailored meal preparations “app”. Surely it is in the interest of the dominating providers who then will see to bundle only their items, and then add bloat as well as only making half the items good. As consumer I desire to have the choice over pre-made bundles vs. self-assembled ones. Like there are times for All-inclusive vacations and times for self-organized ones. So with current KDE but also the larger current Free Software “desktop” scene as real world development environment working on and thinking about component systems has the author feel at odds.

So maybe the experiment with Kasten as a higher-level component system could also stop here. Perhaps some research instead could be done why such systems failed in comparison. Like, was it due to the inflexibility presented by fixed published interfaces, where on new feature needs implementations cannot simply do temporary shortcuts and adaptions where needed to be quick back to the market? Could it be due to the possible need for more abstract and generic thinking with component systems, where the majority of developers working for the market might prefer to think more concrete and case-by-case? Then, might there still be a middle-ground, where any advantages of high-level component systems are the deciding factor in the competition?

Next Release Scheduled: October 10th, version 0.26.18

As described already for the early stages, there always have been ideas and plans.. and delays… and also doubts… and then things happened. Locally there are lots of notes with ideas, and a number of code drafts and sketches stacked by the years. And at least in the near future it seems there are still time windows, electricity, a laptop, and enough human capabilities to carry on and tinkering over this stack.

The Okteta (& Kasten) project for now is alive, just lurking around in front of the next evolution step to take. Which might find it new ground. Or extinction anyway. And while it is lurking, it gets a tad more feathers polished, by another bug fix release already scheduled for next month