Wednesday, June 7, 2023

Mojo is a much better "Objective-C without the C" than Swift ever was

One of the primary things that people don't understand about Objective-C is that it is a solution of the two language problem, or more precisely a generalisation of the two language problem to the scripted component pattern.

The scripted component pattern itself is a (common) solution to the problem, first identified in the 70s that programming-in-the-large is not the same as programming-in-the-small, that module implementation languages are not necessarily suitable as module interconnection languages.

And so we have all sorts of flexible connection languages, often interpreted (aka glue, scripting, and orchestration languages), starting with the Unix shell, in addition to fast, compiled component languages such as C, C++ and Rust, and a system will usually incorporate at least one of each kind.

But then you run into the two language problem: you have to deal with these two distinct languages, with how they integrate, and with the boundaries of the integration often not matching up very well with the boundaries of the problem you're trying to solve.

Objective-C solved the two language problem by just jamming the two languages into one: Smalltalk for the scripting/integration and C for the component language. Interoperability is smooth and at the statement level, thougha there is some friction due to overlaps caused by integrating two existing languages that were not designed to be integrated.

Mojo essentially uses the Objective-C approach of jamming the two languages into one. Except it doesn't repeat Objective-C's mistake of using the component language as the base (which, inexplicably, Swift didn't just repeat, but actually doubled down on by largely deprecating objects). The reason this is a mistake is that it turns out that the connection language is actually the more general one, the component language is a specialisation of the connection language.

With this realisation, Mojo's approach of making the connection language the base language make sense. In addition, the fact that the component language is a specialisation also means that you don't actually need to jam a full second language into your base, a few syntactic markers to to indicate the specialisations are sufficient.

This is pretty much exactly stage 2 of the 4 stages of Objective-S, so I think they are using exactly the right approach for this. Except of course for the use of Python as the base instead of Smalltalk, which is a pragmatic choice given what they are trying to accomplish, but means your connection language is unduly limited.

Objective-S has the same basic structure, but with a much more capable connection language as the base.

Monday, May 8, 2023

Setting up Hetzner ARM instances with and for Objective-S

The recent introduction of reasonably-priced ARM64 VPS instances by Hetzner was accompanied by a big smile and sigh of relief on my part, as I had previously made the decision to prioritize ARM with Objective-S, for example the native-compiler is currently ARM64-only, but the simple and low-cost VPS providers like Digital Ocean were sticking to x86 exclusively.

Although it is possible to operate in a mixed ARM/x86 environment, the added complexity is not something I want as a default, which is why I also switched the hosting of the Objective-S site from DO to the Oracle cloud (on their "free forever" tier), as it was the only way to host on ARM without incurring monthly charges upwards of $40. With a number of alternatives spanning the spectrum, I now felt it

I've long had a strong hunch that there is both room and a strong need for something between the "we'll just hack together a few simple shell scripts" of the (very good!) Deployment from Scratch and the aircraft carrier that is Kubernetes.

With the external pieces finally in place, it's time to follow that hunch, and what better way than to control the Hetzner server API using Objective-S?

Talking to the API

Perusing the documentation, we see that the base URL for talking to the API is So let's set up an API scheme handler for talking to the Hetzner API, and also set up the authentication header and indicate that we will be using JSON:

scheme:https setHeaders: #{ 
    #Content-Type: 'application/json'
    #Authorization: "Bearer {keychain:password/hetzner-api/metaobject}",
scheme:api := ref: asScheme.

It's not a lot of code, but there is quite a bit going on: first, the token is stored in the macOS keychain, accessed via keychain:password/hetzner-api/metaobject. This is interpolated into the Bearer string inside a dictionary literal. The api: scheme is now available for talking to the Hetzner API, so for example api:servers will be sent as

That setup now allows us to define a simple class that allows us to interact with the API:

class HetznerCloud {
   var api.
   -schemeNames { [ 'api' ]. }
   -images {
   -types {

It currently has two user-facing methods: -images, which lists the kinds of images that are available and -types, which lists the server types. The method bodies may appear to be a little short, but that really is all that's needed. The -schameNames method makes the api: scheme handler available within method bodies of this class.

Below is an excerpt of an interactive st-shell session first asking the API for image types and then for server types:

] cloud images
{ "images" = ( { "id" = 3;
"description" = "CentOS 7";
"created_from" = ;
"bound_to" = ;
"rapid_deploy" = true;
"deprecated" = ;
"os_flavor" = "centos";
"type" = "system";
"protection" = { "delete" = false;
} ;
"image_size" = ;
"labels" = { } ;
"deleted" = ;
"architecture" = "x86";
"created" = "2018-01-15T11:34:45+00:00";
"os_version" = "7";
"disk_size" = 5;
"status" = "available";
] cloud types
{ "memory" = 4;
"prices" = ( { "price_monthly" = { "net" = "3.2900000000";
"gross" = "3.9151000000000000";
} ;
} ;
} ) ;
"storage_type" = "local";
"id" = 45;
"cpu_type" = "shared";
"disk" = 40;
"deprecated" = ;
"architecture" = "arm";
"description" = "CAX11";
"name" = "cax11";
"cores" = 2;

The "CAX11" instance type is the entry-level ARM64 instance that we want to use.

Creating a server

Creating a VPS is accomplished by POSTing a dictionary describing the desired properties of the server to the servers endpoint:
extension HetznerCloud {
   -baseDefinition {
	    #location: 'fsn1',
	    #public_net: #{
                #enable_ipv4: true,
                #enable_ipv6: false,
   -armServerDefinition {
           #name:  'objst-2',
           #image: '103908070',
           #ssh_keys: ['marcel@naraht.local' ],
           #server_type: 'cax11',
	} , self baseDefinition.
   -create {
	  ref:api:servers post: self armServerDefinition  asJSON.

The -create sends the post: message directly to the reference of the endpoint.

Interacting with servers

Once we have a server, we probably want to interact with it in some way, at the very least to be able to delete it again. Although we could do this using methods of the cloud API taking an extra server_id parameter, it is nicer to create a separate server abstraction that lets us interact with the server and encapsulates the necessary information.

The HetznerHost is initialized with a server response from which it uses the ip address and the server id, the latter to define a server: scheme handler. The fact that it's a subclass of MPWRemoteHost will become relevant later.

class HetznerHost : MPWRemoteHost {
   var hostDict.
   var id.
   var server.

   +withDictionary:theServer {
	self alloc initWithDictionary:theServer.
   -initWithDictionary:theServer {
       self := super initWithName:(theServer at:'public_net' | at:'ipv4' | at:'ip') user:'root'.
       self setHostDict:theServer.
       self setId: theServer['id'].
       self setServer: ref:api:/servers/{this:id} asScheme.

     -schemeNames { ['server']. }
     -status { this:hostDict at:'status'. }
     -delete {
         ref:server:/ delete.


The DELETE is handled similarly to the POST above, by sending a delete message to the root reference of the server: scheme.

We get server instances with a GET from the API's servers endpoint, the same one we POSTed to create the server. The collect HOM makes it straightforward to map from the dictionaries returned by the APU to actual server objects:

extension HetznerCloud {
   -servers {
	HetznerHost collect withDictionary: (api:servers at:'servers') each.

At this point, you're probably thinking that having a class representing servers, with its own scheme-handler to boot, is a bit of overkill if all we are going to do is send a DELETE. And you'd be right, so here are some of the other capabilities:
extension HetznerHost {
     -actions { api:servers/{this:id}/actions value.  }
     -liveStatus { server:status. }
     -refresh {
         self setHostDict: (server:/ value at:'server').
     -shutdown {
         ref:server:actions/shutdown post:#{}.
     -start {
         ref:server:actions/poweron post:#{}.
     -reinstall:osName {
         ref:server:actions/rebuild post: #{ #image: osName }.
     -reinstall {
         self reinstall:'ubuntu-20.04'.

With this, we have complete lifecycle control over the server, with a surprisingly small amount of surprisingly straightforward code, thanks to Objective-S abstractions such as Polymorphic Identifiers, Storage Combinators and Higher Order Messaging.

What's more, this control is available both immediately in script form, as well as for reuse in other applications as objects.

Installing Objective-S

Now that we can create, start, stop and destroy virtual servers, it would be nice to actually do something with them. For example: run Objective-S and Objective-S-based web-servers.

This is where the MPWRemoteHost comes in. This is what it says on the tin: a representation of a remote host, very rudimentary for now. One of the few things it knows how to do is set up an ssh connection to that remote host to execute commands and transfer files via SFTP. The latter is surfaced as a store, so you can create files on a remote host as easily as assigning to a local variable:

dest:hello.txt := 'Hello world!'.

Copying files is similar:
dest:hello.txt := file:hello.txt.

The script copies a tar archive containing both GNUstep and the Objective-S libraries, which it then untars into the '/usr' directory of the target machine. In addition it transfers the interactive Objective-S shell st, the runsite command that serves ".sited" bundles via HTTP, and a .bashrc that sets up some needed environment variables.
extension MPWHost { 
 -installObjS {
	scheme:dest := self store.
	filenames := [ 'ObjS-GNUstep-installed.tgz', 'st', '.bashrc', 'runsite' ].
	filenames do: { :filename | 
	     dest:{filename} := file:{filename}.
	self run:'chmod a+x st runsite';
	     run:'cd /usr ; tar zxf ~/ObjS-GNUstep-installed.tgz';
	     run:'mv st /usr/local/bin';
	     run:'mv runsite /usr/local/bin'.
host := MPWHost host:hostip user:'root'.
host installObjS.

As this is an extension to MPWHost, which is the superclass of the MPWRemoteHost we used as the base for our HetznerHost, the server objects we use have the ability to install Objective-S on them. Neat.

And so do the server objects for the very similar script controlling DO droplets.


When I started out on this little excursion, my goal was not to demonstrate anything about Objective-S, I only needed to be able to use these cloud systems, and my hunch was that Objective-S would be good for the task.

It turned out even better than my hunch had suggested: the various features and characteristics of Objective-S, such as Polymorphic Identifiers, first class references, nested scheme handlers, and Higher Order Messaging, really work together quite seamlessly to allow interaction with both a REST API and with a remote host to be expressed compactly and naturally. In addition, it manages to naturally bridge the gap between ad-hoc scripting and proper modelling, remaining hackable without creating a mess.

It's working...

Friday, January 13, 2023

Setting the Bozo Bit on Apple

The other day I was fighting once again with Apple Music. Not the service, the app. What I wanted to do was simple: I have some practice recordings for my choir and voice lessons that I want on my iPhone and Apple Watch. How hard could it be?

Apple: hold my beer.

These are sent via WhatsApp so the audio recordings are mp4 files, which for some bizarre reason won't open in Music and instead open in QuickTime Player, despite definitely being audio files.

OK, not a biggie, so export to m4a from QT Player. Transfer to the machine that has my audio library. Create a new playlist, transfer some previous songs over, then try to drop the new m4a's onto the open playlist. No go. Play around for a while, figure out that the entity that accepts the drops is the TableView, not the surrounding view. So you can't drop the new files into the empty space below the songs, you have to drop them onto the existing songs.

Who programmed this? Who didn't pay attention to this when doing QA? Who approved it for release? iTunes used to be if not the, then certainly a flagship app for Apple.

OK, plug in the iPhone, as for some reason wireless transfers don't seem to be overly reliable.

No Finder, I don't want to back...too late. Ok, do your backup. Waiting. Spinner. Waiting. Repeat. After a while it says it's finished. Unplug and ... the songs are not there.

I quit, relaunch it, and lo-and-behold, the songs are now no longer in the playlist in either. Re-add them, carefully aiming for the table, sync again (hey, it remembered we just did a backup and doesn't try again, kudos!), and now they show up.

Whew! Only took 15 minutes or so, the last time I was futzing with it for over an hour and the songs never synced. Or one did and two did not, which is obviously Much Better.

How can such basic functionality be this incredibly broken? And of course this is just one tiny example, there are legions others, as many others have reported.

With this, I noticed that I hadn't actually expected better. I knew it should be better but I hadn't expected Apple to actually make it work.

In other words, I had set the Bozo Bit on Apple. By default, when Apple does something new these days, I fully and quietly expect it to be broken. And I am surprised when they actually get something right, like Apple Silicon. And it wasn't an angry reaction to anything, in fact, it wasn't even much of conscious decision, more a gradual erosion of expectations.

It Just Doesn't Work™.

And that's sad.