Monday, May 8, 2023

Setting up Hetzner ARM instances with and for Objective-S

The recent introduction of reasonably-priced ARM64 VPS instances by Hetzner was accompanied by a big smile and sigh of relief on my part, as I had previously made the decision to prioritize ARM with Objective-S, for example the native-compiler is currently ARM64-only, but the simple and low-cost VPS providers like Digital Ocean were sticking to x86 exclusively.

Although it is possible to operate in a mixed ARM/x86 environment, the added complexity is not something I want as a default, which is why I also switched the hosting of the Objective-S site from DO to the Oracle cloud (on their "free forever" tier), as it was the only way to host on ARM without incurring monthly charges upwards of $40. With a number of alternatives spanning the spectrum, I now felt it

I've long had a strong hunch that there is both room and a strong need for something between the "we'll just hack together a few simple shell scripts" of the (very good!) Deployment from Scratch and the aircraft carrier that is Kubernetes.

With the external pieces finally in place, it's time to follow that hunch, and what better way than to control the Hetzner server API using Objective-S?

Talking to the API

Perusing the documentation, we see that the base URL for talking to the API is So let's set up an API scheme handler for talking to the Hetzner API, and also set up the authentication header and indicate that we will be using JSON:

scheme:https setHeaders: #{ 
    #Content-Type: 'application/json'
    #Authorization: "Bearer {keychain:password/hetzner-api/metaobject}",
scheme:api := ref: asScheme.

It's not a lot of code, but there is quite a bit going on: first, the token is stored in the macOS keychain, accessed via keychain:password/hetzner-api/metaobject. This is interpolated into the Bearer string inside a dictionary literal. The api: scheme is now available for talking to the Hetzner API, so for example api:servers will be sent as

That setup now allows us to define a simple class that allows us to interact with the API:

class HetznerCloud {
   var api.
   -schemeNames { [ 'api' ]. }
   -images {
   -types {

It currently has two user-facing methods: -images, which lists the kinds of images that are available and -types, which lists the server types. The method bodies may appear to be a little short, but that really is all that's needed. The -schameNames method makes the api: scheme handler available within method bodies of this class.

Below is an excerpt of an interactive st-shell session first asking the API for image types and then for server types:

] cloud images
{ "images" = ( { "id" = 3;
"description" = "CentOS 7";
"created_from" = ;
"bound_to" = ;
"rapid_deploy" = true;
"deprecated" = ;
"os_flavor" = "centos";
"type" = "system";
"protection" = { "delete" = false;
} ;
"image_size" = ;
"labels" = { } ;
"deleted" = ;
"architecture" = "x86";
"created" = "2018-01-15T11:34:45+00:00";
"os_version" = "7";
"disk_size" = 5;
"status" = "available";
] cloud types
{ "memory" = 4;
"prices" = ( { "price_monthly" = { "net" = "3.2900000000";
"gross" = "3.9151000000000000";
} ;
} ;
} ) ;
"storage_type" = "local";
"id" = 45;
"cpu_type" = "shared";
"disk" = 40;
"deprecated" = ;
"architecture" = "arm";
"description" = "CAX11";
"name" = "cax11";
"cores" = 2;

The "CAX11" instance type is the entry-level ARM64 instance that we want to use.

Creating a server

Creating a VPS is accomplished by POSTing a dictionary describing the desired properties of the server to the servers endpoint:
extension HetznerCloud {
   -baseDefinition {
	    #location: 'fsn1',
	    #public_net: #{
                #enable_ipv4: true,
                #enable_ipv6: false,
   -armServerDefinition {
           #name:  'objst-2',
           #image: '103908070',
           #ssh_keys: ['marcel@naraht.local' ],
           #server_type: 'cax11',
	} , self baseDefinition.
   -create {
	  ref:api:servers post: self armServerDefinition  asJSON.

The -create sends the post: message directly to the reference of the endpoint.

Interacting with servers

Once we have a server, we probably want to interact with it in some way, at the very least to be able to delete it again. Although we could do this using methods of the cloud API taking an extra server_id parameter, it is nicer to create a separate server abstraction that lets us interact with the server and encapsulates the necessary information.

The HetznerHost is initialized with a server response from which it uses the ip address and the server id, the latter to define a server: scheme handler. The fact that it's a subclass of MPWRemoteHost will become relevant later.

class HetznerHost : MPWRemoteHost {
   var hostDict.
   var id.
   var server.

   +withDictionary:theServer {
	self alloc initWithDictionary:theServer.
   -initWithDictionary:theServer {
       self := super initWithName:(theServer at:'public_net' | at:'ipv4' | at:'ip') user:'root'.
       self setHostDict:theServer.
       self setId: theServer['id'].
       self setServer: ref:api:/servers/{this:id} asScheme.

     -schemeNames { ['server']. }
     -status { this:hostDict at:'status'. }
     -delete {
         ref:server:/ delete.


The DELETE is handled similarly to the POST above, by sending a delete message to the root reference of the server: scheme.

We get server instances with a GET from the API's servers endpoint, the same one we POSTed to create the server. The collect HOM makes it straightforward to map from the dictionaries returned by the APU to actual server objects:

extension HetznerCloud {
   -servers {
	HetznerHost collect withDictionary: (api:servers at:'servers') each.

At this point, you're probably thinking that having a class representing servers, with its own scheme-handler to boot, is a bit of overkill if all we are going to do is send a DELETE. And you'd be right, so here are some of the other capabilities:
extension HetznerHost {
     -actions { api:servers/{this:id}/actions value.  }
     -liveStatus { server:status. }
     -refresh {
         self setHostDict: (server:/ value at:'server').
     -shutdown {
         ref:server:actions/shutdown post:#{}.
     -start {
         ref:server:actions/poweron post:#{}.
     -reinstall:osName {
         ref:server:actions/rebuild post: #{ #image: osName }.
     -reinstall {
         self reinstall:'ubuntu-20.04'.

With this, we have complete lifecycle control over the server, with a surprisingly small amount of surprisingly straightforward code, thanks to Objective-S abstractions such as Polymorphic Identifiers, Storage Combinators and Higher Order Messaging.

What's more, this control is available both immediately in script form, as well as for reuse in other applications as objects.

Installing Objective-S

Now that we can create, start, stop and destroy virtual servers, it would be nice to actually do something with them. For example: run Objective-S and Objective-S-based web-servers.

This is where the MPWRemoteHost comes in. This is what it says on the tin: a representation of a remote host, very rudimentary for now. One of the few things it knows how to do is set up an ssh connection to that remote host to execute commands and transfer files via SFTP. The latter is surfaced as a store, so you can create files on a remote host as easily as assigning to a local variable:

dest:hello.txt := 'Hello world!'.

Copying files is similar:
dest:hello.txt := file:hello.txt.

The script copies a tar archive containing both GNUstep and the Objective-S libraries, which it then untars into the '/usr' directory of the target machine. In addition it transfers the interactive Objective-S shell st, the runsite command that serves ".sited" bundles via HTTP, and a .bashrc that sets up some needed environment variables.
extension MPWHost { 
 -installObjS {
	scheme:dest := self store.
	filenames := [ 'ObjS-GNUstep-installed.tgz', 'st', '.bashrc', 'runsite' ].
	filenames do: { :filename | 
	     dest:{filename} := file:{filename}.
	self run:'chmod a+x st runsite';
	     run:'cd /usr ; tar zxf ~/ObjS-GNUstep-installed.tgz';
	     run:'mv st /usr/local/bin';
	     run:'mv runsite /usr/local/bin'.
host := MPWHost host:hostip user:'root'.
host installObjS.

As this is an extension to MPWHost, which is the superclass of the MPWRemoteHost we used as the base for our HetznerHost, the server objects we use have the ability to install Objective-S on them. Neat.

And so do the server objects for the very similar script controlling DO droplets.


When I started out on this little excursion, my goal was not to demonstrate anything about Objective-S, I only needed to be able to use these cloud systems, and my hunch was that Objective-S would be good for the task.

It turned out even better than my hunch had suggested: the various features and characteristics of Objective-S, such as Polymorphic Identifiers, first class references, nested scheme handlers, and Higher Order Messaging, really work together quite seamlessly to allow interaction with both a REST API and with a remote host to be expressed compactly and naturally. In addition, it manages to naturally bridge the gap between ad-hoc scripting and proper modelling, remaining hackable without creating a mess.

It's working...

No comments: