Blackbird: A reference architecture for local-first connected mobile apps

Wow, what a mouthful! Although this architecture has featured in a number of my other writings, I haven't really described it in detail by itself. Which is a shame, because I think it works really well and is quite simple, a case of Sophisticated Simplicity.

Why a reference architecture?

The motivation for creating and now presenting this reference architecture is that the way we build connected mobile apps is broken, and none of the proposed solutions appear to help. How are they broken? They are overly complex, require way too much code, perform poorly and are unreliable.

Very broadly speaking, these problems can be traced to the misuse of procedural abstraction for a problem-space that is broadly state-based, and can be solved by adapting a state-based architectural style such as in-process REST and combining it with well-known styles such as MVC.

More specifically, MVC has been misapplied by combining UI updates with the model updates, a practice that becomes especially egregious with asynchronous call-backs. In addition, data is pushed to the UI, rather than having the UI pull data when and as needed. Asynchronous code is modelled using call/return and call-backs, leading to call-back hell, needless and arduous transformation of any dependent code into asynchronous code (see "what color is your function") that is also much harder to read, discouraging appropriate abstractions.

Backend communication is also an issue, with newer async/await implementations not really being much of an improvement over callback-based ones, and arguably worse in terms of actual readability. (They seem readable, but what actually happens is different  enough that the simplicity is deceptive).

Overview

The overall architecture has four fundamental components:

1. The model
2. The UI
3. The backend
4. The persistence

The main objective of the architecture is to keep these components in sync with each other, so the whole thing somewhat resembles a control loop architecture: something disturbs the system, for example the user did something in the UI, and the system responds by re-establishing equilibrium.

The model is the central component, it connects/coordinates all the pieces and is also the only one directly connected to more than one piece. In keeping with hexagonal architecture, the model is also supposed to be the only place with significant logic, the remainder of the system should be as minimal, transparent and dumb as possible.

memory-model := persistence.
persistence  |= memory-model.
ui          =|= memory-model. 
backend     =|= memory-model.

Graphically:

Elements

Blackbird depends crucially on a number of architectural elements: first are stores of the in-process REST architectural style. These can be thought of as in-process HTTP servers (without the HTTP, of course) or composable dictionaries. The core store protocol implements the GET, PUT and DELETE verbs as messages.

The role of URLs in REST is taken by Polymorphic Identifiers. These are objects that can reference identify values in the store, but are not direct pointers. For example, they need to be a able to reference objects that aren't there yet.

Polymorphic Identifiers can be application-specific, for example they might consist just of a numeric id,

MVC

For me, the key part of the MVC architectural style is the decoupling of input processing and resultant output processing. That is, under MVC, the view (or a controller) make some change to the model and then processing stops. At some undefined later time (could be synchronous, but does not have to be) the Model informs the UI that it has changed using some kind of notification mechanism.

In Smalltalk MVC, this is a dependents list maintained in the model that interested views register with. All these views are then sent a #changed message when the model has changed. In Cocoa, this can be accomplished using NSNotificationCenter, but really any kind of broadcast mechanism will do.

It is then the views' responsibility to update themselves by interrogating the model.

For views, Cocoa largely automates this: on receipt of the notification, the view just needs invalidate itself, the system then automatically schedules it for redrawing the next time through the event loop.

The reason the decoupling is important to maintain is that the update notification can come for any other reason, including a different user interaction, a backend request completing or even some sort of notification or push event coming in remotely.

With the decoupled M-V update mechanism, all these different kinds of events are handled identically, and thus the UI only ever needs to deal with the local model. The UI is therefore almost entirely decoupled from network communications, we thus have a local-first application that is also largely testable locally.

Blackbird refines the MVC view update mechanism by adding the polymorphic identifier of the modified item in question and placing those PIs in a queue. The queue decouples model and view even more than in the basic MVC model, for example it become fairly trivial to make the queue writable from any thread, but empty only onto the main thread for view updates. In addition, providing update notifications is no longer synchronous, the updater just writes an entry into the queue and can then continue, it doesn't wait for the UI to finish its update.

Decoupling via a queue in this way is almost sufficient for making sure that high-speed model updates don't overwhelm the UI or slow down the model. Both these performance problems are fairly rampant, as an example of the first, the Microsoft Office installer saturates both CPUs on a dual core machine just painting its progress bar, because it massively overdraws.

An example of the second was one of the real performance puzzlers of my career: an installer that was extremely slow, despite both CPU and disk being mostly idle. The problem turned out to be that the developers of that installer not only insisted on displaying every single file name the installer was writing (bad enough), but also flushing the window to screen to make sure the user got a chance to see it (worse). This then interacted with a behavior of Apple's CoreGraphics, which disallows screen flushes at a rate greater than the screen refresh rate, and will simply throttle such requests. You really want to decouple your UI from your model updates and let the UI process updates at its pace.

Having polymorphic identifiers in the queue makes it possible for the UI to catch up on its own terms, and also to remove updates that are no longer relevant, for example discarding duplicate updates of the same element.

The polymorphic identifier can also be used by views in order to determine whether they need to update themselves, by matching against the polymorphic identifier they are currently handling.

Backend communication

Almost every REST backend communication code I have seen in mobile applications has created "convenient" cover methods for every operation of every endpoint accessed by the application, possibly automatically generated.

This ignores the fact that REST only has a few verbs, combined with a great number of identifiers (URLs). In Blackbird, there is a single channel for backend communication: a queue that takes a polymorphic identifier and an http verb. The polymorphic identifier is translated to a URL of the target backend system, the resulting request executed and when the result returns it is placed in the central store using the provided polymorphic identifier.

After the item has been stored, an MVC notification with the polymorphic identifier in question is enqueued as per above.

The queue for backend operations is essentially the same one we described for model-view communication above, for example also with the ability to deduplicate requests correctly so only the final version of an object gets sent if there are multiple updates. The remainder of the processing is performed in pipes-and-filters architectural style using polymorphic write streams.

If the backend needs to communicate with the client, it can send URLs via a socket or other mechanism that tells the client to pull that data via its normal request channels, implementing the same pull-constraint as in the rest of the system.

One aspect of this part of the architecture is that backend requests are reified and explicit, rather than implicitly encoded on the call-stack and its potentially asynchronous continuations. This means it is straightforward for the UI to give the user appropriate feedback for communication failures on the slow or disrupted network connections that are the norm on mobile networks, as well as avoid accidental duplicate requests.

Despite this extra visibility and introspection, the code required to implement backend communications is drastically reduced. Last not least, the code is isolated: network code can operate independently of the UI just as well as the UI can operate independently of the network code.

Persistence

Persistence is handled by stacked stores (storage combinators).

The application is hooked up to the top of the storage stack, the CachingStore, which looks to the application exactly like the DictStore (an in-memory store). If a read request cannot be found in the cache, the data is instead read from disk, converted from JSON by a mapping store.

For testing the rest of the app (rather than the storage stack), it is perfectly fine to just use the in-memory store instead of the disk store, as it has the same interface and behaves the same, except being faster and non-persistent.

Writes use the same asynchronous queues as the rest of the system, with the writer getting the polymorphic identifiers of objects to write and then retrieving the relevant object(s) from the in-memory store before persisting. Since they use the same mechanism, they also benefit from the same uniquing properties, so when the I/O subsystem gets overloaded it will adapt by dropping redundant writes.

Consequences

With the Blackbird reference architecture, we not only replace complex, bulky code with much less and much simpler code, we also get to reuse that same code in all parts of the system while making the pieces of the system highly independent of each other and optimising performance.

In addition, the combination of REST-like stores that can be composed with constraint- and event-based communication patterns makes the architecture highly decoupled. In essence it allows the kind of decoupling we see in well-implemented microservices architectures, but on mobile apps without having to run multiple processes (which is often not allowed).

UIs Are Not Pure Functions of the Model - React.js and Cocoa Side by Side

When I first saw React.js, I had a quick glance and thought that it was cool, they finally figured out how to do a Cocoa-like MVC UI framework in JavaScript.

Of course, they could have just used Cappuccino, but "details". Some more details were that their version of drawRect: was called render() and returned HTML instead of drawing into a graphics context, but that's just the reality of living inside a browser. And of course there was react-canvas.

Overall, though, the use of a true MVC UI framework seemed like a great step forward compared to directly manipulating the DOM on user input, at least for actual applications.

Imagine my surprise when I learned about React.native! It looked like they took what I had assumed to be an implementation detail as the main feature. Looking a little more closely confirmed my suspicions.

Fortunately, the React.js team was so kind as to put their basic ideas in writing: React - Basic Theoretical Concepts (also discussed on HN) ). So I had a look and after a bit of reading decided it would be useful to do a side-by-side comparison with equivalents of those concepts in Cocoa as far as I understand them.

---

React	Cocoa
Transformation The core premise for React is that UIs are simply a projection of data into a different form of data. The same input gives the same output. A simple pure function. function NameBox(name) { return { fontWeight: 'bold', labelContent: name }; }	Transformation A core premise of Cocoa, and MVC in general, is that UIs are a projection of data into a different form of data, specifically bits on a screen. The same input gives the same output. A simple method: -(void)drawRect:(NSRect)aRect { ... } Due to the fact that screens and the bits on them are fairly expensive, we use the screen as an accumulator instead of returning the bits from the method. We do not make the (unwarranted) assumption that this transformation can or should be expressed as a pure function. While that would be nice, there are many reasons why this is not a good idea, some pretty obvious.
Abstraction You can't fit a complex UI in a single function though. It is important that UIs can be abstracted into reusable pieces that don't leak their implementation details. Such as calling one function from another. function FancyUserBox(user) { return { borderStyle: '1px solid blue', childContent: [ 'Name: ', NameBox(user.firstName + ' ' + user.lastName) ] }; } { firstName: 'Sebastian', lastName: 'Markbåge' } -> { borderStyle: '1px solid blue', childContent: [ 'Name: ', { fontWeight: 'bold', labelContent: 'Sebastian Markbåge' } ] };	Abstraction Although it is be possible to render an entire UI in a single View's drawRect:: method, and users of NSOpenGLView tend to do that, it is generally better practice to split complex UIs into reusable pieces that don't leak their implementation details. Fortunately we have such reusable pieces, they are called objects, and we can group them into classes. Following the MVC naming conventions, we call objects that represent UI views, in Cocoa they are instance of NSView or its subclasses, in CocoaTouch the common superclass in called UIView.
Composition To achieve truly reusable features, it is not enough to simply reuse leaves and build new containers for them. You also need to be able to build abstractions from the containers that compose other abstractions. The way I think about "composition" is that they're combining two or more different abstractions into a new one. function FancyBox(children) { return { borderStyle: '1px solid blue', children: children }; } function UserBox(user) { return FancyBox([ 'Name: ', NameBox(user.firstName + ' ' + user.lastName) ]); }	Composition To achieve truly reusable features, it is not enough to simply reuse leaves and build new containers for them. You also need to be able to build abstractions from the containers that compose other abstractions. The way I think about "composition" is that they're combining two or more different abstractions into a new one. Examples of this are the NSScrollView, which composes the actual scrollers, themselves composed of different parts, a NSClipView to provide a window onto the user-provided contentView. Other examples are NSTableViews coordinating their columns, rows, headers and the system- or user-provided Cells.
State A UI is NOT simply a replication of server / business logic state. There is actually a lot of state that is specific to an exact projection and not others. For example, if you start typing in a text field. That may or may not be replicated to other tabs or to your mobile device. Scroll position is a typical example that you almost never want to replicate across multiple projections. We tend to prefer our data model to be immutable. We thread functions through that can update state as a single atom at the top. function FancyNameBox(user, likes, onClick) { return FancyBox([ 'Name: ', NameBox(user.firstName + ' ' + user.lastName), 'Likes: ', LikeBox(likes), LikeButton(onClick) ]); } // Implementation Details var likes = 0; function addOneMoreLike() { likes++; rerender(); } // Init FancyNameBox( { firstName: 'Sebastian', lastName: 'Markbåge' }, likes, addOneMoreLike ); NOTE: These examples use side-effects to update state. My actual mental model of this is that they return the next version of state during an "update" pass. It was simpler to explain without that but we'll want to change these examples in the future.	State A UI is NOT simply a replication of server / business logic state. There is actually a lot of state that is specific to an exact projection and not others. For example, if you start typing in a text field. That may or may not be replicated to other tabs or to your mobile device. Scroll position is a typical example that you almost never want to replicate across multiple projections. Fortunately, the view objects we are using already provide exactly this UI-specific state, so yay objects.
Memoization Calling the same function over and over again is wasteful if we know that the function is pure. We can create a memoized version of a function that keeps track of the last argument and last result. That way we don't have to reexecute it if we keep using the same value. function memoize(fn) { var cachedArg; var cachedResult; return function(arg) { if (cachedArg === arg) { return cachedResult; } cachedArg = arg; cachedResult = fn(arg); return cachedResult; }; } var MemoizedNameBox = memoize(NameBox); function NameAndAgeBox(user, currentTime) { return FancyBox([ 'Name: ', MemoizedNameBox(user.firstName + ' ' + user.lastName), 'Age in milliseconds: ', currentTime - user.dateOfBirth ]); }	Memoization Calling the same method over and over again is wasteful. So we don't do that. First, we did not start with the obviously incorrect premise that the UI is a simple "pure" function of the model. Except for games, UIs are actually very stable, more stable than the model. You have chrome, viewers, tools etc. What is a (somewhat) pure mapping from the model is the data that is displayed in the UI, but not the entire UI. So if we don't make the incorrect assumption that UIs are unstable (pure functions of model), then we don't have to expend additional and fragile effort to re-create that necessary stability. In terms of optimizing output, this is also handled within the model, rather than in opposition to it: views are stable, so we keep a note of which views have collected damage and need to be redrawn. The application event loop coalesces these damage rectangles and initiates an optimized operation: it only redraws views whose bounds intersect the damaged region, and also passes the rectangle(s) to the drawRect:: method. (That's why it's called drawRect::).
Lists Most UIs are some form of lists that then produce multiple different values for each item in the list. This creates a natural hierarchy. To manage the state for each item in a list we can create a Map that holds the state for a particular item. function UserList(users, likesPerUser, updateUserLikes) { return users.map(user => FancyNameBox( user, likesPerUser.get(user.id), () => updateUserLikes(user.id, likesPerUser.get(user.id) + 1) )); } var likesPerUser = new Map(); function updateUserLikes(id, likeCount) { likesPerUser.set(id, likeCount); rerender(); } UserList(data.users, likesPerUser, updateUserLikes); NOTE: We now have multiple different arguments passed to FancyNameBox. That breaks our memoization because we can only remember one value at a time. More on that below.	Lists Most UIs are some form of lists that then produce multiple different values for each item in the list. This creates a natural hierarchy. Fortunately there is nothing to do here, the basic hierarchical view model already takes care of it. There are specific view classes for lists, but there is nothing special about them in terms of the conceptual or implementation model.
Continuations Unfortunately, since there are so many lists of lists all over the place in UIs, it becomes quite a lot of boilerplate to manage that explicitly. We can move some of this boilerplate out of our critical business logic by deferring execution of a function. For example, by using "currying" (bind in JavaScript). Then we pass the state through from outside our core functions that are now free of boilerplate. This isn't reducing boilerplate but is at least moving it out of the critical business logic. function FancyUserList(users) { return FancyBox( UserList.bind(null, users) ); } const box = FancyUserList(data.users); const resolvedChildren = box.children(likesPerUser, updateUserLikes); const resolvedBox = { ...box, children: resolvedChildren };	Continuations Fortunately, it doesn't matter how many lists of lists there are all over the place in UIs, since our composition mechanism actually works for this use case.
State Map We know from earlier that once we see repeated patterns we can use composition to avoid reimplementing the same pattern over and over again. We can move the logic of extracting and passing state to a low-level function that we reuse a lot. function FancyBoxWithState( children, stateMap, updateState ) { return FancyBox( children.map(child => child.continuation( stateMap.get(child.key), updateState )) ); } function UserList(users) { return users.map(user => { continuation: FancyNameBox.bind(null, user), key: user.id }); } function FancyUserList(users) { return FancyBoxWithState.bind(null, UserList(users) ); } const continuation = FancyUserList(data.users); continuation(likesPerUser, updateUserLikes);	State Map Don't need it. (Just how many distinct mechanism do we have now for re-introducing state? At what point do we revisit our initial premise that UIs are pure functions of the model??)
Memoization Map Once we want to memoize multiple items in a list memoization becomes much harder. You have to figure out some complex caching algorithm that balances memory usage with frequency. Luckily, UIs tend to be fairly stable in the same position. The same position in the tree gets the same value every time. This tree turns out to be a really useful strategy for memoization. We can use the same trick we used for state and pass a memoization cache through the composable function. function memoize(fn) { return function(arg, memoizationCache) { if (memoizationCache.arg === arg) { return memoizationCache.result; } const result = fn(arg); memoizationCache.arg = arg; memoizationCache.result = result; return result; }; } function FancyBoxWithState( children, stateMap, updateState, memoizationCache ) { return FancyBox( children.map(child => child.continuation( stateMap.get(child.key), updateState, memoizationCache.get(child.key) )) ); } const MemoizedFancyNameBox = memoize(FancyNameBox);	Memoization Map Huh? Seriously?
Algebraic Effects It turns out that it is kind of a PITA to pass every little value you might need through several levels of abstractions. It is kind of nice to sometimes have a shortcut to pass things between two abstractions without involving the intermediates. In React we call this "context". Sometimes the data dependencies don't neatly follow the abstraction tree. For example, in layout algorithms you need to know something about the size of your children before you can completely fulfill their position. Now, this example is a bit "out there". I'll use Algebraic Effects as proposed for ECMAScript. If you're familiar with functional programming, they're avoiding the intermediate ceremony imposed by monads. function ThemeBorderColorRequest() { } function FancyBox(children) { const color = raise new ThemeBorderColorRequest(); return { borderWidth: '1px', borderColor: color, children: children }; } function BlueTheme(children) { return try { children(); } catch effect ThemeBorderColorRequest -> [, continuation] { continuation('blue'); } } function App(data) { return BlueTheme( FancyUserList.bind(null, data.users) ); }	Algebraic Effects It turns out that it is kind of a PITA to pass every little value you might need through several levels of abstractions.... So don't do that. This is another reason why it's advantageous to have a stable hierarchy of stateful objects representing your UI. If you need more context, you just ask around. Ask your parent, ask your siblings, ask your children, they are all present. Again, no special magic needed.

---

So that was the comparison. I have to apologise for getting somewhat less detail-oriented near the end, but the level of complexity just started to overwhelm me. Add to that, for React.native, the joy of having to duplicate the entire hierarchy of view classes, just to have the "components" generate not-quite-temporary temporary views that than generate layers that draw the actual UI. Maybe there's one too many layers. Or two.

The idea of UI being a pure function of the model seems so obviously incorrect, and leads to such a plethora of problems, that it is a bit puzzling how one could come up with it in the first place, and certainly how one would stick with it in face of the avalanche of problems that just keeps coming. A part of this is certainly the current unthinking infatuation with functional programming ideas. These ideas are broadly good, but not nearly as widely or universally applicable as some of their more starry-eyed proponents propose (I hesitate to use the word "think" in this context).

Another factor is the the usefulness of immediate-mode graphics, compared to retained-state graphics. This is actually an interesting topic by itself, IMHO, one of those eternal circles where we move from a fully retained model such as the early GKS or laterr Phigs to immedate drawing models such as Postscript, Quartz, und OpenGL, only to then re-invent the retained model (sort of) with things like CoreAnimation, Avalon and, of course, the DOM (and round and round we go). Cocoa's model represents a variable approach, where you can mix-and-match object-oriented and immediate-mode drawing as you see fit. But more on that later.

Last not least, it's probably not entirely coincidental that this idea was hatched for Facebook and Instagram feed applications. Similar to games, these sorts of apps have displays that really are determined mostly by their "model", the stream of data coming from their feed. I am not convinced that feed application generalizes well to application.

Anyway, for me, this whole exercise has actually motivated me to start using react.js a little. I still think that Cappuccino is probably the better, less confused framework, but it helps to know about what the quasi-mainstream is doing. I also think that despite all the flaws, react.js and react.native are currently eating native development's lunch. And that's certainly interesting. Stay tuned!

UPDATE:
Dan Abramov responds:

I think you’re overfocusing on the “pure” wording and theoretical definitions over practical differences.

To elaborate a bit, React components aren’t always “pure” in strict FP sense of the word. They’re also not always functions (although we’re adding a stateful function API as a preferred alternative to classes soon). Support for local state and side effects is absolutely a core feature of React components and not something we avoid for “purity”.

I added a PR to remove the misleading "pure" from the concepts page.

Refactoring Towards Language

When implementing MVC in iOS and macOS, one common pattern is to use NSNotification to implement the notification that the model has changed and the view should redraw itself. This is a good pattern, as that is how MVC is intended to work, but it does lead to quite a bit of boilerplate in this incarnation.

Specifically, you typically have some version of the following code in one of the initialisation methods of your ViewControllers, which play the MVC role of the View in Cocoa and Cocoa Touch.

---

- (void)viewDidLoad
{
	...
    [[NSNotificationCenter defaultCenter] addObserver:self
                                             selector:@selector(modelDidChange:)
                                                 name:MPWAppNotificationModelDidChange
                                               object:nil];
	...
}

---

Refactor I

That's all good, except that you tend to repeat that code in every single ViewController, and there are usually quite a few of them. So one way of avoiding all that duplication is to refactor by extracting the duplicated functionality into a method.

---

- (void)subscribeToModelDidChangeNotification
{
    [[NSNotificationCenter defaultCenter] addObserver:self
                                             selector:@selector(modelDidChange:)
                                                 name:MPWAppNotificationModelDidChange
                                               object:nil];
}


- (void)unsubscribeFromModelDidChangeNotification
{
    [[NSNotificationCenter defaultCenter] removeObserver:self
                                                    name:MPWAppNotificationModelDidChange
                                                  object:nil];
}

- (void)subscribeToModelWasDeletedNotification
{
    [[NSNotificationCenter defaultCenter] addObserver:self
                                             selector:@selector(modelDidChange:)
                                                 name:MPWAppNotificationModelWasDeleted
                                               object:nil];
}


- (void)unsubscribeFromModelWasDeletedNotification
{
    [[NSNotificationCenter defaultCenter] removeObserver:self
                                                    name:MPWAppNotificationModelWasDeleted
                                                  object:nil];
}
...

---

Refactor II

The interesting thing that happens when you remove duplication is that it will often reveal more duplication. In particular, if you look at more of these, you will notice that the method bodies are largely identical. You always send a message to the defaultCenter of the NSNotificationCenter class, you pretty much always add yourself as the Observer, and I've rarely seen the object: parameter used. The only things that vary are the selector to send and the name of the notification.

---

    [[NSNotificationCenter defaultCenter] addObserver:self
                                             selector: <some selector>
                                                 name: <some notification>
                                               object:nil];

---

So yet another classic case for refactoring by extracting a helper method that encapsulates the things that do not change and takes the things that do vary as arguments.

---

-(void)subscribeToNotification:(NSString *)notificationName usingSelector:(SEL)selectorName
{
    [[NSNotificationCenter defaultCenter] addObserver:self
                                             selector:selectorName
                                                 name:notificationName
                                               object:nil];
}

---

We can also apply the same refactoring to unsubscribing.

---

-(void)unsubscribeFromNotification:(NSString *)notificationName
{
    [[NSNotificationCenter defaultCenter] removeObserver:self
                                                    name:notificationName
                                                  object:nil];
}

---

With the parameters extracted, our helper methods simplify to the following.

---

- (void)subscribeToModelDidChangeNotification
{
    [self subscribeToNotification: MPWAppNotificationModelDidChange usingSelector:@selector(modelDidChange:)];
}

- (void)unsubscribeFromModelDidChangeNotification
{
    [self unsubscribeFromNotification: MPWAppNotificationModelDidChange];
}

- (void)subscribeToModelWasDeletedNotification
{
    [self subscribeToNotification: MPWAppNotificationModelWasDeleted usingSelector:@selector(modelDidChange:)];
}

- (void)unsubscribeFromModelWasDeletedNotification
{
    [self unsubscribeFromNotification: MPWAppNotificationModelWasDeleted];
}

---

Refactor III

As happens very often when you extract code and remove duplication we discover more duplication that was previously somewhat hidden, for example because code that was too far away to really notice the duplication is now close enough together to make it more obvious. Specifically: we always define the same methods: subscribeTo<notification> and unsubscribeFrom<notification>. We also repeat the notification name in each of the method names, and also in each of the method bodies (with different prefixes/suffixes).

---

- (void)subscribeTo<notification-name>Notification
{
    [self subscribeToNotification: MPWAppNotification<notification-name> usingSelector:@selector(<notification-selector>:)];
}

- (void)unsubscribeFrom<notification-name>Notification
{
    [self unsubscribeFromNotification: MPWAppNotification<notification-name>];
}
- (void)<notification-selector>:(NSNotification *)notification
{
}

---

Up to this point, we managed to achieve our goals using plain refactoring techniques. Often, that is enough, and it’s generally a good idea to stop there, because beyond lies metaprogramming, which tends to make the code significantly more obscure. In Objective-C, we generally have runtime tricks and macro-programming using the C pre-processor (well: and code generation). In this case, I decided to give the C preprocessor a try, because I wanted things done at compile-time and the problem was parametrising parts of identifiers, i.e. string processing.

Doing some pattern matching, we have two actual “parameters", the “base” of the notification name ( for example ‘ModelDidChange’ or ‘ModelWasDeleted’ ) and the name of the method to call. So we create a Macro that takes these parameters and applies them to a template of the methods we need:

---

#define SUBSCRIBE_UNSUBSCRIBE_WITHMETHODNAME( name , notification, theMessage ) \
- (void)subscribeTo##name \
{\
    [self subscribeToNotification:notification usingSelector:@selector(theMessage:)];\
}\
\
- (void)unsubscribeFrom##name \
{\
    [self unsubscribeFromNotification:notification];\
}\
-(void)theMessage:(NSNotification *)notification\
{\
}\

#define SUBSCRIBE_UNSUBSCRIBE( commonName, theMessage ) SUBSCRIBE_UNSUBSCRIBE_WITHMETHODNAME( commonName##Notification, MPWAppNotification##commonName , theMessage )

---

The first macro has three arguments, a version of the notification name suitable for adding to the method names, the full notification name and the message name. The ‘##’ is the token pasting operator, which allows us to create new lexical tokens out of existing ones. We “need” it in this case because we have commonality we want to express that is on a sub-token, sub-identifier level.

There are a bunch of issues with this approach. First, it is pretty unreadable. You need to terminate every line with backslashes because Macro definitions can only be on a single logical line, the backslashes continue the logical line over physical lines. Dealing with compiler errors and warnings is rather tricky, because the error will be with the result of the macro expansion, which the error message will usually not contain. (You can tell the compiler to run just the pre-processor and give you the result, in case you need to debug and can’t do the macro expansion in your head…). Finally, you cannot search for the tokens that get generated by the pre-processor. So for example no command-clicking, and you have to be careful if you ever change one of the notification names.

The advantage is that you get a representation that succinctly states what you want to accomplish, without any duplicated boilerplate. No duplication also means no chance to get those duplicates wrong: the first few code samples in this post actually contain an error. (MPWAppNotificationModelWasDeleted also sends the modelDidChange: message instead of its own modelWasDeleted:) This error, almost certainly a copy-and-paste bug, was actually in the original code and had gone unnoticed for months until I tried to remove that duplication.

I don't know about you, but my eyes just glaze over when scanning large swathes of mostly duplicated code.

Anyway, with the Macros, our help methods are now defined as follows:

---

SUBSCRIBE_UNSUBSCRIBE( ModelDidChange, modelDidChange )
SUBSCRIBE_UNSUBSCRIBE( ModelWasDeleted, modelWasDeleted )

SUBSCRIBE_UNSUBSCRIBE( ActivityCountDidChange, activityCountDidChange )
SUBSCRIBE_UNSUBSCRIBE( ObjectIdDidChange, objectIdDidChange )
SUBSCRIBE_UNSUBSCRIBE( UserDidLogin, userDidLogin )

---

Note that previously we showed just two notifications, now we are showing all five, which would have been quite unwieldy before.

Refactor IV

As is typical when you remove duplication, you notice more duplication, because stuff is closer together: the two parameters are almost identical, except for capitalisation. This doesn't have to be the case, but it is a good convention to have. Alas, the pre-processor can’t change the capitalisation of strings so we are stuck.

To recap, this was a process of detecting duplication, removing that duplication using available mechanisms and then detecting more duplication, over several iterations. What then usually happens is that you either manage to remove all the duplication, or you notice that you cannot reduce duplication any further. Not being able to remove duplication typically means that you have reached limitations of your language, with metaprogramming facilities allowing you to push those limitations at least a little, and sometimes quite a bit.

This particular exploration relied on a somewhat formulaic use of NSNotificationCenter, one that always matches a notification with the same message. Since messages are already late-bound, this doesn't really present much of a restriction and seems a useful simplification, and it is a simplification that I've seen used widely in practice. The other pattern is that specific classes typically observe notifications for essentially their entire lifetime, meaning that the ability to dynamically turn notifications on and off is often not needed.

If we imagine language support for notifications (the implicit invocation architectural style), we would probably like to be able to declare notifications, declare that a class listens to a specific notification and and check conformance. This would make usage even more convenient than the macros we refactored to, while at the same time addressing the problems of the macro solution.

And of course that is what Notification Protocols became: a slight misappropriation of Objective-C and Swift Protocols to get something that is extremely close to actual language support. I can now actually declare my notifications as almost a programming-language thing (at least better than a string), and also specify the relationship between a message and that notification, which otherwise is purely lost in convention:

---

@protocol ModelDidChange <MPWNotificationProtocol>

-(void)modelDidChange:(NSNotifiction*)notification;

@end

---

I can also easily and declaratively specify that a particular class listens to a notification:

---

@interface NotifiedView:NSView <ModelDidChange>

@end

---

Not only does this remove the problems with the Macro approach, unreadability and untraceability, it is actually quite a bit better than the approach without Macros, all while being at least as compact. Last not least, notification sending is not just compact, but also obvious and at least somewhat compiler-checked:

---

[@protocol(ModelDidChange) notify];

---

This is very, very close to native language support, due to some lucky coincidences and the wise decision to make Protocols first class objects in Objective-C. It also does not match the flexibility of the NSNotificationCenter APIs, but I doubt whether that additional flexibility is actually used/useful.

Had I not experimented with removing duplication, and iterated on removing duplication, I never would have come to the point where notification protocols became an obvious solution. And now that I have notification protocols, I also have a good idea what actual language support should look like, because I've been using something pretty close.

So for me, plain old refactoring, different kinds of metaprogramming and language support all live on a continuum of improving expressiveness, and all are, or at least should be part of our feedback loops. How can we improve our language so we don't need metaprogramming for common use cases? How can we improve our metaprogramming facilities so they are sufficient and we don't feel a need for replacing them with actual language support? How can we do both, turn things that currently require metaprogramming look more like plain base-level programming while making it integrate better than metaprogramming solutions?

That, Detective, is the right question. Program terminated

Model Widget Controller (MWC) aka: Apple "MVC" is not MVC

I probably should have taken more notice that time that after my question about why a specific piece of the UI code had been structured in a particular way, one of my colleagues at 6wunderkinder informed me that Model View Controller meant the View must not talk to the model, and instead the Controller is responsible for mediating all interaction between the View and the Model. It certainly didn't match the definition of MVC that I knew, so I checked the Wikipedia page on MVC just in case I had gone completely senile, but it checked out with that I remembered:

1. the controller updates the model,
2. the model notifies the view that it has changed, and finally
3. the view updates itself by talking to the model

(The labeling on the graphic on the Wikipedia is a bit misleading, as it suggests that the model updates the view, but the text is correct).

What I should have done, of course, is keep asking "Why?", but I didn't, my excuse being that we were under pressure to get the Wunderlist 3.0 release out the door. Anyway, I later followed up some of my confusion about both React.native and ReactiveCocoa (more on those in a later post) and found the following incorrect diagram in a Ray Wenderlich tutorial on ReactiveCocooa and MVVC.

Hmm...that's the same confusion that my colleague had. The plot thickens as I re-check Wikipedia just to be sure. Then I had a look at the original MVC papers by Trygve Reenskaug, and yes:

A view is a (visual) representation of its model. It would ordinarily highlight certain attributes of the model and suppress others. It is thus acting as a presentation filter. A view is attached to its model (or model part) and gets the data necessary for the presentation from the model by asking questions.

The 1988 JOOP article "MVC Cookbook" also confirms:

So where is this incorrect version of MVC coming from? It turns out, it's in the Apple documentation, in the overview section!

I have to admit that I hadn't looked at this at least in a while, maybe ever, so you can imagine my surprise and shock when I stumbled upon it. As far as I can tell, this architectural style comes from having self-contained widgets that encapsulate very small pieces of information such as simple strings, booleans or numbers. The MVC architecture was not intended for these kinds of small widgets:

MVC was conceived as a general solution to the problem of users controlling a large and complex data set.

If you look at the examples, the views are large both in size and in scope, and they talk to a complex model. With a widget, there is no complex model, not filtering being done by the view. The widget contains its own data, for example a string or a number. An advantage of widgets is that you can meaningfully assemble them in a tool like Interface Builder, with a more MVC-like large view, all you have in IB is a large blank space labeled 'Custom View'. On the other hand, I've had very good experiences with "real" (large view) MVC in creating high performance, highly responsive user interfaces.

Model Widget Controller (MWC) as I like to call it, is more tuned for forms and database programming, and has problems with more reactive scenarios. As Josh Abernathy wrote:

Right now we write UIs by poking at them, manually mutating their properties when something changes, adding and removing views, etc. This is fragile and error-prone. Some tools exist to lessen the pain, but they can only go so far. UIs are big, messy, mutable, stateful bags of sadness.

To me, this sadness is almost entirely a result of using MWC rather than MVC. In MVC, the "V" is essentially a function of the model, you don't push or poke at it, you just tell it "something changed" and it redraws itself.

And so the question looms: is react.native just a result of (Apple's) misunderstanding (of) MVC?

As always, your comments are welcome here or on HN.