Looking at a scripting language...and imagining the consequences

After thinking about the id subset and being pointed to WebScript, Brent Simmons imagines a scripting language. I have to admit I have been imagining pretty much the same language...and at some time decided to stop imagining and start building Objective-Smalltalk:

• Peer of Objective-C: objects are Objective-C objects, methods are Objective-C methods, added to the runtime and indistinguishable from the outside. "You can subclass UIViewController, or write a category on it."

  <void>alertView:alertView clickedButtonAtIndex:<int>buttonIndex
     self newGame.
     self view setNeedsDisplay.
  

  The example is from the site, it was copied from an actual program. As you can see, interoperability with the C parts of Objective-C is still necessary, but not bothersome.
• It has blocks:

  <void>deleteFile:filename
     thumbs := self thumbsView subviews.
     viewsToRemove := thumbs selectWhereValueForKey:'filename' isEqual:filename.
     aView := viewsToRemove firstObject.
  
     UIView animateWithDuration:0.4
            animations: [ aView setAlpha: 0.0. ]
            completion: [ aView removeFromSuperview. 
                          UIView animateWithDuration: 0.2
                                 animations: [ self thumbsView layoutSubviews. ]
                                 completion: [ 3  ]. 
                        ].
     url := self urlForFile:aFilename.
     NSFileManager defaultManager removeItemAtURL:url  error:nil.
     (self thumbsView afterDelay:0.4) setNeedsLayout.
  

  This example was also copied from an actual small educational game that was ported over from Flash.

  You also get Higher Order Messaging, Polymorpic Identifiers etc.

• Works with the toolchain: this is a a little more tricky, but I've made some progress...part of that is an llvm based native compiler, part is tooling that enables some level of integration with Xcode, part is a separate toolset that has comparable or better capabilities.

While Objective-Smalltalk would not require shipping source code with your applications, due to the native compiler, it would certainly allow it, and in fact my own BookLightning imposition program has been shipping with part of its Objective-Smalltalk source hidden its Resources folder for about a decade or so. Go ahead, download it, crack it open and have a look! I'll wait here while you do.

Did you have a look? The part that is in Smalltalk is the distilled (but very simple) imposition algorithm shown here.

---

	drawOutputPage:<int>pageNo
		isBackPage := (( pageNo / 2 ) intValue ) isEqual: (pageNo / 2 ).

		pages:=self pageMap objectAtIndex:pageNo.
		page1:=pages integerAtIndex:0.
		page2:=pages integerAtIndex:1.

		self drawPage:page1 andPage:page2 flipped:(self shouldFlipPage:pageNo).

	drawPage:<int>page1 andPage:<int>page2 flipped:<int>flipped
		drawingStream := self drawingStream.
		base := MPWPSMatrix matrixRotate:-90.

		drawingStream saveGraphicsState.
		flipped ifTrue: [ drawingStream concat:self flipMatrix ].
		width := self inRect x.

		self drawPage:page1 transformedBy:(base matrixTranslatedBy: (width * -2) y:0). 
		self drawPage:page2 transformedBy:(base matrixTranslatedBy: (width * -1) y:0). 
		
		drawingStream restoreGraphicsState.

Page imposition code

---

What this means is that any user of BookLightning could adapt it to suit their needs, though I am pretty sure that none have done so to this date. This is partly due to the fact that this imposition algorithm is too limited to allow for much variation, and partly due to the fact that the feature is well hidden and completely unexpected.

There are two ideas behind this:

1. Open Source should be more about being able to tinker with well-made apps in useful ways, rather than downloading and compiling gargantuan and incomprehensible tarballs of C/C++ code.
2. There is no hard distinction between programming and scripting. A higher level scripting/programming language would not just make developer's jobs easier, it could also enable the sort of tinkering and adaptation that Open Source should be about.

I don't think the code samples shown above are quite at the level needed to really enable tinkering, but maybe they can be a useful contribution to the discussion.

Cargo-cult typing, or: Objective-C's default type is id

In discussing some feedback to a chapter of my upcoming book, I was surprised to get the following code flagged:

---

-objectAtIndex:(NSUInteger)anIndex
{
   if ( anIndex < [self count] ) {
	  return objects[anIndex];
   }
   return nil;
}

---

The feedback was, effectively: "This code is incorrect, it is missing a return type". Of course, the code isn't incorrect in the least bit, the return type is id, because that is the default type, and in fact, you will see this style in both Brad Cox's book:

and the early NeXTStep documentation:

Having a default type for objects isn't entirely surprising, because at that time id was not just the default type, it was the only type available for objects, the optional static typing for objects wasn't introduced into Objective-C until later. In addition the template for Objective-C's object system was Smalltalk, which doesn't use static types, you just use variable names.

Cargo-cult typing

So while it is possible (and apparently common) to write -(id)objectAtIndex:(NSUInteger)anIndex, it certainly isn't any more correct. In fact, it's worse, because it is just syntactic noise [1][2], although it is arguably even worse than what Fowler describes because it isn't actually mandated by the language, the noise is inflicted needlessly.

And while we could debate as to whether it is better or not to write things that are redundant syntactic noise, we could also not, as that was settled almost 800 years ago: entia non sunt multiplicanda praeter necessitatem. You could also say KISS or "when in doubt, leave it out", all of which just say the the burden of proof is on whoever wants to add the redundant pieces.

What's really odd about this phenomenon is that we really don't gain anything from typing out these explicit types, the code certainly doesn't become more readable. It's as if we think that by following the ritual of explicitly typing out a type, we made the proper sacrifice to the gods of type-safety and they will reward us with correctness. But just like those Pacific islanders that built wooden planes, radios and control towers, the ritual is empty, because it conveys no information to the type system, or the reader.

The id subset

Now, I personally don't really care whether you put in a redundant (id) or not, I certainly have been reading over it (and not even really noticing) for my last two decades of Objective-C coding. However, the mistaken belief that it has to be there, rather than this is a personal choice you make, does worry me.

I think the problem goes a little deeper than just slightly odd coding styles, because it seems to be part and parcel of a drive towards making Objective-C look like an explicitly statically typed language along the lines of C++ or maybe Java, with one of the types being id. That's not the case: Objective-C is an optionally statically typed language. This means that you may specify type information if you want to, but you generally don't have to. I also want the emphasize that you can at best get Objective-C to look like such a language, the holes in the type system are way too big for this to actually gain much safety.

Properties started this trend, and now the ARC variant of the language turns what used to be warnings about unknown selectors needlessly into hard compiler errors. Of course, there are some who plausibly argue that this always should have been an error, or actually, that it always was an error, we just didn't know about it.

That's hogwash, of course. There is a subset of the language, which I'd like to call the id subset, where all the arguments and returns are object pointers, and for this it was always safe to not have additional type information, to the point where the compiler didn't actually have that additional type information. You could also call it the Smalltalk subset.

Another thing that's odd about this move to rigidify Objective-C in the face of success of more dynamic languages is that we actually have been moving into the right direction at the language base-level (disregarding the type-system): in general programming style, with new syntax support for object literals and subscripting, SmallInteger style NSNumbers modern Objective-C consists much more of pure objects than was traditionally the case. And as long as we are dealing with pure objects, we are in the id subset.

A dynamic language

What's great about the id subset is that it makes incremental, explorative programming very easy and lots of fun, much like other dynamic languages such as Smalltalk, Python or Ruby. (Not entirely like them, due to the need to compile to native code, but compilers are fast these days and there are possible fixes such as Objective-Smalltalk.)

The newly enforced rigidity is starting to make explorative programming in Objective-C much harder, and a lot less fun. In fact, it feels much more like C++ or Java and much less like the dynamic language that it is, and in my opinion is the wrong direction: we should be making our language more dynamic, and of course that's what I've been doing. So while I wouldn't agree with that tradeoff even if it were true, the fact is that we aren't actually getting static type safety, we are just getting a wood prop that will not fly.

Discussion on Hacker News.

---

UPDATE: Inserted a little clarification that I don't care about bike-shedding your code with regard to (id). The problem is that people's mistaken belief both that and why it has to be there is symptomatic of that deeper trend I wrote about.

codesign lies

Just had a case of codesign telling me my app was fine, just for the same app to be rejected by GateKeeper. The spctl tool fortunately was more truthful, but didn't really say where the problem was.

A little sleuthing determined that although I had signed all my frameworks with the Developer ID, two auxiliary executables were signed with my development certificate.

Lesson learned: don't trust codesign, use spctl to verify your binaries.

Objective-C: TIOBE programming language of the year third time in a row!

Actually: no it isn't, Transact-SQL got the honors. Apart from the obvious question, "Transact-Who?", it really should have been Objetive-C, because Tiobe readjusted the index mid-year in a way that resulted in a drop of 0.5% for the popular languages, which is fine, but without readjusting the historical data! Which is...not...especially if you make judgements based on relative performance.

In this case, Transact-SQL beat Objective-C by 0.17%, far less than the roughly 0.5% drop suffered by Objective-C mid-year. So Objective-C would have easily done the hat-trick, but I guess Tiobe didn't want that and rigged the game to make sure it doesn't happen.

Not that it matters...

UPDATE: I contacted Tiobe and they confirmed, both the lack of rebaselining and that Objective-C would likely have won an unprecedented third time in a row.

Switching contact from SMS to iMessage

So my girlfriend finally got an iMessage capable phone, but Messages on my phone still insisted on sending SMSes. Even after starting to receive iMessages in the same conversation. Even after a message sent from the Messages app on OS X was duly noted as being an iMessage!

Various attempts to fix this state of affairs had no effect: changing the contact number to iPhone, deleting the conversation(s), twiddling with Messsages settings on both phones, including the "Send as SMS" preference.

What did work was performing a reset of the Network Settings.

Should you use CoreData?

The answer, of course, is "it depends".

Now that we have that out of the way, I want to have a look at Drew Crawford's You should use Core Data, which manages to come up with a less nuanced answer in its 2943 words. It's an older article (2012), but recently came to my attention via Drew McCormack (@drewmccormack): "Great post", he wrote, and after reading the article I not just disagreed, but found that Twitter wasn't really adequate for writing up all the things wrong with that article.

Where to start? Maybe at the beginning, right in the first paragraph the following is called out as a "myth":

Among them, Core Data is designed for something–not really sure what–but whatever it is, it’s a lot more complicated than what I need to do in my project.

First here is a categorical mistake, because unless Drew knows the exact requirements and engineering trade-offs in every iOS application, he can't know whether this is true or false, fact or myth.

The second mistake is that the basic statement "CoreData was designed for something else" is actually true. CoreData's design dates back to NeXT's Enterprise Object Framework or EOF for short, and EOF was designed as an ORM for talking to corporate relational database servers, with a variety of alternate back-ends for non-relational DBs (including the way cool 3270 adapter!).

Obviously the implementation is different and the design has diverged by now, but that is the basic design, and yes, that does do something that is more complicated than what some (many?) developers need.

Details.

Next sentence, next problem:

I just want to save some entities to disk.

I may well be reading too much into this, but using the word entities already bakes so many assumptions into the problem statement. When I actually do want to save entities, databases in general and CoreData in particular are somewhat higher on my list of technologies, but quite often I don't start with ERM, and therefore just have objects, XML or other data. While these could be ER-modeled with varying degrees of difficulty/success, they certainly don't have to be, and it's often not the best choice.

In the enterprise system described in REST: Advanced Research Topics and Practical Applications, we removed the database from the rewrite, because the variety of the data meant converting the DB into a key-value store one way or another, or having a schema that's about an A0 page in small print (a standards body had developed such a schema and I think we even got a poster). We instead ended up converging on an EventPoster architecture that kept the original XML feed files around and parsed them into objects as necessary. No ERM here.

The next couple of paragraphs go off on an ad-hominem straw man tangent making WAG assumptions about the provenance of iOS developers and more WAGs about why that (assumed) provenance causes said developers to have these misconceptions. Those "misconceptions" that actually turn out to be true. Although largely irrelevant, it does contain some actual misinformation. For example the fact that categories don't get linked with static libraries is not an LLMV bug, it's a consequence of the combined semantics of static libraries and Objective-C categories irrespective of compiler/linker versions.

Details.

Joel

Then there's another tangent to Joel Spolsky's article on Things You Should Never Do (the link in Drew's article is dead), such as rewriting legacy code from scratch, which Joel describes categorically as "single worst strategic mistake" that any software company can make. In the words of the great Hans Bethe: "Ach, that isn't in wrong!":

1. Just because Joel wrote something makes it right or applicable why?
2. While Joel makes some good points and is right to counter a tendency to not want to deal with existing code, his claim is most certainly wrong in its absoluteness, both empirically (the system referenced above was a rewrite with tremendous benefits, then there's OS X vs. trying to keep fixing Classic Mac OS indefinitely etc.) and logically: it only holds true if all your accumulated complexity is of the essential kind, and none if of the accidental kind. That idea seems ludicrous to me, virtually all software is rushed, has shifting requirements that are only understood after the fact, has historical limitations (such as having to run in 128KB of RAM on a 7MHz CPU with no MMU) etc.

   Sometimes a rewrite is warranted, though you should obviously be wary and not undertake such a project lightly.

3. Of course the biggest non-sequitur in the whole tangent is "How on earth does the problem of reading source code and rewriting legacy systems apply to this situation?". Apart from "not at all"? We don't have access to CoreData source code and there is no question about us rewriting it. Well, actually, I used to have such access, but even though my remit would have allowed some rewrites, I don't think I could have done a better job for the technology constraints chosen. The question is whether to use it or not, including whether the technology constraints are appropriate.
4. And no, not every persistence solution ends up as effectively a rewrite of CoreData.

Details

After that ill-conceived tangent, the article goes right back to the next unsubstantiated ad-hominem:

Here are some things you probably haven’t thought about when architecting your so-called data stack:

Apart from the attitude somewhat unbefitting to someone who gets so much wrong, well, Nein, but lets's look at some of these in detail:

• Handling future changes to the data schema: this just isn't hard if you're not using a relational database. In fact high variability of the schema was one of the reasons for ditching the DB in you know...
• Bi-directional synchronization with servers...hah hah...!
• Bi-directional synchronization with peers...see above
• Undo support: gosh, I wish there were some sort of facility that would manage undo support on my model objects, if I had my way, Apple would add this and call it NSUndoManager. Without such a useful class, I'll just have to do complicated things like renaming old versions of my data file or storing deltas.
• Multithreading. Really? Multithreading?? If you think CoreData makes multithreading easier rather than harder, I have both a nice bridge and some oceanfront property in Nevada to sell you.
• Device/time constraints: the performance ca(na)rd. CoreData is slow and memory intensive. It makes up for this by adding the ability and the requirement for the developer to continuously tune the working set, if that is an option. If continuously minimizing/tuning the working set is not an option (i.e. you have some large datasets), you're hosed.

Details

Magic

Then we're back to the familiar ad-hominems/straw-men and non-sequitur analogies for a couple of paragraphs, nothing to see there. The whole analogy to Cocoa is useless: yes, Cocoa is good. How does this make CoreData good (it may or may not be, there simply is no connection) or appropriate for my tasks? Also, Cocoa in particular and Apple code in general is not "magic". I've been there, I've seen it and I fixed some of it. A lot of it is good, some excellent, but some not so much (sleep(5) anyone?).

The section entitled But, there are times not to use CoreData, right? could have been the saving grace, but alas the author blows it again. Having a "mark all as read" option in a NewsReader application is a "strange" "corner case"? Right. Let me turn this section around: there is a very narrow range of application areas where CoreData is or might be appropriate. Mostly read-only data, highly regular (table-like) structure, no need for bulk operations ever, preferably no trees either and no binary data. Fairly loose performance requirements.

What Apple says

The section on "what Apple says" is also gold, though I have to give credit to the Apple doc writers for managing to strongly suggest things without actually explicitly claiming them, strongly enough to fool this particular writer:

Apple’s high-level APIs can be much faster than ‘optimized’ code at lower levels.

This is obviously true, but completely meaningless. My tricycle can be faster than a Ferrari, for example if the Ferrari is out of gas or has a flat tire, or just parked. When we actually measured the performance of applications adopting CoreData at Apple we invariably got a significant performance regression. Then a lot of effort would be expended by all the teams involved in order to fix the regression, optimization effort that hadn't been expended on the original application, usually making up some of the shortfall.

I find the code reduction claim also specious, at least when stated as an unquestionable fact like this. In my experience, code size was reduced when removing CoreData and its predecessor EOF. Had we had exactly the requirements that CoreData/EOF were designed for (talking to existing relational enterprise databases), the result would almost certainly been different, but those were not our requirements, and I doubt that most iOS apps have those requirements (and in fact, CoreData didn't even support taking to external SQL databases, at all, a puzzling development for all the EOF veterans).

For managing small amounts of data inside in iOS application, CoreData is almost always overkill, because it effectively simulates an entire client/server enterprise application stack within your phone or desktop. The performance and complexity costs to pay for that overkill are substantial.

So Should You Use Core Data?

As I wrote: it depends.

The article in question at least adds no useful information to answering that question, only inappropriate analogies, repeated claims without any supporting evidence and lots of ad-hominems that are probably meant to be witty. If you believe its last section, the only reason developers don't use CoreData is because they are naive, lazy or ignorant.

This is evidently not true and ignores even the most basic concepts that one would apply to answering the question, such as, say, fitness for purpose. CoreData may well be the best implementation of an in-process-ORM simulating a client-server enterprise app there is, and there is good evidence that this is in fact true (certainly the people on it are top notch and some of the code is amazing). However, all that doesn't help if that's not what you need.

Considering the gleefully paraded ignorance of not just alternatives but various other programming aspects, I'd take the unconditional advice this article gives with a pinch of salt. Mountain-sized pinch, that is.

News from the Blogocalypse

So Google shut down Reader. When it happened all my news feeds went dead. I looked through the settings in my news reader, NetNewsWire 3.3.2, found the checkbox for "sync with Google Reader". Unchecked it.

It started syncing again, I did a "mark all as read" and things were back to normal. Now about that Snowden fellow...