Exploring the Weather Underground with Objective-XML and Objective-Smalltalk

Having taken up various forms of flying last year, I have developed a strong interest in the weather, particularly wind information. While there are various web-sites with relevant information, for example Jeff Greenbaum's excellent Wind Conditions Page for Pacifica page, they don't really present the information quite the way I need, and also don't really work well on small mobile devices...

Fixing that should hopefully just be ASMOP. The Weather Underground fortunately has some reasonably well-documented XML APIs, let's see what they have to offer and wether we can get to the data we want.

First, let's fire up the interactive Smalltalk Shell (stsh) and load the Objective-XML framework.

marcel@spock[Projects]stsh
> context loadFramework:'MPWXmlKit'

Next, let's have a look at the raw XML returned by the Weather Underground APIs.

> urlstr:='http://api.wunderground.com/weatherstation/WXCurrentObXML.asp?ID=KCADALYC1'
> url:=NSURL URLWithString: urlstr
> NSString stringWithContentsOfURL:url.
> 

Hmm...no result. (It turns out that Weather Underground checks the user agent and errors if it doesn't find one. The various convenience methods do not send a User Agent). Maybe curl can help?

>context addExternalCommand:'curl'.
>curl run:'http://api.wunderground.com/weatherstation/WXCurrentObXML.asp?ID=KCADALYC1'
<?xml version="1.0"?>
	<current_observation>
		<credit>Weather Underground Personal Weather Station</credit>
		<credit_URL>http://wunderground.com/weatherstation/</credit_URL>
		<image>
		<url>http://icons.wunderground.com/graphics/bh-wui_logo.gif</url>
		<title>Weather Underground</title>
		<link>http://wunderground.com/weatherstation/</link>
		</image>
		<location>
		<full>Mussel Rock, Daly City, CA</full>
		<neighborhood>Mussel Rock</neighborhood>
		<city>Daly City</city>
		<state>CA</state>
		<zip></zip>
		<latitude>37.667347</latitude>
		<longitude>-122.489342</longitude>
		<elevation>514 ft</elevation>
		</location>
		<station_id>KCADALYC1</station_id>
		<station_type>Fan-aspirated Davis Vantage Pro 2 Plus</station_type>
		<observation_time>Last Updated on November 7, 1:55 PM PST</observation_time>
		<observation_time_rfc822>Sat, 07 November 2009 21:55:21 GMT</observation_time_rfc822>
		<weather></weather>
		<temperature_string>56.9 F (13.8 C)</temperature_string>
		<temp_f>56.9</temp_f>
		<temp_c>13.8</temp_c>
		<relative_humidity>83</relative_humidity>
		<wind_string>From the NW at 15.0 MPH Gusting to 16.0 MPH</wind_string>
		<wind_dir>NW</wind_dir>
		<wind_degrees>313</wind_degrees>
		<wind_mph>15.0</wind_mph>
		<wind_gust_mph>16.0</wind_gust_mph>
		<pressure_string>30.07" (1018.2 mb)</pressure_string>
		<pressure_mb>1018.2</pressure_mb>
		<pressure_in>30.07</pressure_in>
		<dewpoint_string>51.8 F (11.0 C)</dewpoint_string>
		<dewpoint_f>51.8</dewpoint_f>
		<dewpoint_c>11.0</dewpoint_c>
		
		<heat_index_string></heat_index_string>
		<heat_index_f></heat_index_f>
		<heat_index_c></heat_index_c>
		
		<windchill_string></windchill_string>
		<windchill_f></windchill_f>
		<windchill_c></windchill_c>
		
		<solar_radiation>483.00</solar_radiation>
		<UV>2.5</UV>
		<precip_1hr_string>0.00 in (0.0 mm)</precip_1hr_string>
		<precip_1hr_in>0.00</precip_1hr_in>
		<precip_1hr_metric>0.0</precip_1hr_metric>
		<precip_today_string>0.01 in (0.0 mm)</precip_today_string>
		<precip_today_in>0.01</precip_today_in>
		<precip_today_metric>0.0</precip_today_metric>
		<history_url>http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KCADALYC1</history_url>
		<ob_url>http://www.wunderground.com/cgi-bin/findweather/getForecast?query=37.667347,-122.489342</ob_url>
	</current_observation>
<!-- 0.029:0 -->

Much better. Now let's see if we can parse that XML data into a Cocoa Property List.

> parser := MPWMAXParser parser.
> parser parsedDataFromURL: 'http://api.wunderground.com/weatherstation/WXCurrentObXML.asp?ID=KCADALYC1'
{
    UV = "2.5";
    credit = "Weather Underground Personal Weather Station";
    "credit_URL" = "http://wunderground.com/weatherstation/";
    "dewpoint_c" = "11.1";
    "dewpoint_f" = "51.9";
    "dewpoint_string" = "51.9 F (11.1 C)";
    "heat_index_c" =     {
    };
    "heat_index_f" =     {
    };
    "heat_index_string" =     {
    };
    "history_url" = "http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KCADALYC1";
    image =     {
        link = "http://wunderground.com/weatherstation/";
        title = "Weather Underground";
        url = "http://icons.wunderground.com/graphics/bh-wui_logo.gif";
    };
    location =     {
        city = "Daly City";
        elevation = "514 ft";
        full = "Mussel Rock, Daly City, CA";
        latitude = "37.667347";
        longitude = "-122.489342";
        neighborhood = "Mussel Rock";
        state = CA;
        zip =         {
        };
    };
    "ob_url" = "http://www.wunderground.com/cgi-bin/findweather/getForecast?query=37.667347,-122.489342";
    "observation_time" = "Last Updated on November 7, 1:55 PM PST";
    "observation_time_rfc822" = "Sat, 07 November 2009 21:55:51 GMT";
    "precip_1hr_in" = "0.00";
    "precip_1hr_metric" = "0.0";
    "precip_1hr_string" = "0.00 in (0.0 mm)";
    "precip_today_in" = "0.01";
    "precip_today_metric" = "0.0";
    "precip_today_string" = "0.01 in (0.0 mm)";
    "pressure_in" = "30.07";
    "pressure_mb" = "1018.2";
    "pressure_string" = "30.07\" (1018.2 mb)";
    "relative_humidity" = 83;
    "solar_radiation" = "482.00";
    "station_id" = KCADALYC1;
    "station_type" = "Fan-aspirated Davis Vantage Pro 2 Plus";
    "temp_c" = "13.9";
    "temp_f" = "57.0";
    "temperature_string" = "57.0 F (13.9 C)";
    weather =     {
    };
    "wind_degrees" = 342;
    "wind_dir" = NNW;
    "wind_gust_mph" = "24.0";
    "wind_mph" = "18.0";
    "wind_string" = "From the NNW at 18.0 MPH Gusting to 24.0 MPH";
    "windchill_c" =     {
    };
    "windchill_f" =     {
    };
    "windchill_string" =     {
    };
}

That looks good, we can see the wind information near the bottom of the output, with keys "wind_degrees" and "wind_mph". So let's grab the values for those keys using the collect Higher Order Message and -objectForKey:.

> (parser parsedDataFromURL:'http://api.wunderground.com/weatherstation/WXCurrentObXML.asp?ID=KCADALYC1' ) collect objectForKey: #( wind_mph wind_dir wind_string ) each. 
21.0
NW
From the NW at 21.0 MPH Gusting to 24.0 MPH

Almost what we wanted, except that we grabbed the wind direction as a string instead of the exact numeric direction. Easy fix:

> (parser parsedDataFromURL: 'http://api.wunderground.com/weatherstation/WXCurrentObXML.asp?ID=KCADALYC1' ) collect objectForKey: #( wind_mph wind_degrees wind_string ) each.
12.0
318
From the NW at 12.0 MPH Gusting to 24.0 MPH
>

Perfect. We have the wind speed, the direction and an informative text in case we want to display that.

Blocked-C

Update: It appears that the original article has been removed, and has been superseded by material at: http://developer.apple.com/mac/articles/cocoa/introblocksgcd.html. The original article had more on the Cocoa block APIs and gave a refreshingly honest assessment of the for-loop vs. Block-iteration comparison.

While the news that Apple is adding blocks to C and Objective-C in the SnowLeopard time frame has been around for some time, a recent article shed some light on the actual API.

While there probably are some places where Objective-C blocks can be useful, I am not really impressed. In the following samples, red is used to show noise, meaning code that is just there to make the compiler happy.

---

NSMutableArray *filteredItems= [NSMutableArray array];
[items enumerateObjectsWithOptions:0 withBlock:
    ^(id item, NSUInteger index, BOOL *stop) {
        [filteredItems addObject:[item stringByAppendingString:@"suffix"]];
    }
];

---

As you can see, the version using blocks is very, very noisy, both syntactically and semantically, especially compared with the HOM version:

---

[[items collect] stringByAppendingString:@"suffix"];

---

No prizes for guessing which I'd prefer. To put some numbers on my preference: 234 characters vs. 52, 19 tokens vs. 3, 5 lines vs. 1. In fact, even a plain old C for-loop is more compact and less noisy than our "modern" blocked version:

---

NSMutableArray *filteredItems= [NSMutableArray array];
for (int i=0; i < [items count]; i++ ) {
     [filteredItems addObject:[items objectAtIndex:i] stringByAppendingString:@"suffix"];
    }
];

---

Why not Objective-C?

Patrick Logan can't understand why projects use C++ rather than Ojective-C. Neither can I.

For the 95% (or more) of code that isn't performance sensitive, it gives you expressiveness very close to Smalltalk, and for the 5% or less that need high performance, it gets you the performance and predictability of C.

Cocoa(touch) memory management is as easy as 1-2-3

There is a common misconception that Cocoa memory management is hard. It's not.

1. Use auto-generated accessors religiously
2. Release your instance variables in dealloc
3. Always use convenience methods to create objects

Wow, that wasn't too hard!

Objective-XML-5.0.1

Just pushed out a minor bugfix release to Objective-XML-5.0:

• Re-enabled character-set conversion code that had gotten disabled
• Fixed a compile-error for some targets
• Other minor improvements

Download here: http://www.metaobject.com/downloads/Objective-C/Objective-XML-5.0.1.tgz

Objective-XML 5.0

I've just pushed out a new release of Objective-XML, with some pretty significant new features.

Incremental parsing

This feature, which was already discussed a little in an earlier post, is now available in an official release. In short, Objective-XML will now stream data from network data sources (specified by URL) and produce results incrementally, rather than reading all of the data first and then parsing it. This can make a huge difference in responsiveness and perceived performance for slow networks. CPU and memory consumption will be slightly higher because of extra buffering and buffer stitching required, so this should only be used when necessary.

Static iPhone library

Although Objective-XML has always been compatible with the iPhone, previous releases required copying the pre-requisite files into your project. This burden has now been eased by the inclusion of a static library target. You still need to copy the headers, either MPWMAXParser.h or MPWXmlParser.h (or both).

Unique keys

Previous releases of Objective-XML had an -objectForTag:(int)tag method for quickly retrieving attribute or element values.

---

enum songtags {
  item_tag=10, title_tag, category_tag	
};
...
  [parser setHandler:self forElements:[NSArray arrayWithObjects:@"item",@"title",@"category",nil]
          inNamespace:nil prefix:@"" map:nil tagBase:item_tag];
...
-itemElement:(MPWXMLAttributes*)children attributes:(MPWXMLAttributes*)attributes parser:(MPWMAXParser*)p
{
   ...
   [song setTitle:[children objectForTag:title_tag]];
   ...

---

Objective-XML adds an -objectForUniqueKey:aKey method that removes the need for these additional integer tags.

---

...
  [parser setHandler:self forElements:[NSArray arrayWithObjects:@"item",@"title",@"category",nil]
          inNamespace:nil prefix:@"" map:nil];
...
-itemElement:(MPWXMLAttributes*)children attributes:(MPWXMLAttributes*)attributes parser:(MPWMAXParser*)p
{
   ...
   [song setTitle:[children objectForUniqueKey:@"title"]];
   ...

---

In addition to providing faster access, the integer tags also served to disambiguate tag names that might occur in multiple namespaces. To handle these conflicts, there now is a -objectForUniqueKey:aKey namespace:aNamespace method. The namespace objects required for this disambiguation process are now returned by the -setHandler:... and -declareAttributes:... methods, which were previously void.

Default methods

One of the attractive features of DOM parsers is that they do something useful "out of the box": point a DOM parser at some XML and you get back a generic in-memory representation of that XML that you can then start taking apart. However, once you go down that road, you are stuck with the substantial CPU and memory overheads of that generic representation.

Streaming parser like SAX or MAX can be a lot more efficient, but it takes a lot more time and effort until achieving a first useful result. Default methods overcome this hurdle by also delivering an immediately useful generic representation without any extra work. Unlike a DOM, however, this generic representation can be incrementally replaced by more specialized and efficient processing later on.

Cocoa HTML parsing with Objective-XML

Although Objective-XML's MPWSAXParser mostly provides NSXMLParser compatibility it also provides a number of useful additional features. Among these features is the ability to parse HTML files via the settings of two flags: enforceTagNesting and ignoreCase. By default, these are on and off, respectively, which gives you strict XML behavior. However, by setting enforceTagNesting to NO and ignoreCase to YES, you get a SAX parser that will happily and speedily process HTML.