a glob of nerdishness

October 28, 2009

A moderate improvement to Cocoa’s Key-Value Observing

written by natevw @ 3:55 pm

The background

Key-Value Observing (KVO) allows Objective-C instances to track changes in the properties of other objects. It is a powerful programming tool and is implemented in an interesting manner. But the API to use it is lamentable — Apple’s interfaces usually stand out as shining examples of object-oriented design; the interface for registering and unregistering a KVO observer feels like an accident that can’t even be fully documented.

It is something of a rite-of-passage for Cocoa programmers to to workaround the weaknesses they see in Apple’s provided KVO interface. The inability to unregister an observation without upsetting a potential subclass had bothered me, but only in a theoretical sense, so I mostly ignored it. When I started writing a Garbage Collected app, I realized the time for my workaround had come, and I finally wrote NSObject+TLKVO.

The foreground

As implied by the name, NSObject+TLKVO is a category on NSObject. It replaces Cocoa’s built-in KVO -addObserver and -removeObserver methods with ones that solve two problems:

  • Each class can safely start and stop observing a key path without disturbing its subclasses
  • To run under Garbage Collection, a class does not need to implement some rubbish extra reference counting to safely remove itself from observed objects

As a bonus, it’s not even necessary to remove TLKVO-registered observers in garbage collected code at all. Memory-managed code can conveniently unregister all observations for a given class with one line of code as well.

I keep mentioning how a “class” registers for an observation. This is the key to the TLKVO design: it doesn’t change the overall KVO mechanism, it just refines it by making registration/unregistration for a key path unique to an instance AND a Class, rather than just an instance.

The code

Using it can be very simple:

TLKVORegisterSelf(watchedObject, @"watchedPath.watchedKey", NSKeyValueObservingOptionNew);
TLKVORegisterSelf(watchedObject, @"anotherWatchedKey", NSKeyValueObservingOptionNew);
TLKVOUnregisterSelf(watchedObject, nil);


[watchedObject tl_addObserver:self ofClass:[MyClass class]
forKeyPath:@"watchedPath.watchedKey" options: NSKeyValueObservingOptionNew]
[watchedObject tl_addObserver:self ofClass:[MyClass class]
forKeyPath:@"anotherWatchedKey" options: NSKeyValueObservingOptionNew]
[watchedObject tl_removeObserver:self ofClass:[MyClass class] forKeyPath:nil];

It’s important to note that in second (non-convenience) example, I used [MyClass class] instead of [self class]. This distinction is key. At runtime, [self class] will return the class of the instance, which may be a subclass. Instead, TLKVO needs to know the class which you are currently implementing so it can properly unique the registration and properly target the change observation messages.

Observing the changes is basically, well, unchanged:

- (void)observeValueForKeyPath:(NSString*)keyPath
if (context == &TLKVOContext) {
NSLog(@"Observed %@ of %@", keyPath, object);
else {
[super observeValueForKeyPath:keyPath ofObject:object change:change context:context];

The only difference to what you would write with plain-old KVO is the use of &TLKVOContext instead of your own unique pointer. If your class’s method is asked to observe a change with a context of &TLKVOContext, you can be sure that it is due to an observer registration made by the same class.

The what it doesn’t do

If you thought the context pointer could be useful for passing some extra information to your change observing method, you might be disappointed by not being able to define it yourself. This omission is intentional. Due to the design of Cocoa’s KVO, you can’t do any introspection of the pointer until you are certain that it belongs to your class. If you want to use an NSDictionary instance as this context, you need to keep its pointer accessible to the change observing method anyway.

You might argue that a single class may want to divide itself internally into separate sections and use different unique pointers to help sort out changes observed by each section. In that case, you probably also need more fine-grained registration and unregistration. Your -observeValueForKeyPath method is probably getting pretty complicated, too. At this point, you might want to check out…

The further reading

Another potential shortcoming of Cocoa’s KVO architecture is that all observer notifications get delivered to a single method. This can be convenient if all the changes are related, but sometimes this method becomes a readability bottleneck instead.

Several other developers have shared their improvements to KVO:

Mike Ash explains the problems with KVO much more clearly than I have, and provides an NSNotificationCenter-style replacement. Note that this was written before Garbage Collection, and read the discussion in the comments as well.
Jonathan “schwa” Wight has extended Mike’s idea and made it work better with Garbage Collection and blocks.

Andy Matuschak presented a clean, simplified API for having a block fire for every change before 10.6 was released. I’m not sure if he’s released the code yet.

If you just want more information on using KVO as Cocoa provides it, check out Dave Dribin’s article on Proper KVO usage or this Apple framework engineer’s recommendations on the matter.

I may update this post if more KVO extensions or further reading come to my attention.

The random heading

September 2, 2009

Use Trail Location as a Core Location testing framework

written by natevw @ 8:55 am

Trail Location is a subproject I’ve been working on at Calf Trail. Its primary goal is to be a drop-in replacement for Core Location that can use position data from any location service.

This enables a number of new possibilities.

But first a demonstration

More interesting location data in the simulator

Testing a Core Location app in the simulator is really boring. Apple might eventually wire the simulator up to Snow Leopard’s Core Location implementation, but that would just move your app from Apple’s headquarters to yours. Trail Location lets you send your own test data, which your app receives just as from Core Location.

Trail Location includes a sample app for your device that can send locations to the simulator as you pan around the map. It should also be easy to make an app that plays back recorded location patterns.

Send real GPS data to an iPod touch during development

This one’s a bit more complicated, but worth mentioning. The first two iPod touch models (hopefully NOT the third, but we’ll see) don’t have GPS chips and rely exclusively on Skyhook positioning, which is based on visible wireless networks. With Trail Location, you can use a laptop and a GPS receiver for better results in rural areas. (I bought an Amod ABG 108 Bluetooth GPS mouse for about $20! Unfortunately, you can’t use a Bluetooth GPS directly even with a second-generation iPod touch, because it would degrade the core experience or something.)

Trail Location includes a sample Mac app that can pass locations from gpsd to a listening iPhone app via WiFi.

Better Core Location on the desktop?

Snow Leopard adds the Core Location API to Mac OS X. This is interesting, because while on current hardware the Core Location framework itself is still limited to Skyhook positioning, the rest of the Mac platform is much, much more developer friendly. Trail Location could serve as the glue that connects third-party apps to third-party GPS receivers using the same well-designed API that Core Location provides. Just like you can use your device to send data to Trail Location on the simulator, you could use an iPhone as a GPS mouse for a desktop app.

Eventually, I hope Trail Location can grow mature enough to be left in code that ships to end-users, giving more people the benefits of an open location platform. Currently, you should consider it as an alpha release, NOT ready for anything but debug builds. Right now the code is overbuilt (debatably) but under-implemented (definitely). We’ve released it now so that it can help other developers test their Core Location apps, and so that any progress towards the long-term goals can be influenced by other developers. Get the source at Google code; it’s shared under a non-viral license but your patches are welcome.

January 6, 2009

Exceptions versus NSError

written by natevw @ 10:07 pm

Michael J. Tsai:

“NSError, ugly though it is, actually seems to be a pretty good system for managing and propagating errors generated by different layers of code and even different languages.”

When I first read that, I was taken aback. The way Cocoa uses NSError is almost exactly like exceptions, but with no language support. Instead of getting a return value OR an exception when calling a function, you need to declare storage for both the return value AND the error. With NSError, the return value must also be of a type with “room” for an invalid flag, like a pointer that can be nil or a “succeeded” boolean that can be false. As if the lack of tuples in C wasn’t occasionally frustrating enough, now even many structures must be returned by reference rather than value just because they weren’t able to convey the “failed, check error” message. Even generating an NSError is a pain, and handing a lower one up the stack isn’t much more fun.

But in the end I think Michael is right. In pure C++, where idiomatic code leads to the stack itself doing most of the memory management, exceptions are great. I’d expect similar benefits in completely garbage collected languages as well. Cocoa on the other hand is such a mixed bag: reference counting with varying amounts of autorelease, or garbage collected; with C API usage encouraged, C++ contamination allowed (and fairly well supported as of Leopard). Then factor in the growing trend of bridging Cocoa into other languages. Having to bubble up errors through an ugly old explicit error mechanism starts to look a bit more appealing.

December 10, 2008

Cocoa NSNumber “constant” macro for Objective-C

written by natevw @ 1:22 pm

The following tlnum() macro (which I dedicate to the public domain), will allow you to easily create NSNumbers from constant floats and integers in your Objective-C code:

#define TL_FLOATARG(x) ( (x) ? ((x) / (2 * (x))) : (((x) + 1) / 2) )
#define tlnum(x) (TL_FLOATARG(x) ? [NSNumber numberWithDouble:(x)] : [NSNumber numberWithLong:(x)])

You could even rename tlnum() to simply N() if it won’t conflict with anything in your code. Update: Mike Ash pointed out that the TL_FLOATARG trick will not work if 2 * (x) overflows. So until a better macro can be developed, use this with care.

Why would you use this? Well, Objective-C lets you make a constant NSString-like object at compile-time @”like this”. This is extremely handy, as the next easiest way to create a string object is [NSString stringWithUTF8String:"like this"]. Since string objects are used all over the place — as format specifiers, when querying for localized strings or application resources, in any sort of situation where a key is needed… — the @”" syntax saves a lot of typing! Constant number objects aren’t needed quite as often, but almost every Cocoa programmer has at some point wanted to create constant NSNumber-like objects in a similarly succinct and easy way. Using this macro lets you do just that.

While it would be impractical (if not impossible) to create truly constant NSNumber-like objects at compile time using preprocessor macros, I still wanted something that would allow me to replace code like [NSNumber numberWithInt:42] and [NSNumber numberWithDouble:3.14] with something shorter. Furthermore, I wanted *it* to worry about the number’s type for me. I didn’t want to have to call a different macro depending on the number, as this would be require more thought for each use and maybe even more typing.

My first thought was to do something like this, following similar tricks that Dave Dribin used for his to-NSString converter:

#define tlnum(x) ({ typeof(x) _x = (x); [NSNumber value:&_x withObjCType:@encode(typeof(_x))]; })

This has two problems. For one, the NSNumber subclass does not override -[NSValue value:withObjCType:] and so the object created does not work as an NSNumber at all! I worked around this with my own function that checked the encoded type, but that seemed like overkill. The second problem is that even with a helper function, the code depends on GCC extensions for the block macro and the typeof() operator. I wanted to keep my code within the C99 standard.

Since I was intending this for constant numbers only, and since NSNumber (or at least the CFNumber implementation in the most recent CFLite distribution) uses at least 8 bytes of storage for even chars and bools, the only important distinction is between floats and integers. Thus the TL_FLOATARG, which takes advantage of the fact that 1 / 2 yields the integer 0, while 1.0 / 2 yields a floating-point 0.5. The one gotcha with the final version is that it evaluates its arguments multiple times. My compile settings give a warning, but be careful not to do something like tlnum(i++), as you’ll end up incrementing i multiple times. Jens Ayton kindly provided another macro trick to turn this into an error, but in the end I opted to leave it out for simplicity.

Please enjoy, and I’d be glad to know if this is helpful!

December 2, 2008

Using fully synthesized instance variables in Objective-C 2.0

written by natevw @ 7:12 pm

One of the great new features added to Objective-C in version 2.0 was property declarations. While I’m not a big fan of the dot syntax, the ability to clarify the semantics of setter methods is a big win in my opion. With old-style setters, one was rarely sure if the value passed would be copied, retained, or simply referenced. The other neat feature of properties is that the compiler can provide instance variable storage automatically when compiling for the “modern” ObjC runtime. So you don’t have to declare an instance variable matching (or matched to) a fully-synthesized property if you’d prefer not to.

If you intend to take full advantage* of synthesized variables in the future, you must never use the instance variable directly. Unfortunately, if you’re also compiling an application that needs to run on 32-bit Leopard (as is pretty likely these days whether you’re supporting a PowerPC desktop app or testing an iPhone app in the simulator) you will need to declare these storage for your properties for the sake of those builds — Leopard needed to be backwards compatible with the “legacy” ObjC runtime, and the compiler relies on runtime support to implement this ObjC 2 magic. This conflicts with best practice for init/awake and dealloc methods, which should prefer to use the instance variable directly to avoid unwanted side effects of custom setters (undo, notifications, redisplay, etc.). If you’re in the habit of using instance variables directly in other places, you may need a little help from the compiler to remind you that the ivar may not exist.

What we might want is a set of macros that does the right thing in these special circumstances, but yet will cause compiler errors during universal builds for any other direct use of the instance variables:

/* The following macros allow one to mark uses of declared instance variables
 that could be replaced by automatic property storage when only the modern ObjC
 runtime needs to be supported.

 TL_SYNTHESIZABLE_IVAR_DECL is used to declare the instance variable.
 TL_SYNTHESIZABLE_IVAR_SET should be used only in init/awake implementations.
 TL_SYNTHESIZABLE_IVAR_RELEASE should be used only in dealloc implementation. */
#ifdef __OBJC2__
#define TL_SYNTHESIZABLE_IVAR_DECL(type, name)
#define TL_SYNTHESIZABLE_IVAR_SET(name, value) (self.name = value)
#define TL_SYNTHESIZABLE_IVAR_RELEASE(name) (self.name = nil)
#define TL_SYNTHESIZABLE_IVAR_DECL(type, name) type name
#define TL_SYNTHESIZABLE_IVAR_SET(name, value) (name = value)
#define TL_SYNTHESIZABLE_IVAR_RELEASE(name) ([name release])
#endif /* __OBJC2__ */

You would use them like this:

@interface MyClass : NSObject {
@property (nonatomic, retain) id myVar;

@implementation MyClass
- (id)init {
	self = [super init];
	if (self) {
		id defaultValue;
		TL_SYNTHESIZABLE_IVAR_SET(myVar, defaultValue);
	return self;
- (void)dealloc {
	[super dealloc];
@synthesize myVar;

*Now the real question is, what advantage is there to synthesized instance variable storage? In short, you’re able to declare the property in one place only, which results in less duplicate code. So why would you use these macros, when they require you to declare the property twice anyway? I’ll tell you: I don’t know. If you like to mark “deprecatable” code, TL_SYNTHESIZABLE_IVAR_DECL will be handy. The other two could be replaced by their __OBJC2__ setter equivalents with no real harm. Regardless, here they are, and I dedicate them to the public domain. If you think they’d be helpful, feel free use them at your own risk.

July 22, 2008

Comparing Objective-C selectors

written by natevw @ 9:58 pm

The key feature Objective-C adds to C is the concept of sending messages to objects. In most lines of code, the messages and receiving objects are basically intertwined: [thisObject getsThisMessage]. But just as you can pass around an object without necessarily sending it any messages, you can also deal with messages without sending them to objects. This is done via message selectors, which can be stored in variables of the single SEL type.

Apple’s Objective-C reference doesn’t make very many guarantees about what is stored in a SEL. It is a mostly opaque structure, to be probed only by functions “in the know”. But one perfectly normal thing to do is to ask an object if it will respond to a particular message selector, or if the action associated with a particular selector is currently enabled. To answer questions like this, the object may want to check if the selector in question is equal to a known selector.

Therein lies the rub. Until 10.5, the Objective-C runtime didn’t provide any function to compare these opaque selectors. The documentation does say that “The runtime system makes sure each identifier is unique: No two selectors are the same”. While this is not completely clear, it is understood by even Apple’s sample code (eg) to mean that the contents of identical SEL structs were identical, and thus could be compared using C’s == equality operator.

Apple is extremely unlikely to change this behaviour lightly (Update: it won’t change), as it would basically break binary compatibility across much code regardless of what the documentation is actually promising. However, the fact that the Objective-C 2.0 runtime added a sel_isEqual() function seems to imply that they do kinda sorta maybe kinda reserve the right to start treating SELs as a truly opaque type. (Even then, I can’t think of a good reason why the bitwise-equivalence would change.) But all this aside…there is now a function we can use.

Unfortunately, the sel_isEqual() function is defined in <objc/runtime.h> — a header that Foundation does not include — and doesn’t exactly blend in with the kind of high-level code discussed here. I’ve filed a bug (rdar://6094233) requesting the addition of an NSEqualSelectors() function next to NSStringFromSelector() and NSSelectorFromString(), but in the meantime I’ve written some wrapper code that is both “legacy runtime” and Objective-C 2.0 compatible:


There’s not really anything to it, it basically just wraps an inline function around sel_isEqual() if available, or does the old == comparison if not. I thought about moving the runtime function wrapper implementation into its own code so as not to leave all the runtime functions in the namespace; this might be something to consider for your own use, but this version optimizes for low overhead. Hopefully within a few years, link-time optimization will be ours, but even then keeping everything in one header is probably the best way to distribute this tiny little wrapping hack.

April 25, 2008

Debugging Universal applications in Rosetta

written by natevw @ 11:02 am

Apple have a guide on debugging PowerPC binaries on an Intel-based Macintosh. Unfortunately, those instructions do not work if the application is a Universal Binary.

If Xcode’s build settings dialog hadn’t gone from kinda bad to utterly screwy, it might have been easiest just to temporarily edit your application target’s build settings to build just ppc. As an alternative, you can use the following steps to debug the PPC code of a Universal binary on an Intel Mac:

  1. First, make sure your Debug version of your application’s target is set to include the ppc architecture. It may be set to just the “Native Architecture” to speed up normal debug builds.
  2. You will need the OAH_GDB environment variable set. In bash, use: export OAH_GDB=YES.
  3. Instead of running your application directly as Apple’s reference states, use /usr/libexec/oah/translate /<path>/<your_application>.app/Contents/MacOS/<your_application> from the shell in which you set the OAH_GDB flag.
  4. Open another terminal tab and start gdb using gdb --oah.
  5. Instead of attaching to <your_application>, you will need to attach to the “translate” process that’s running your app. GDB may autocomplete the whole thing if you press Tab after typing just “attach “.
  6. Now use gdb’s c command to Continue (or start, in this case) execution. Prepare for extremely slow execution, since your app is running under both Rosetta and the debugger.

Note that only the odd numbered steps, especially steps 3 and 5, differ from Apple’s instructions. Since it’s a Universal Binary, you have to force PPC behaviour by manually starting your application in the Rosetta translator.

Whichever method (compiling ppc-only or running via translate) you like better, you will likely want to load your app’s symbols with the gdb command symbol-file /<path>/<your_application>.app/Contents/MacOS/<your_application>. If you’re like me and generally inept with raw gdb, you may find the tips and links Apple’s Using GDB for Testing article to be a good starting guide for learning your way around gdb. There are some caveats to debugging under Rosetta mentioned in the first Apple guide, especially that you can’t call functions. Of course, I didn’t realize that was even possible, so I guess I’ll have to try that when I’m debugging normally sometime!

January 10, 2008

Playing with Mail and Leopard’s Latent Semantic Mapping

written by natevw @ 6:35 pm

While clearing Mail.app’s junk mail folder, I might have accidentally deleted a non-spam message. I’ll never know for sure, but as a result I learned a bit more about Leopard’s cool new Latent Semantic Analysis framework that I’d been wondering about since the mailing list leaked back in November.

Mail stores its spam information in ~/Library/Mail/LSMMap2. In addition to the Latent Semantic Mapping framework, Apple also provides lsm, a command line utility that provides the same functionality (with a little better documentation, I might add). As described in the man page, you can use lsm dump ~/Library/Mail/LSMMap2 to get a list of all the words that Mail’s spam filter knows about. (Some words probably NSFW, of course!) The first column is how many times the word has appeared in a “Not Junk” message, and the second is the count in spam messages. The last line gives a few overall statistics: how many zeroe values there were versus total values, and a “Max Run” value I don’t understand.

Between this and CFStringTokenizer and its language-guessing coolness, Leopard provides some fun tools for playing around with text analysis. Hopefully someday I’ll have a bit more time to dig into it.

Until then (or rather, for my future reference), here’s a bit more information on Latent Semantic Analysis: how it works and how it differs from Bayesian classification.

I’ve also uploaded a really quick and dirty “playground” for testing out the hypotheses the documentation left me with: lsmtest.m

Update: Came across a more “explainy” article about both Mail and Latent Semantic Analysis over on macdevcenter.com.

November 20, 2007

Objective-C 2.0 Fast Enumeration internals

written by natevw @ 8:03 pm

Although the NSFastEnumeration Protocol reference is pretty obtuse, the linked article tells all you need to know about how fast enumeration is implemented. It’s not really all that complicated — or pretty, for that matter — but it nonetheless allows for (... in ...) looping over any conforming class. All that said conforming class has to do is give out a pointer to its own internal structures when possible, or, at worst, copy a handful of pointers at a time into some pre-allocated stack space.

September 15, 2007

[widget setPreference: for:] is not JavaScript!

written by natevw @ 12:08 pm

Widgets can get and set persistent preferences without much hassle. It’s as easy as calling widget.preferenceForKey(key), which will return the value set for the key.

However, sometimes this doesn’t seem to work, because the developer swapped the arguments when setting. The correct order widget.setPreferenceForKey(value, key) is swapped from, say, element.setAttribute(name,value). This one has gotten me way too many times when developing Dashboard widgets.

How did this bizarre ordering come about? I’m guessing it’s because the widget object is actually written as an Objective-C plugin and the developers felt obligated to make the method idiomatic in that language: (void)setPreference:(NSString*)value forKey:(NSString*)key. But the function name is rewritten between Obj-C and JS anyway — I wish they could have broken the idiom a bit and used (void)setPreference:(NSString*)key toValue:(NSString*)value as the method signature, for the sake of the end users.

(Updated to fix method names from original post.)