a glob of nerdishness

It’s been a little over a year since Calf Trail Software, LLC was formed, and a nearly a year and a half since I started working at it full time. I must have learned a lot in those months, even though it always seems I’ve barely scratched the surface of writing software and running a small business.

Mercatalog was my first project, and its biggest lesson was the cost of too much research. I spent a year working out solutions to dozens of problems, in the scope of just one release. Before I had barely begun to succeed, the application became largely unneeded.

From that experience, and from that code, another app was born. Geotagalog is already the easiest tracklog-based geotagging utility on the market, and will only get better through its next major version. Watching Geotagalog gradually gain fans has been teaching us patience and persistence.

…”us” being me and Calf Trail co-founder Jon Hjelle. (And my wife Hannah, who has been helping us with our artwork, and learning patience while we try to make something resembling a living off of this calling!) While I’ve been focusing on the desktop apps and the website, Hjon has been busy developing Resistulator, Cheap GPS and Inbugz, as well as keeping our store and servers running.

It’s been a good year in many ways, though we have yet to even recoup all of our initial expenses. We’ve learned enough to make some good products available, but are still learning how to let people know about them. Besides continuing to grow as programmers and marketers, it’s quite likely we will also offer our iPhone and Mac programming skills as contractors in the coming year.

So there’s a lot of excitement in just my job these days, but that’s not all. I hope to share details regarding another source of excitement and responsibility here soon!

If you follow the Apple news/rumor sites, you’ll have heard that Apple has been (pre-)releasing software like crazy lately. I can neither confirm nor deny these rumors. Between my marginal ISP and the dank, dusty cellars underneath Apple’s public website, I get plenty of half-finished large downloads regardless of what Apple is up to.

When Safari thinks it’s finished downloading an incomplete file, I like to use curl to finish. My usual trick isn’t enough for the Apple Developer Connection’s password-protected files, and it took me a while to figure those out. Eventually, with the help of two helpful posts and some trial-and-error, this is the method I’ve settled on:

1. Copy partially downloaded file if it’s fairly large (because sometimes I mess up)
2. Log into ADC in a browser, and copy the download link
3. curl -c $(mktemp -t curl) -L -o '<Partially Downloaded File>' -C - '<Download URL>'

It is very important to quote the Download URL, because it may contain special shell characters! This is the main reason I copy the partial download; I don’t want to blow away a large partial download with a bit of error page HTML.

This works as follows: -c $(mktemp -t curl) tells curl to keep track of cookies in a temporary file, and -L means to follow redirects. The downloaded output file is specified via -o and -C - means to continue at the offset where this output file stops. I don’t include an option to automatically retry on failure, because if the session expires this could lead to a loss of the newly appended data if the download is replaced by an error page.

Recently the topic of selections in Cocoa apps came up again. While most of the discussion has centered around selections in text, the majority of sane developers use AppKit’s built-in text views and leave it at that. However, when we were developing Mercatalog’s interface over at Calf Trail, we needed to handle mouse selection and dragging of photo icons in a consistent way. Not only did our map and timeline layers need to behave similarly, they had to work like the rest of the operating system.

Cocoa leaves selection handling completely up to each individual view. As our view code developed, we realized that this task was less trivial than it first seemed. Playing around with various views in other apps to determine the “correct” behaviour was even less encouraging — there are many selection idiosyncrasies among Apple’s applications and views.

So we tried to solve the problem once and for all, in a separate class intended to manage selection and drag source behaviour for any view/cell/layer that needs it. It’s mostly geared towards mouse selection at this point, but Calf Trail is making it available as open source in the hopes it will spark discussion and progress towards good selection behaviour. You can find details and source for TLSelectionManager on Google Code.

I tend to let perfectionism get in the way of progress in my software development. This can happen on many levels:

• coding *too* carefully, not leaving any time to write tests or leave good notes
• delaying new code to overdo structure, even though refactoring is always inevitable (and usually fun!)
• overplanning an architecture, in ignorance of all the problems it will *actually* have to solve
• constantly trying to re-”decide” if a product is worthwhile, instead of seeing what users think
• worrying about mastery in a given domain, before even setting foot in it

This isn’t to say that I don’t try to do my best at all levels. It’s to say that time spent practicing — time spent making mistakes and learning from them — is rarely time wasted. Sometimes I catch myself spending as much time trying to make the best decision as it would take to actually try one good option or another. Other times I don’t catch myself!

Is perfect the enemy of better? Grace is someone willing to pay for my mistakes themselves. I see many of my weaknesses as a programmer and designer, I know there are more I don’t see, and I’m not proud of any of them. My pride ends up working against itself, though. There is just enough grace in the world for me to waste time not making mistakes when I should be making things better.

Antonio Rodriguez, of the excellent Tabblo service fame, asks “Why does sync still suck?” without giving an answer. Sync is going to become more and more important as we use more software on more devices from more places. I’m a little hesitant to say this, because a) it might be only true for us nerds in our gadget-filled bubbles and b) Internet access may soon become completely ubiquitous for said nerds. But if the nerds ever figure out sync they’ll be really eager to share it with the rest of the world.

It’s frustrating that sync really belongs in the operating system, or even in standards above the operating system level, but for various reasons even where it is present it’s tough to really justify the extra pain of working with it. Apple does have a system framework (Sync Services) that is available to third party developers, but it is not designed to handle large chunks of media. It can’t be used to sync with iPhones, and MobileMe’s continuing bad reputation gives it a bad stigma as well. Maybe Microsoft has their own solution, but it probably has its own problems and certainly wouldn’t be compatible with Apple’s, or Google’s…

To answer Antonio’s question: Sync still stinks because file formats still stink and file system models still stink. And, in practice, the web hasn’t solved many of the underlying problems either. Standard formats lead to stagnation, proprietary solutions lead to fragmentation. File systems are stagnant, and file formats are usually fragmented. I’m very interested to learn how these difficulties can be overcome, but after so many years of not solving these problems, I don’t expect it will be easy.

Michael J. Tsai:

“NSError, ugly though it is, actually seems to be a pretty good system for managing and propagating errors generated by different layers of code and even different languages.”

When I first read that, I was taken aback. The way Cocoa uses NSError is almost exactly like exceptions, but with no language support. Instead of getting a return value OR an exception when calling a function, you need to declare storage for both the return value AND the error. With NSError, the return value must also be of a type with “room” for an invalid flag, like a pointer that can be nil or a “succeeded” boolean that can be false. As if the lack of tuples in C wasn’t occasionally frustrating enough, now even many structures must be returned by reference rather than value just because they weren’t able to convey the “failed, check error” message. Even generating an NSError is a pain, and handing a lower one up the stack isn’t much more fun.

But in the end I think Michael is right. In pure C++, where idiomatic code leads to the stack itself doing most of the memory management, exceptions are great. I’d expect similar benefits in completely garbage collected languages as well. Cocoa on the other hand is such a mixed bag: reference counting with varying amounts of autorelease, or garbage collected; with C API usage encouraged, C++ contamination allowed (and fairly well supported as of Leopard). Then factor in the growing trend of bridging Cocoa into other languages. Having to bubble up errors through an ugly old explicit error mechanism starts to look a bit more appealing.

The following tlnum() macro (which I dedicate to the public domain), will allow you to easily create NSNumbers from constant floats and integers in your Objective-C code:


#define TL_FLOATARG(x) ( (x) ? ((x) / (2 * (x))) : (((x) + 1) / 2) )
#define tlnum(x) (TL_FLOATARG(x) ? [NSNumber numberWithDouble:(x)] : [NSNumber numberWithLong:(x)])


You could even rename tlnum() to simply N() if it won’t conflict with anything in your code. Update: Mike Ash pointed out that the TL_FLOATARG trick will not work if 2 * (x) overflows. So until a better macro can be developed, use this with care.

Why would you use this? Well, Objective-C lets you make a constant NSString-like object at compile-time @”like this”. This is extremely handy, as the next easiest way to create a string object is [NSString stringWithUTF8String:"like this"]. Since string objects are used all over the place — as format specifiers, when querying for localized strings or application resources, in any sort of situation where a key is needed… — the @”" syntax saves a lot of typing! Constant number objects aren’t needed quite as often, but almost every Cocoa programmer has at some point wanted to create constant NSNumber-like objects in a similarly succinct and easy way. Using this macro lets you do just that.

While it would be impractical (if not impossible) to create truly constant NSNumber-like objects at compile time using preprocessor macros, I still wanted something that would allow me to replace code like [NSNumber numberWithInt:42] and [NSNumber numberWithDouble:3.14] with something shorter. Furthermore, I wanted *it* to worry about the number’s type for me. I didn’t want to have to call a different macro depending on the number, as this would be require more thought for each use and maybe even more typing.

My first thought was to do something like this, following similar tricks that Dave Dribin used for his to-NSString converter:

// WRONG!
#define tlnum(x) ({ typeof(x) _x = (x); [NSNumber value:&_x withObjCType:@encode(typeof(_x))]; })


This has two problems. For one, the NSNumber subclass does not override -[NSValue value:withObjCType:] and so the object created does not work as an NSNumber at all! I worked around this with my own function that checked the encoded type, but that seemed like overkill. The second problem is that even with a helper function, the code depends on GCC extensions for the block macro and the typeof() operator. I wanted to keep my code within the C99 standard.

Since I was intending this for constant numbers only, and since NSNumber (or at least the CFNumber implementation in the most recent CFLite distribution) uses at least 8 bytes of storage for even chars and bools, the only important distinction is between floats and integers. Thus the TL_FLOATARG, which takes advantage of the fact that 1 / 2 yields the integer 0, while 1.0 / 2 yields a floating-point 0.5. The one gotcha with the final version is that it evaluates its arguments multiple times. My compile settings give a warning, but be careful not to do something like tlnum(i++), as you’ll end up incrementing i multiple times. Jens Ayton kindly provided another macro trick to turn this into an error, but in the end I opted to leave it out for simplicity.

Please enjoy, and I’d be glad to know if this is helpful!

One of the great new features added to Objective-C in version 2.0 was property declarations. While I’m not a big fan of the dot syntax, the ability to clarify the semantics of setter methods is a big win in my opion. With old-style setters, one was rarely sure if the value passed would be copied, retained, or simply referenced. The other neat feature of properties is that the compiler can provide instance variable storage automatically when compiling for the “modern” ObjC runtime. So you don’t have to declare an instance variable matching (or matched to) a fully-synthesized property if you’d prefer not to.

If you intend to take full advantage* of synthesized variables in the future, you must never use the instance variable directly. Unfortunately, if you’re also compiling an application that needs to run on 32-bit Leopard (as is pretty likely these days whether you’re supporting a PowerPC desktop app or testing an iPhone app in the simulator) you will need to declare these storage for your properties for the sake of those builds — Leopard needed to be backwards compatible with the “legacy” ObjC runtime, and the compiler relies on runtime support to implement this ObjC 2 magic. This conflicts with best practice for init/awake and dealloc methods, which should prefer to use the instance variable directly to avoid unwanted side effects of custom setters (undo, notifications, redisplay, etc.). If you’re in the habit of using instance variables directly in other places, you may need a little help from the compiler to remind you that the ivar may not exist.

What we might want is a set of macros that does the right thing in these special circumstances, but yet will cause compiler errors during universal builds for any other direct use of the instance variables:

/* The following macros allow one to mark uses of declared instance variables
 that could be replaced by automatic property storage when only the modern ObjC
 runtime needs to be supported.

 TL_SYNTHESIZABLE_IVAR_DECL is used to declare the instance variable.
 TL_SYNTHESIZABLE_IVAR_SET should be used only in init/awake implementations.
 TL_SYNTHESIZABLE_IVAR_RELEASE should be used only in dealloc implementation. */
#ifdef __OBJC2__
#define TL_SYNTHESIZABLE_IVAR_DECL(type, name)
#define TL_SYNTHESIZABLE_IVAR_SET(name, value) (self.name = value)
#define TL_SYNTHESIZABLE_IVAR_RELEASE(name) (self.name = nil)
#else
#define TL_SYNTHESIZABLE_IVAR_DECL(type, name) type name
#define TL_SYNTHESIZABLE_IVAR_SET(name, value) (name = value)
#define TL_SYNTHESIZABLE_IVAR_RELEASE(name) ([name release])
#endif /* __OBJC2__ */

You would use them like this:

@interface MyClass : NSObject {
@private
	TL_SYNTHESIZABLE_IVAR_DECL(id, myVar);
}
@property (nonatomic, retain) id myVar;
@end

@implementation MyClass
- (id)init {
	self = [super init];
	if (self) {
		id defaultValue;
		TL_SYNTHESIZABLE_IVAR_SET(myVar, defaultValue);
	}
	return self;
}
- (void)dealloc {
	TL_SYNTHESIZABLE_IVAR_RELEASE(myVar);
	[super dealloc];
}
@synthesize myVar;
@end

*Now the real question is, what advantage is there to synthesized instance variable storage? In short, you’re able to declare the property in one place only, which results in less duplicate code. So why would you use these macros, when they require you to declare the property twice anyway? I’ll tell you: I don’t know. If you like to mark “deprecatable” code, TL_SYNTHESIZABLE_IVAR_DECL will be handy. The other two could be replaced by their __OBJC2__ setter equivalents with no real harm. Regardless, here they are, and I dedicate them to the public domain. If you think they’d be helpful, feel free use them at your own risk.

The days are getting shorter and the nights are getting colder; 2008 is almost over. The work at Calf Trail is progressing well, but there’s still a lot we’d like to get done before calling it a year. We’ve released two and a half alpha versions of Mercatalog, our Mac geotagging application, and are trying to get to beta as soon as possible. Letting other people finally try Mercatalog has been a great experience, though quite humbling. I feel like I’ve learned so much this year, and hope to eventually find time past this 100th post here to share more thoughts and more code!

Our little two man company just got back from our second trip to attend Seattle Xcoders. Once again, we had great conversations with friends and developers there. Among ourselves on the way there and back, we talked about many hopes and ideas for our company’s future. We discussed the temptation to let overlofty ideals about how software “should be” hamper the smaller goals that must first be achieved along the way. And we imagined ourselves writing great software that helps people do great things, built on top of a great platform.

Then we came back to this bad news. Apple have decided not to reward yet another developer’s hard work, this time on a very legitimate iPhone application. When the iPhone developer’s kit was announced, developers were warned that certain limitations would be enforced — Apple wouldn’t allow apps that were (for example) malicious, violated users’ privacy, “illegal” and other such things to be sold. This generated some immediate speculation about how Apple would use this power they’d reserved. After months of having to work under an NDA that would forbid developers from learning this new operating system as a community, now comes this confirmation of a once-latent fear. It’s a tough struggle to turn ideas, time, money and other limited resources into a product that will improve user’s lives. Now Apple are using their control in a way that makes the expected return on this investment even lower, especially for competitors of Apple or Apple’s partners. They’ve set up a game where they are the only ones who can’t lose, unless customers and third party developers stop playing.

Already one talented developer has announced that he will not be writing any more iPhone applications until the rules are made clear to all contestants and the referees are no longer hidden behind one-way glass. Apple claim they are still “processing” our application into this Secret Sharecropper Society. Should we email them and tell them not to bother, that we don’t want any share in this? If we believe Apple’s market is unnecessarily risky for us (it is) and if we believe Apple’s censorship will hurt our customers (it will) and if we believe this kind of human-centralized control will end up harming everyone everywhere (it may) — if we believe all that — is it hypocritical of us to still want to make iPhone software?

Worse, it is also Apple who control the equally amazing desktop platform for which we have been writing software since before January. What if this is also the future Apple intend for Mac development? Will we be legally obligated to learn and grow alone and leave others to do the same? Will we be unable to properly test our code in real world conditions? Will we have no choice but to give a cut of all our revenue to yet another bureaucracy that thinks it knows what’s best? And after all that, will we occasionally have the freedom to sell our work simply taken away? If so, we’re making a very bad investment.

What should we do? In a number of countries, the World Wide Web still has many of the freedoms and opportunities that the Wild Wild West is rumored to have once held. But just as many covered wagons never made it, and the ones that did learned to do without the modern conveniences they once enjoyed, we like the comfort of the desktop and our software is better for having the plumbing and wiring a web cabin just doesn’t provide. If we stay on Apple’s good side, we might be able to make a lot of money. And except for the ideals we’d have to disregard, it would be honest money too.

The first version of our software will not do all we think it should do for our paying customers. We don’t intend to rip anybody off, and it’s not because we disrespect our users. It’s because we are learning, at least with software development, that sometimes it is impossible to get to where we want to be without making some practical compromises along the way. We’ll always try to avoid doing things the wrong way, but sometimes we don’t yet have the ability to do them the best way either. I wonder if more of life — or at least this business struggle we’ve found ourselves a part of — I wonder if it’s okay to invest in a system we don’t fully agree with, at least until we have the opportunity to trade in for something better? Our company would be grateful for your advocacy to the one who is really in control on our behalf, and I would appreciate reading your thoughts about these “compromises” as well.