Pattern Muscle Memory

Sometimes I catch myself applying a particular pattern in an architecture / application design almost without consciously thinking about it. One of the side benefits of flailing around in Cocoa and Objective-C is the disruption of my “pattern muscle memory”.

A recent example is in the implementation of the model layer of the graphical authoring tool I’m working on. One very basic required feature is the ability to delete a graphical element. The element can be deleted in the usual ways: the user pressing the delete key, the user selecting delete from the edit menu, etc. These actions ultimately invoke the same code and remove the item from the data model. There are other ways that the element might be deleted too: as a result of a cut operation, as the result of a synchronization or merge operation with an external source (like importing data from another file), etc.

My pattern muscle memory would normally have me tackle the ability to undo/redo this operation using classic “gang of four” style command and memento patterns; creating concrete implementations of an abstract command for each action.

In this case, being a new Cocoa programmer, my C#/.NET pattern muscle memory was disrupted by the NSUndoManager’s cool ability to package up “ordinary” invocations rather than requiring any special sub-classing or glue code. Now, I could certainly implement this kind of thing in .NET using reflection, etc, but I would have to implement it rather than using it “off the shelf”. Besides, my pattern muscle memory would have kicked in before I really had given it too much thought.

For my delete implementation, I chose to use one of the common Cocoa approaches exemplified in Aaron Hillegass’ book of using KVO to observe the removal of the object from it’s collection and registering it’s inverse (an insertion) on the undo stack. When KVO tells me that the object was added, I register the inverse of that operation (a deletion) on the undo stack.

One common challenge I’ve faced in the past is what to do about side effects or otherwise dependent actions of the principal action occurring. In the case of deleting my graphical object, there are several side such effects. One example is the need to modify connected or constrained graphical objects. In certain situations, an extreme side effect is the need to delete or create related objects, allocate unique identifiers, etc.

My pattern muscle memory typically has me encapsulate these dependent actions along with the implementation of my original user operation, in this case, the delete. One complication of this approach is that as the application scales and multiple developers work on the code base, it can be hard to ensure that everyone consuming the model uses the same high level delete implementation.

In the case of the delete key, the delete menu item, etc, it’s dead simple to make sure the same high level code (with it’s encapsulated side effect behavior) is invoked. As more complex features (such as the merge example) find themselves needing to delete the graphical object (along with performing all of it’s side effects) the requirement to use the common high level delete implementation can get obscured. Especially when the various types of consumers haven’t been anticipated in the design of the model’s API. I’ve found the murkiness (and naughtiness) increases when a generalized data model framework is being used, as it invites direct access to manipulating the data model via it’s APIs. Such is definitely the case with Core Data.

In the past, my pattern muscle memory has had me implement intermediate model layers that encode the side effects, but it has always seemed like such a waste of time and effort to obfuscate direct access to the generalized data model framework APIs with intermediate layers.

In my Cocoa implementation, I choose to implement my data model side effects in KVO observers. I did this so that deleting my graphical element using the Core Data API results in the requisite side effects and dependent data model changes. The main reason I did this is to avoid requiring special discipline or knowledge of the side effects on the part of the consumer of the model. My KVO observers however, also implement the undo/redo registration as I touched upon above. The problem with this approach when naively done is the exponential accumulation of invocations in the undo and redo queues. Each KVO notification from successive undo and redo operations results in performing the side effects and dependent operations anew (in addition to the queued undo/redo invocation).

This is easily rectified by checking to see if our KVO notification is happening as a result of an undo/redo operation (i.e. [NSUndoManager isUndoing] or [NSUndoManager isRedoing]) before performing side effects and dependent operations. If we are in the midst of an undo or redo, we let the invocations on the queue take care of the side effects and dependent operations, but don’t ourselves reissue them as a result of the KVO notification.

Sure I could have implemented all this in C# / .NET, or Java or JavaScript for that matter, but it’s not set up “out of the box” for it, and besides, my pattern muscle memory would have kicked in first anyway.

The bottom line result is a clear and compact model implementation that is easier to implement, extend and maintain over time, one that does not constrain it’s consumers to specialized APIs or require specialized knowledge.

Well, at least that is the theory… Time will tell, but for now I am love-hating my pattern muscle memory disruption.

Published in: on May 20, 2009 at 11:28 pm  Comments (1)  
Tags: , , , ,

Faulting Fun

Managed memory is awesome. Generally speaking, I would rather gouge my eyes out than go back to manually allocating memory.

It’s not that I can’t do it. I’ve been programming for long enough that I started out manually managing memory in the progression from 6502 assembly to C to C++, etc. True, the past few years I’ve been much farther away from embedded system code than I used to be in the days of the Super Nintendo, Atari Jaguar, PlayStation, et al. It’s just that I’d rather spend my time expressing “big ideas” (cue eye rolling now) than tweaking out string memory management on my sub three pound laptop with 4GB of memory.

I’m not sure I would be able to handle Cocoa and Objective-C programming without automatic memory management, but one recent friction point in my love affair with the Cocoa APIs is the dawning realization that there are a ton of subtle API architecture and implementation choices that were made before the move to Objective-C 2.0 and managed memory.

I’m not talking about fundamental incompatibilities between Cocoa API architecture and managed memory. I’m just talking about stumble inducing speed bumps.

A recent example I’ve tripped over is in combining Core Data and KVO. I am using both raw KVO and data binding heavily to glue together Core Data with the view model and view of the application I’m working on. In one instance, when a Core Data document (well, really an associated NSManagedObjectContext) is loaded, I execute a fetch request to obtain a set of peer “root” NSManagedObjects. I then register to observe some relationships and attributes on the resultant NSManagedObjects via KVO and then continue on with the rest of the application.

Later during the program, when new “root” NSManagedObjects of this particular entity type are created, I do the same thing (i.e. register to observe some relationships and attributes using KVO).

When attributes on these recently added “root” objects are changed, the user interface reflects the changes as expected, however when attributes on loaded “root” objects are changed, the user interface does not reflect the changes. WTF right?

After a curse filled hour or two (and copious amounts of caffine for focus) the problem turned out to be with my original fetch request and register using KVO. When I performed the fetch, I was not keeping a reference to the NSManagedObjects I obtained. Eventually after a garbage collection cycle or two, those initial NSManagedObjects were collected, and the underlying Core Data entity instances were turned back into fault (i.e. are dumped from memory). A very cool feature of Core Data, but very not ideal in this case.

When I later executed a fetch request to obtain one of these initially loaded “root” NSManagedObjects, Core Data created new NSManagedObject instances for me, bringing the data they represent back into memory from the persistent store. At this point, they are NOT the same NSManagedObject instances that I registered to observe with KVO (those instances are gone) because KVO did not keep strong references to the original objects, and neither did I.

Bottom line, while I am conceptually changing values on the same Core Data entity instances that I initially registered to watch with KVO, they are not actually the same NSManagedObject instances (thanks to garbage collection and the efficiency / coolness of Core Data dumping data from memory that’s not currently needed). Hence the lack of change in the user interface. The observed objects didn’t change, they were garbage collected!

In this case, it’s an easy fix; I just retain strong references to my “root” NSManagedObjects. I don’t really want these objects coming in and out of memory anyway.

I suspect had garbage collection been in heavy use by the designers and creators of KVO and Core Data when they were conceived and implemented, there might have been some subtle changes to their approach. Or maybe at the very least to the documentation.

I’m not really complaining; both of these systems are damn cool and when I’m not learning how to avoid tripping over my own feet, they are accelerating the crap out of my development.

Here’s looking forward to the days of less tripping per stride!

Published in: on May 19, 2009 at 6:19 pm  Comments (4)  
Tags: , , , ,

Core Data Rough Riding

Over the years, I’ve written a few different data models.  Some of them were even (gasp) documented!  Taking the time to nail the data model stack early on in app development has really helped with overall development efficiency and the ability to roll with the punches.  The more recent hand implemented data model stacks evolved to have some common features:

  • Data driven schema definitions
  • Some form of relational data representation
  • Files are first class entities
  • Ability to perform incremental loads  / be made partially resident
  • The ability to represent  inter-file entity references
  • Support for Undo / redo support
  • Support for Change tracking / change management
  • XML file format

Moving from the .Net world to Cocoa the obvious data model stack choice was Core Data.  Super promising and drool worth feature list!

  • Data driven schema definitions
    • Yup, can do.
  • Some form of relational data representation
    • Yup, can do.
  • Files are first class entities
    • Nope, you have to roll your own file referencing, but no biggie.
  • Ability to perform incremental loads  / be made partially resident
    • As long as you use SQLite as your backing store, but that’s cool.
  • The ability to represent  inter-file entity references
    • Nope, you have to roll your own, but no biggie.
  • Support for Undo / redo support
    • Yup, can do (or at least so I thought).
  • Support for Change tracking / change management
    • Yup, can do with KVC/KVO (or at least so I thought)
  • XML file format
    • Not if you want incremental load / save, but SQLite is hand debuggable enough.

Learning curve aside, the happy turned to sad as I started running into what turned out to be pretty common problems with Core Data’s support for undo.  Cocoa has a pretty slick undo paradigm.  Not exactly gang of four, but pretty cool.  The problem comes with the way Core Data interacts with it.  After a few days of spelunking and pain, I built enough vocabulary to ask Google the right questions, and the answers were not promising.

Take this blog post by Will Shipley (the Delicious Library guy):

“It’s pretty obvious I should be managing my OWN undoManager, turn off the one in CoreData, and just use CoreData for what it is EXTREMELY good at, which is minimal change tracking and fetching and storing data VERY VERY quickly.”

And then this other blog post by Mike Abdullah:

“Turning off undo registration doesn’t remove your changes from this system, it just stops the undo manager hearing about them. The change is still undone the moment the user wants to go back to a point before you made the change.”

“Sadly I don’t have a complete solution yet, but continue to ponder it. Do you have an idea? Get in touch, I’d love to hear from you.”

The kinds of problems I ran into were having Core Data KVO notify me in a redo that a relationship was being reconnected, but a query on the NSManagedObject in question would not yet be populated with any fields that I could use to index / sync with my controller or view representation.  Trying to force Core Data to sync with the undo system in bite sized chunks (i.e. subscribing to NSUndoManager notifications to hear about begin/end undo groups and calling NSManagedObjectContext’s processPendingChanges) to guarantee ordering doens’t work reliably (i.e. that insert object->set value->add gets undone in reverse order, and then redone in forward order).

Digging in further, CocoaDev mailing lists posts from purported Apple engineering dudes seem to indicate really isn’t much guarantee that Core Data will fire off KVO in undo/redo in rational orderings.  The problem is related to what Mike Abdullah describes in his blog post above: that Core Data implements undo / redo like commits and rollbacks to an SVN repository. Reading a few other blog posts like Will’s regarding spending years fighting Core Data’s undo system, I opted to turn it off and register my operations with NSUndoManager manually.  Unfortunately, my first attempt failed miserably.

My first approach was to capture the insertions, property changes, set additions/removals and deletions in the NSUndoManager.  A promising method on NSManagedObjectContext “insertObject” seemed like it would happily reverse “deleteObject”.  Unfortunately, instrumenting the save process and inspecting the NSManagedObjectContext’s pending insertions, deletions and registered objects proves this to be false.  Some more Cocoa Dev posts seem to confirm that while this will work with an XML backed persistent store, it won’t work with an SQLite backed store.  Without the ability to reverse an insert without recreating a new NSManagedObject, the tactic of capturing subsequent property changes and set additions/removals was mooted.  Time for my second approach.

My second approach was still to capture insertions, property changes, set additions/removals and deletions in the NSUndoManager, but using a primary key on each NSManagedObject rather than depending on the NSManagedObject to remain constant.  In this case the primary key is a string representation of a UUID (hey, what can I say, I’m an ex-Microsoft programmer).  Now, each operation in the undo/redo queue incurs a fetch request to find the NSManagedObject it needs to operate on.  Sucesss!  This method (while not the most performant undo/redo in the world) works reliably.

The next problem I ran into was saving.  Seems simple enough, but when loading a document, editing, saving, editing and saving a second time, Core Data deadlocks.  WTF?  Again, Google had a not very promosing (albiet not fatal) answer for me in the way of a post from Apple dude Ben Trumbell:

“There’s a known issue on the current Leopard using NSPersistentDocument with an SQLite store under Garbage Collection. Change any one of the 3 pieces in that configuration and things will work fine. We will be addressing the issue.”

Sweet, and that was from 2007.  Well, there is no freaking way I’m not going back to unmanaged programming, and the features offered by SQLite are more than half the reason I’m interested in Core Data, so don’t let the door hit you on the way out NSPersistentDocument!

On the bright side after all this spelunking, it was only about a ten minute task to create a (minimalistic) alternative implementation of NSPersistentDocument supporting Save, Save As, etc.

Published in: on May 7, 2009 at 2:34 am  Comments (1)  
Tags: , ,
Follow

Get every new post delivered to your Inbox.