An Introductory Mac OS X Leopard Review: Developer Tools

Among all of the new and improved applications Apple delivers to users in Mac OS X 10.5 Leopard, some of the most important are those that developers can use to build applications of their own. If you're not a hard core programmer, Leopard also offers a variety of more accessible tools: Automator for scripting, DashCode for building widgets, and Quartz Composer for creating audio and visual plugins, screen savers, image filters for iChat and music visualizations for iTunes. Here's the trail Apple's developer tools have followed, and how Apple nearly killed itself several times by not providing adequate tools in the past.

The Early Origins of Apple Developer Tools

In the late 70s, Apple founder Steve Wozniak wrote the original BASIC interpreter for the Apple II. Like most early software, it originally shipped on audio tape and required tedious loading from a data cassette player. Once loaded, users could write their own programs. The ability to write those software programs contributed a lot of value to systems, as the base system couldn't really do anything itself. In 1979 VisiCalc arrived and delivered a the first really practical application for buying a home computer.

Wozniak never got around to delivering support for floating point math in his "Apple BASIC," so the company licensed a BASIC interpreter from a tiny operation from New Mexico called Microsoft. That product was called Applesoft BASIC, and Wozniak's earlier version became known as Integer BASIC.

Microsoft's version was very slow, creating a demand for an additional BASIC compiler, which Microsoft also supplied as a solution to the problem. Relying on Microsoft for its developer tools turned out to be a catastrophic problem for Apple, which signed itself into an eight year contract covering Applesoft Basic.

Apple's MacBASIC Disaster

As Apple's Macintosh project was beginning to take shape in 1981, the company decided that it should deliver its own programming environment for the Mac rather than again delegating the task away to a third party. Apple feared that outsiders might not "get" the new Mac user interface that it had been investing so much research and development into perfecting.

Microsoft's contract for Applesoft BASIC expired in 1985, just as Apple was preparing to release its own MacBASIC. Sure enough, when Microsoft delivered its own BASIC for the Mac, it used a console interface rather than taking advantage of the Mac's new graphical environment.

After discovering Apple was planning to release its own, superior MacBASIC, Bill Gates was livid. His Applesoft BASIC was destined to be obsolete within a few years, but he also knew Apple was still making most of its profits from Apple II systems. Gates exploited that fact to tie the renewal of the ongoing Applesoft BASIC contract into a deal where Microsoft would buy Apple's MacBASIC for $1 and sandbag it.

Gates also tied in two years of exclusive development of the new Excel for Macintosh but then demanded a perpetual license of Apple's Mac interface for use in Windows 1.0. Apple CEO John Sculley agreed to the deal, and the result was that Apple was again left without control over its own development tools, in addition to now being unable to stop Microsoft from eventually appropriating the entire Mac desktop over the next decade. Two years later, Microsoft ported Excel for Windows and Applesoft became largely obsolete; however, Apple end up stranded with poor development tools from Microsoft while giving away its unique technologies to the company.

Andy Hertzfeld notes in Folklore.org: Macintosh Stories: MacBasic that beta versions of MacBASIC— distributed before Microsoft canned it— ended up being widely pirated, and two books on MacBASIC were published and sold well for years after that, despite never being officially released as a commercial product.

Mac vs Lisa in Application Development

While MacBASIC was intended to be a consumer level development tool, Apple also delivered tools geared toward professional developers. These were partly inherited from the Lisa, which had shipped a year prior to the Macintosh with the Lisa Workshop, a complete development environment, albeit based in a text console environment.

Lisa also shipped with a full productivity suite called the Lisa Office System 7/7, which included a the LisaWrite word processor, LisaCalc spreadsheet, LisaGraphs for charting, LisaList for outlines, LisaProject for task scheduling, LisaDraw for art, and LisaTerminal for serial communications.

The original Mac only shipped with MacWrite, MacPaint, MacDraw, MacProject, MacTerminal, and a beta of MacBASIC. It lacked the full Lisa suite, in part because Apple didn't assign the same resources to the smaller Mac team, and in part because it was trying to encourage more third party support, and didn't want to step on the toes of the very developers it was courting. In large part, that again meant catering to and dependence upon Microsoft, which shipped Word and Excel shortly after the Mac's release as its first significant third party applications.

The Mac Toolbox and Human Interface Guidelines

In order to make the Mac easier to program than previous generations of computers such as the Apple II or the IBM PC, Apple included libraries called the Mac Toolbox that made it easier to follow the conventions of the Apple Human Interface Guidelines, and gave applications a lot of common functionality for free.

For example, prior to the Mac every application managed its own printer support; individual word processors came with support files for a specific list of printers. On the Mac, a unified printing interface made installing a printer a system-wide feature that enabled all apps to print using the same dialog box. How everything worked was defined in great detail in "Human Interface Guidelines: The Apple Desktop Interface," (excerpt below) which defined details we now take for granted as obvious.

Apple's guidelines also defined standard, system-wide key combinations for all applications, so uses didn't need to learn a different set for each application they bought. Standardized key combinations, such as Command + O to open a file, had no equal in the mixed up world of DOS, where every app invented its own key combinations, as noted in How Apple Keyboards Lost a Logo and Windows PCs Gained One. For example, to simply open a file:

• WordPerfect used the command F7 + 3.
• WordStar used Ctrl + K + O.
• Lotus 1-2-3 used / to open the menu, W for Workspace + R for Retrieve.
• Microsoft Word used Esc to open the menu, T for Transfer + L for Load.

The Macintosh Programmers Workshop and HyperCard

In 1985, Rick Meyers, Jeff Parrish, and Dan Smith began work at Apple for the successor to the Lisa Workshop for the Mac, called the Macintosh Programmers' Workshop. It shipped in late 1986.

MPW (below) supported object-oriented Pascal (and later C, and C++ compilers licensed from Lucid under the name MrC/MrCpp) and a 68k assembler, and shipped with a separate debugging tool. Apple's MPW was priced sky high, well out of the range of hobbyist programmers.

In order to offer a more accessible development environment, Bill Atkinson, Apple's developer behind QuickDraw, created HyperCard. It made the task of building a scripted presentation easy even for non-technical users. Atkinson insisted that Apple bundle the program for free, and this led to it being widely used and wildly popular from its release in 1987 through the mid 90s.

In 1991's Mac System 7, AppleScript appeared as a new system service to allow any application to support a scriptable dictionary of actions that could be triggered by an external scripts. AppleScript was derived from the HyperTalk scripting language of HyperCard.

Smooth Transitions for Users, Rocky for Developers

In 1989, Apple began touting the release of System 7, encouraging developers to produce "System 7 Savvy" applications without providing them the tools to actually do this. System 7 introduced a full color interface and support for lots of end user features, including multiple concurrent applications.

However, while Apple repeated its admonitions to get ready for System 7 at three WorldWide Developer Conferences from 1989 through 1991, Apple itself didn't have a compatible version of MPW to run on the new system when it finally arrived. This forced developers to use MPW under System 6, then boot into System 7 to see if their apps worked, as Will Flor notes in Development under System 6.

In 1991, Apple announced plans for another transition to PowerPC. While it eventually provided PowerPC development tools in MPW, by then Apple's own development tools had long been considered overpriced and underwhelming. Symantec's Think C had taken the lead as the standard environment among most 68K Mac developers.

However, Symantec didn't offer PowerPC support, leaving Mac developers trapped between dead tools and Apple's expensive MPW. Metrowerks, an independent Canadian developer, began work on its own PowerPC development tools and was ready to release them under the name CodeWarrior in 1994, just as PowerPC Macs were about to go on sale.

Without Metrowerks, Apple's transition to PowerPC may have failed completely, just as the company itself was getting hammered by the growth of Windows and its own work on a System 7 replacement was beginning to fall apart. Apple continued to support MPW through 2001, but most developers rapidly transitioned to Metrowork's CodeWarrior (below).

NeXT Developer Tools

Jean-Marie Hullot had visited Apple in the late 80s to show off his ideas for Interface Builder, a new development tool for creating a palette of interface elements which could be graphically arranged, then attached to functions. Interface Builder promised to make the development of graphical applications far faster than the existing tools offered by Apple or third parties.

After deciding he didn't want to work at the corporate environment of Apple, Hullot shopped his idea around and ended up demonstrating Interface Builder (below, running on NeXTSTEP) at NeXT, where Jobs hired him immediately. Hullot became NeXT's Chief Technology Officer and worked on both Workspace Manager and the underlying Application Kit development frameworks that shipped as part of NeXTSTEP in 1988.

NeXT supplied Project Builder as its development coding tool, and centered development around Objective-C, a language developed by Dr. Brad Cox. Objective-C enhanced C in subtle ways to deliver an easy to use, component-based approach to development. Rather than writing everything from scratch, object-oriented development seeks to reuse existing components by adding or changing features and behaviors as needed. Objective-C also strongly influenced the development of Oak at Sun, which was later branded as Java.

NeXT organized dynamically loadable library into frameworks called kits. The Application Kit presented everything in the graphical interface with an objective interface; the Foundation Kit provided a similar abstraction to lower level tasks such as memory allocation, file access, and thread management.

NeXT's development tools and the frameworks they used were the main value NeXT added; they attracted attention from a wide variety of serious developers, including the CIA, investment firms, academic researchers (including Tim-Berners Lee in the development of the Web) and high performance developers like id, which chose NeXTSTEP as the development tool for its DOOM World Builder. When NeXT gave up on selling its own hardware in 1993, it delivered a port of NeXTSTEP running on standard PCs, and later on HP's PA-RISC and Sun's SPARC workstations.

NeXT later delivered its development and deployment tools separate from its underlying operating system, allowing "OpenStep" to run on top of Sun Solaris and Windows NT (below, running Project Builder). This portability and sophistication caught the attention of Apple in 1996, when it was clear the company's own in house development efforts were in shambles and needed outright replacement.

Apple's NeXT Tools

Apple quickly ported NeXTSTEP to PowerPC, thanks to preliminary work at NeXT to deliver a new generation of PowerPC-based NeXT computers shortly before it pulled out the hardware business entirely. After outfitting it with an appearance derived for the ill-fated Mac OS 8 Copland project, Apple shipped the results as the Rhapsody developer preview.

Rhapsody's Project Builder (below) supported the development of Yellow Box (OpenStep) applications that could be deployed on both Rhapsody for Mac, Rhapsody running on PCs, and Windows NT PCs running the Yellow Box runtime. Apple also planned to investigate reviving the ports for Solaris, and would not have have much trouble delivering a commercial Yellow Box on Linux. GNUStep had already begun development of an OpenStep implementation for Linux.

Interface Builder for Rhapsody (below) showed off the drag and drop simplicity in rapidly creating graphical interfaces for new applications using Yellow Box.

Back to My Mac

Mac developers didn't want to rewrite their applications to run on Rhapsody using the Yellow Box, however. Instead, they insisted Apple add support for existing Mac code written to the classic System 7 application programming interfaces.

This required a complete re-architecture of how development would work, and the result was a new strategy designed to sell Mac hardware rather than to solve the world's incompatibility problems between platforms. NeXT had already tried to push OpenStep as a open specification for delivering cross-platform compatibility on any operating system, but enthusiastic support from partners came and went as they became distracted by other subjects and strategies.

After realizing that it wouldn't be able to convince the world to adopt OpenStep under the new name Yellow Box, Apple developed Carbon as a way to port existing Mac code to run on the new system with minimal effort, leveraging work already done to make QuickTime (and the underlying QuickDraw) run on Unix. In order to call Foundation from both Carbon and the new Cocoa (a subset of Yellow Box), the system had to change from an object-oriented Kit to a procedural C library, called Core Foundation. Along with the removal of Display PostScript, this destroyed compatibility with the former OpenStep specification, and rendered Yellow Box an obsolete technology, as described in Cocoa and the Death of Yellow Box and Rhapsody.

Core Foundation and the CFLite Skirmish

As Apple worked to release the core OS of its new system as an open source project, it found that portions of the new Core Foundation would need to be opened up as well. The IOKit needed the ability to handle plist files and CFRunLoop functionality, but the IOKit was also part of Darwin. That forced Apple choose between opening up its Foundation technologies, or implementing a separate way to perform these tasks.

The Foundation and Application Kit developers at Apple are not interested in giving away the company's core technology frameworks as open source code. Mac OS X software engineering lead Bertrand Serlet initially insisted that the IOKit developers would have to roll their own Core Foundation-like functionality. Apple's Core OS team had been pushing the concept of open source as fair play upon the rest of the company, but nothing in Apple's XNU kernel— including portions of BSD— had any obligation to be released.

After some convincing, Bertrand agreed that the IO team's case made sense and allowed the team to split Core Foundation into two libraries, Core Foundation proper and CFLite. The split also solved another serious internal build problem for Apple: when a new image of Mac OS X was built, 1500 projects had to be built with it. They are all built in ordered layers, but IOKit is a bit odd in that it relies on BSD, Kernel, and Core Foundation. That means it must be built after Core Foundation, but Core Foundation itself also depends on the IOKit. This circular dependency was unacceptable to Apple's Build and Integration group and would make it impossible to build a complete image.

CFLite was the part of Core Foundation that IOKit depended upon and did not in turn depend on IOKit. Thus the build order became: XNU kernel, LibSystem (BSD user), CFLite, IOKit User, then Core Foundation and the rest of the system. CFLite was included as open source in Darwin, and serves as a way to share some low level Mac OS X conventions on software designed to run cross-platform.

Developer Tools Become Xcode

After working out a solid underlying architecture for developing future advancements, Apple released its first three versions of Mac OS X with the development tools ported from NeXT (below), along with support for Apple technologies such as AppleScript.

In Mac OS X, Apple made it much easier for applications to make themselves scriptable, by building support for AppleScript into the Cocoa Application Kit. Apple offered AppleScript Studio as a visual development toolset for building applications in AppleScript that made use of Cocoa application frameworks.

With Mac OS X 10.3 Panther, Apple incorporated AppleScript Studio and ProjectBuilder (above) into an integrated development environment called Xcode (below), using an entirely redesigned user interface. It offered Smart Lists, symbol browsing, integrated search results, build errors and warnings notifications, and a three pane window interface for browsing project code.

It also integrated Apple's documentation, featured code completion for Objective-C, and offered predictive compilation. A year later, apple released Xcode 1.5, which extended code completion for Java and AppleScript.

With Mac OS X 10.4 Tiger, Apple launched Xcode 2.0, with support for the new GCC 4.0 compiler and the new Quartz Composer visual programming language. A follow up update added support for WebObjects development and launched a new transition to Intel processors.

Apple announced that work to move to Intel would be much simpler for developers using its own Xcode tools rather than Metroworks' CodeWarrior. Motorola had acquired Metroworks in 1999, and discontinued CodeWarrior for the Mac OS as Apple announced its transition to Intel. Apple has since regularly updated its Xcode and other developer tools including Interface Builder, finally proving that it has discovered that, if it wants to be in control of its own platform, it has to manage and fully support its own development tools.

New In Leopard: Xcode 3.0

In Leopard, Apple added support in Xcode 3.0 (below) for the new Objective-C 2.0, which adds support for optional garbage collection, and 64-bit application development. It also incorporates basic version control with Project Snapshots, and offers a Research Assistant for looking up context sensitive documentation. An Organizer panel lets you drop other documentation and related files in for quick access; you can also bookmark items from Apple's included documentation in the Xcode sidebar, and subscribe to Apple's online documentation to automatically maintain the most up to date information.

Instruments 1.0

Xcode also integrates with Sun's OpenSolaris DTrace tool, which Apple includes a graphical interface for in Leopard under the name Instruments (below, formerly code named X-Ray). Using an interface similar to GarageBand, Instruments can be set to dynamically trace memory allocation, processor use, file activity, and other factors as code runs, and compare how performance changes over time. It can also track mouse moves and keyboard clicks and then replicate a given action repeatedly to help determine why an inconsistent error is occuring. Instruments also introduces some novel, if odd, new user interface concepts, including a modal information bubble panel (below). The panel sports a Done box at the bottom, but if there's a lot of options, it can easily end up out of view behind the Dock or off the screen.

Instruments also features a full screen view that blows up just the instrument trace view without displaying a toolbar or any window controls, and a mini view that shrinks the application into a translucent panel (below). To recover normal view, you counterintuitively close the panel, much as closing a Screen Sharing or iChat Theater window ends the connection. In Leopard, closing a translucent panel sometimes means quit, and sometimes means "zoom."

Interface Builder 3.0 offers a larger library of objects that can be dragged into the interface of an application, making it easier for developers to implement the human interface features Apple has introduced in its own applications, including the path display used in the iTunes Store; the Smart Groups filters used in iTunes and the Finder; and the drag and drop image controls from iPhoto. The new version also adds the ability to create standard toolbars using the graphical interface (below).

DashCode 1.0 makes it easy for even entry level developers to create Dashboard widgets, starting from prebuilt widget templates based on a countdown, map, RSS feed, Quartz Composition, or a podcast or photocasts, or beginning from scratch to build something new. Below is an RSS feed template that requires little more than an RSS feed itself.

DashCode presents an interesting new workflow interface, which expands steps into items that may be tagged as required or optional, marked as completed by the user, and effectively breaks complex configuration tasks into simple steps with explanatory help icons along the way. DashCode deserves an article unto itself. Some comments were made on DashCode earlier in Road to Mac OS X Leopard: Dashboard, Spotlight and the Desktop.

Automator 2.0 makes it easy to set up workflows that draw upon prebuilt Automator Actions, or you can create your own in AppleScript or Cocoa using Xcode. Templates get you started in setting up workflows for working with files, iTunes audio (below), iPhoto libraries, or text related jobs, or you can start from scratch to automate anything.

Quartz Composer 3.0 is visual development tool for creating Quartz Compositions, which are used throughout Mac OS X as image filters, in graphic transitions and effects for iMovie and Keynote, in visualizations such as screen savers or visual effects for playback within iTunes or QuickTime, or dropped into other standalone applications using Interface Builder.

Quartz Composer uses colored Patches that are linked together by binding ports between them: pink renderers, green data processors, and blue external input patches. Input patches can listen to and generate keyboard and mouse clicks, scrolls, and movements or MIDI data and audio signals. Renders and processors can use OpenGL, Core Image, and Core Video to create and modify signals in complex ways. A collection of patches can be assembled into a macro; macros can also be nested.

A complex set of patches can incorporate user actions, music changes, or even pull data pulled from the web or an RSS feed to create interactive information displays and animated visualizations. Below is the Quartz Composition file used as the new Shell screen saver in Leopard, along with a live visualization.

Apple's developer tools are free with Mac OS X. NeXT charged $5000 per seat for its developer tools, on top of $799 licensing for the operating system. WebObjects cost even more; that too is thrown in for free. Additionally, Apple's tools also support unique development features related to Quartz and AppleScript that didn't exist for NeXT. Whether you're a serious programmer or just want to play around with the diverse collection of tools Apple throws in for free with Leopard, there's a lot to explore. Even if you're not a developer at all, you can be glad that it's now that much easier to develop new software for the Mac.

Don't forget to check out our earlier Road to Leopard series:

Road to Mac OS X Leopard: QuickTime, iTunes, and Media Features

System Preferences

Parental Controls and Directory Services

What's new in Mac OS X Leopard Server

Dashboard, Spotlight and the Desktop

Safari 3.0

iCal 3.0

iChat 4.0

Mail 3.0

Time Machine

Spaces

Dock 1.6

Finder 10.5

Dictionary 2.0

Preview 4.0.

Also check out earlier installments of our Leopard review series that followed our Road to Leopard series:

An Introductory Mac OS X Leopard Review: Meet Your New Desktop

An Introductory Mac OS X Leopard Review: Mail and iCal

An Introductory Mac OS X Leopard Review: Address Book and iChat

An Introductory Mac OS X Leopard Review: Core Graphics and the New UI