I have been moving quickly between tasks since my return from Boston. I'm finally taking some time off for the Independence Day holiday. Floating in the pool, I keep finding the subject of Ted Neward's essay coming to mind. It appears I still have something I want to say about ORM. In this series of posts, I hope to explain why Visual FoxPro has more mojo for resolving Object Relational Mapping issues than any other technology, of which I am aware.
At the speaker's dinner for the Atlanta Code Camp, I discussed LINQ with Charlie Arehart, and Wally. I brought up the doomed Object Spaces project, and quoted Ted Neward's pithy line that "Object relational mapping is the Vietnam of computer science." It is a good line. It is also true, as much as a pithy metaphor can be true.
Now that Ted has expanded his metaphor, we can see the limitations of metaphor in general. I recommend Jeff Atwood's analysis of Ted's essay. I agree with everything Jeff has written there. Primarily, I agree that Ted tried too hard to stretch his Vietnam analogy which distracted from his primary, quite sound, thesis that "there is no good solution to the object/relational mapping problem." Full stop.
Despite any over-reaching in support of his Vietnam analogy, I must state clearly that Ted Neward is possibly the most qualified person in the industry to make the criticisms he does of ORM. Here's a short bio with his credentials. Suffice it to say that he is not arguing from a position of ignorance. Ted works with many technologies, and has seen the limitations of ORM manifested in different languages and platforms.
To continue with my love letter to Ted, he acknowledged something that few people in the industry are even aware of. Visual FoxPro eases the dissonance between objects and relational data better than any other technology on the planet. I first came to this realization upon reading this post by Joel Spolsky in 2004. Joel identified the impedance mismatch between objects and relational data as a shortcoming in all modern programming languages. Joel didn't bother listing VFP among "modern high-level programming languages."
I'm not going to bitch and moan about the low profile of VFP. I know what it can do, and it's been earning me a decent living for years. I just can't believe other languages have not done a better job of "borrowing" the best of what VFP does. I'm damn tired of all the LISP articles on Reddit telling me how I'm missing out because my language of choice doesn't have feature foo which LISP has had since the '60's. I also take issue with Paul Graham's essay about the RAD nature of LISP for startup companies. Any experienced VFP developer could easily make the same arguments in favor of the Fox.
When I first saw Ted's mojo comment, I felt vindicated. I made a quick post, and forwarded the link to "the Craigs". I secretly hoped they would write this argument for me. Fortunately, I have to sort this one out for myself. I am currently developing an object interface to encapsulate a VFP cursor. This is totally unnecessary in Visual FoxPro, but I need to leverage my object model from .NET, so I must encapsulate all the tasty VFP goodness behind a COM interface. This is the ultimate test of VFP's ORM mojo in my experience to date.
It is my contention that the "secret sauce" to VFP's ORM mojo is the cursor. A VFP cursor is a data structure like no other. Most non-VFP developers associate VFP with the DBF storage format. People seem to assume that it is "just" a database. Visual FoxPro does have a very powerful in-process data engine which includes a wickedly fast ISAM storage format, but it can use data from any source as a cursor with that same data engine. This is where we find the secret sauce.
Other languages need to map data from relational sources to primitive types such as arrays or dictionaries (collections). The ADO recordset is actually an object oriented implementation of VFP's cursor engine. The ADO.NET dataset takes this further to include relations in the definition. Strongly typed datasets are a particularly easy way to handle data in managed code. The advantage that VFP has is that it handles data in cursors natively. That is what it is made to do. Handling data in a columnar format, and setting relations between these table-like data structures is the core of the VFP mission.
VFP developers use SQL commands as first class citizens of the language. They are handled by the runtime in the same manner as conditional and looping statements such as IF and FOR. There is no conceptual difference to a VFP developer between addressing a local variable, a property on an object, or a field in a cursor.
Let's pause for a moment and refine our terms. Here is what Whil Hentzen has to say about VFP cursors in his excellent (though out of print) introduction to VFP development Fundamentals: Building Visual Studio Applications on a Visual FoxPro 6.0 Foundation:
We’ve been using the term "result set" to refer to the collection of records created by the execution of a SELECT command. While Visual FoxPro is "record oriented" in that we can move from record to record as we desire, other languages that use SQL do not have this capability and can only address a block of records as a single entity. As a result, all SQL operations are oriented toward manipulating this "set of records." They refer to this block of records as a "cursor," which stands for CURrent Set Of Records. A cursor is a table-like structure held in memory; it overflows to disk as memory is used up. In some cases, such as queries that essentially just filter an existing table (in other words, no calculations or joins), Visual FoxPro opens a second copy of the table in another work area and then applies the filtering condition to present a different view of the table as the query requests it. Cursors have two advantages. First, they are fast to set up, partly because of the possible filtering of an existing table, and also because of their initial creation in RAM. Second, maintenance is greatly reduced, because when a cursor is closed, any files on disk created due to RAM overflow are automatically deleted.
We’ve been using the term "result set" to refer to the collection of records created by the execution of a SELECT command. While Visual FoxPro is "record oriented" in that we can move from record to record as we desire, other languages that use SQL do not have this capability and can only address a block of records as a single entity. As a result, all SQL operations are oriented toward manipulating this "set of records." They refer to this block of records as a "cursor," which stands for CURrent Set Of Records. A cursor is a table-like structure held in memory; it overflows to disk as memory is used up. In some cases, such as queries that essentially just filter an existing table (in other words, no calculations or joins), Visual FoxPro opens a second copy of the table in another work area and then applies the filtering condition to present a different view of the table as the query requests it.
Cursors have two advantages. First, they are fast to set up, partly because of the possible filtering of an existing table, and also because of their initial creation in RAM. Second, maintenance is greatly reduced, because when a cursor is closed, any files on disk created due to RAM overflow are automatically deleted.
I hope it is clear from the above that any data that can be understood conceptually as a row/column data source can be handled very elegantly by VFP. The key point to recognize is that the data runtime and the language runtime are one. They even share a common garbage collection mechanism, which you can read about here. VFP developers are able to enumerate, search, sort or perform set based operations against one or more cursors without making a context shift from the surrounding procedural code.
Cursors are not perfect. They are not objects, and can therefore not be passed around by reference. There are two workarounds for this conundrum: ADO recordsets, and XML. I use XML exclusively. I no longer consider ADO recordsets to be a maintainable solution. While the shortcomings of XML are well documented, the flexibility is irresistible. Furthermore, the VFP XmlAdapter class makes it a snap to translate between cursors and XML, and even take and apply diffgrams. Therefore, I use XML whenever I am passing data across object/process boundaries, and I convert the XML to VFP cursors within my object interfaces.
In the next installment on this subject, I will address the object component of Object Relational Mapping. For now, I hope that I have clarified that Visual FoxPro is not "just" a database. From my perspective, it is primarily a runtime library for the Visual FoxPro language which seamlessly blends typical imperative programming constructs with relational data handling.
++Alan