Let is snow, let it snow, let it snow.

Well, I finally finished building a lift in the back of my garage so that I can now store my utility trailer up against the back wall, giving me use of my trailer and also use of my car during the winter months. (Usually the trailer is out in the side yard and snowed in for several months of the year). Just in time, too -- we should see our first snow tonight. Funny how a deadline can be so motivating.

Let it snow, let it snow, let it snow.

Happy Holidays, everyone.

The word from Seoul

Becky, my daughter, is half way through a one-year contract teaching English to grade schoolers in Seoul, South Korea. So I got pretty concerned when North Korea started shelling an island in South Korea earlier this week. But when we finally got to talk this morning, Becky told me that most of the Koreans she knows do not seem alarmed and do not expect this to escalate into something bigger. I guess you can get used to anything, even a mad neighbor constantly kicking sand into your face.

I hope she's right that this won't get any worse, but it is hard for me not to worry. In any case, she promised to keep a bag packed and to have a plan in case things go bad.

It's Thanksgiving tomorrow. Things like this remind me of how thankful I am to be living in a stable country with choices and opportunities everywhere. The things that normally stress me out from day to day seem a lot smaller now.

It's been a while, and things have changed.

About the time I became a blogger, a lot of other things happened in my life, the big thing being the death of my father and then spending the next 1-1/2 years settling his estate on top of everything else I was doing. We also sold a house, bought a house, fixed up the house, rented out the house, and have now bought another one, this one planned for our own use when we retire sometime over the next few years. And, oh yes, I moved off the Lotus Notes/Domino team at IBM and onto the LotusLive Meetings team as a technical lead. (So there will be no more blogging about Domino APIs). And we have a new granddaughter, Alexa Mia, who, of course, is the most beautiful baby in the whole wide world.

Whew.

And so, not being a blogger by inclination or habit, blogging fell by the wayside. Over that time period, though, I've changed my opinion about blogging. Before, blogging seemed like yet another way for people to shower their opinions onto as many other people as possible, something I usually try to get away from, and so why would I want to join that crowd and promulgate my opinions on others?

In the meantime, though, I've found a number of blogs that have useful information, good advice, and are a forum where ideas are advanced by the contributions and interaction of many. Because there are work- and life-related issues I care about, I am reviving my blog as an experiment to see if it really can be used as a tool for getting me information and ideas about those things.

Notes / Domino Backend API Architecture

Click the diagram to see a bigger image.

There's the old saying, "Ask an engineer what something does, and they will tell you how it works." Makes sense to me, so I must have some engineer in me.

Since I will be blogging a lot about Domino backend APIs, it seemed appropriate to provide baseline information and an architectural diagram I can refer back to in the future. I assume you are a developer that has an interest in (and maybe some experience already) writing programs that use the APIs to create or modify Domino data, so I am not going to explain a lot of really, really basic things about Domino programming in my blogs. The topics will definitely be pitched to the intermediate to advanced developer.

The diagram shows that the Domino API programming stack has four major layers -- the Application layer, the Backend layer, the Services layer, and the Database layer. The top layers call upon services provided by the lower layers, which I think is best described from the bottom up ...

The Database Layer. This layer contains the Notes/Domino databases (more recently called "applications) in the file system. I know this is a gross oversimplification, but each database contains a) a header record, b) design notes which contain UI and business logic for creating, indexing, viewing, and updating documents, and c) the documents themselves. A user "runs" a Notes/Domino database either in the context of the Notes Client or in the context of the HTTP task in a Domino server. These two programs interpret the design information in a database and present content to the user through a window application (e.g. a Notes Client running on Windows) or through a browser connected to the HTTP task in the Domino server.

The Services Layer. This layer has thousands of individual functions for creating and maintaining database content. You can run your applications offline using the Notes Client because the functions in this service layer are installed on both the Notes Client and on the Domino Server. (And also because Notes data is universally replicatable. You make a "replica" copy from the Domino server to your workstation, then use Notes on your workstation to run your application locally, and then replicate your changes back to the Domino server later).

The service functions are written either as C functions or as C++ functions with C wrappers, and many of these C functions and wrappers are exposed through the Notes/Domino SDK. In general, the most functionality is available to your programs by tapping directly into the Services Layer using the SDK, while the higher-level layers, because of the work involved in developing and testing them after the C layer is created, make a little less functionality available.

An interesting feature of the Services layer is NRPC (Notes Remote Procedure Call). NRPC is a switch that directs database-related calls by the higher-layer programs shown in the diagram to either the local implementation (to work on local databases) or to an implementation on a remote Domino server (to access a database there). So except for the one case when an application first opens a database, specifying either a local one or one on a remote Domino server, NRPC makes it totally transparent whether they are working with local or remote data for all other database operations after that.

The Backend Layer. This layer implements the different higher-level language bindings onto the Services layer. For example, when the Notes Client is interpreting a Notes application and encounters an @formula, it calls the formula-language engine in the Backend layer to "run" the formula, which results in one or more calls into the Services layer. Ditto for LotusScript and Java code. Three of the language bindings -- LotusScript, Java, and COM -- are layered on top of the "Backend Object Model," which provides a common Object-Oriented data model across all three languages. For example, LotusScript presents a class called "NotesDocument," which has a set of methods and properties similar to those of a Java class called "Document" presented by the Java binding. Each language binding has its own idiosyncrasies, resulting in some minor differences from binding to binding, but otherwise all bindings are substantially the same object model.

The Backend layer provides two ways to access the Java bindings onto the Backend Object Model -- "local" Java applications (Java applications that run on a computer with the Notes or Domino dlls installed) link with Notes.jar, while "remote" Java applications (Java applications that run on a computer without the Notes or Domino dlls installed) link with NCSO.jar. Remote Java applications require that the Domino server runs the DIIOP server task to complete the server-side connection and hook into the backend instance on the Domino server. A local Java application can access both local and remote Domino databases through the magic of NRPC. A remote Java application can access only remote databases on Domino servers.

The Application Layer. This is the layer where you, the developer who writes programs that access Notes/Domino data, do your work.

Notice that the diagram shows two types of application runtime environments -- a "local" runtime environment where Notes or Domino dlls are installed, and a "remote" runtime environment where they are not.

Let's look at applications that run in a local runtime environment first.

The vast majority of Notes/Domino developers use the Domino Designer to create Notes/Domino applications, which are always run in a local runtime environment because Notes/Domino applications are read up and presented to a user by either the Notes Client running on a GUI operating system or by the Domino server, which presents the application via HTML to a browser user -- in both cases, either Notes or Domino must be installed.

In addition, there are many situations -- usually encountered by business partners who need to access Notes/Domino data a part of a larger application -- where using the Domino Designer to write a Notes application does not meet their needs. For example, a Java program running on a non-Domino machine may want to input some data from a Domino database. Or a Visual Basic application running on a workstation where Notes is installed may need some data from Notes' local database or from a database on a remote Domino server (via NRPC). We provided alternative bindings to satisfy these types of requirements, too.

Local Java applications, .NET applications, and C/C++ applications are always run in a local runtime environment, calling down on a single programming stack to the functionality provided by the Services layer. Up above, I described how a local Java application uses Notes.jar to bind to the backend object model. We also provide a way for .NET applications to use COM to bind to the backend object model. C/C++ applications bypass the backend object model completely and call directly into the low-level functions in the Service layer, as shown in the diagram.

Now let's turn to applications that run in a remote runtime environment, where neither Notes nor Domino is installed.

As noted above, if a remote application is written in Java, it can bind to NCSO.jar and connect to the Java backend object model on a Domino server that is running the DIIOP task. Whether or not a remote application is written in Java, it can also access data in a Domino database via HTTP calls made to a Domino server running the Domino HTTP task. Most of the HTTP requests supported by the HTTP task return information in HTML format. But others -- for example, ReadViewEntries -- return a response data payload in xml format.

Wrap-up. Whew! That's a lot of stuff to condense down into one diagram and a few paragraphs. As I said earlier, though, this day's blog -- especially the architecture diagram, which took me a long, long time to prepare and is why I haven't posted blogs for a while -- will provide the basis of many future blogs.

It's all about renovation

Like most software developers, I've had only a few opportunities in my career (35 years and counting) to write a brand new program from scratch. Not that there hasn't been a lot of invention in the process, but it has mostly involved enhancing or fixing software that already existed and was usually originally authored by someone other than myself.

So I find a lot of similarities between my work as a software developer and my other primary pastime, renovating the house I am living in (number 9 and counting). In both cases, I want to do a good, professional, and aesthetically-pleasing job; I (usually) want to maintain the character of the basic product; and the end result has to serve its intended purpose.

There are a lot of other similiarities, too. A renovation can be big (add on a second story, construct a backend mail or calendar service), or small (paint a room or fix a bug). You often work with and/or manage others, especially if the job is big.

I know I am not the first to compare software development with building construction, and I know there are differences. But I still feel the similarities far outweigh the differences.

This blog will primarily be about Notes and Domino APIs, but don't be surprised when I ramble off from time to time to describe the issues I'm having getting my kitchen tiled (while it's still in use), or finding the time to complete the upstairs bathroom. They are all just projects to me, needing design, construction, and cleanup, whether it's software or house. There are punchlists everywhere, at work and at home, always demanding attention. The trick, as always, is to strike the right balance between attending to those demands and pausing from time to time to reconnect with the people, places, and other activities we enjoy.