There’s an old saying: There’s nothing new under the sun. Maybe there’s something to it. Details change, but the general patterns of things remain consistent. The wheel keeps turning, and if you live long enough you see the same spokes come around again and again.
James Shore, a well-respected figure in the software development field and the author of the highly-regarded book, The Art of Agile Development, has developed an approach to software development he has called a pattern language for testing without mocks.
I struggled to get started using James’ pattern language. Initially, I was trying to write code in two different ways in parallel – one using mocks and the other using the pattern language. I thought it would be interesting to apply the patterns to different types of languages and different programming problems. It was extremely frustrating. It seemed no matter what I did, I was breaking one of more of the “rules.”
Fortunately, James was willing to help me quite a lot, pointing out where I had gone astray and suggesting reference material to help me learn. He mentioned that people who have internalized testing with mocks often have similar difficulties picking up this approach, and that it requires a mindset shift.
That’s exactly what I was doing. I was approaching it from the perspective of testing with mocks and then seeing what would have to change to build the same solution using the pattern language. It proved to be a fool’s errand.
I paused and took a couple of days off from it. Then I read through all the material describing the various patterns and so forth. It struck me that what he was describing was very similar to the way we used to apply common application patterns in the 1970s-80s. By focusing on the comparison between “using mocks” and “not using mocks” I was missing the point.
Sorting a few things
In his article, A Light Introduction to Nullables, James writes of his pattern language, “Whatever you call it, it’s a set of patterns for combining narrow, sociable, state-based tests with a novel infrastructure technique called ‘Nullables.'” In the same article, he states, “At first glance, Nullables seem like test doubles, but they’re actually production code with an ‘off’ switch.”
Not Slytherin, eh?. Um…I mean, not Test Double, eh? <gesture type=”squint”> First glance. </gesture> <gesture type=”headTilt”/> Second glance. </gesture> <gesture type=”reverseHeadTiltWithSquint”> Third glance. </gesture>
Production code with an “off” switch
Setting aside the question of whether a thing that stands in for real code during testing is a Test Double or not, let’s consider the other point. Is it production code with an “off” switch, or it is something “novel”…because production code with an “off” switch certainly isn’t “novel.”
There’s a fairly common practice called feature toggles that teams use to disable selected functionality in production while they’re in the midst of a series of mini-releases that incrementally builds up that functionality. They can turn the functionality “on” for testing and “off” in production.
The same technique may be used to phase in a major change in an application, activating it for a select user base as one step in a series of steps to introduce the change carefully. If something goes badly wrong, they can switch the application back to the old functionality without interrupting users’ work.
Application patterns and skeleton programs
The use of production code with an “off” switch may be older than feature toggles. The pattern language – particularly in conjunction with A-Frame Architecture – reminds me of something from the dim past. Back in Olden Times, when the mainframe was king, nearly all applications were examples of a handful of common patterns. Many of us developed template code for these patterns to jump-start our work on new projects.
The template programs contained the boilerplate code necessary to implement one of the common application patterns, and placeholders for adding custom code to support the business rules for each application. We called these incomplete programs “skeleton programs.”
Each application pattern called for skeleton programs that were tailored to a particular execution environment, programming language, and back-end data store. A single set of “rules” wouldn’t have supported all solutions, even if the overarching shape of applications to solve the same category of problem was roughly the same. You couldn’t implement the same idea in PL/I and COBOL using exactly the same code. You couldn’t interact with the TSO and CICS runtime environments using the same APIs. You couldn’t access IMS/DB and DB/2 databases using the same commands. The patterns were similar, but we had to be flexible about the details.
To get a sense of how this approach works, let’s consider just one category of applications, as it corresponds with a lot of Webapp work today: interactive CRUD applications. We used to call them “online” applications, before the word “online” came to mean “on the Internet.” The word just meant “not batch.”
In those days, several teleprocessing monitors (roughly analogous to web servers and app servers) had significant market share – CICS, IMS/DC, DATACOM/DC, TSO with ISPF Dialog Management Services (DMS). A few programming languages were used in the mainframe environment for this type of application – COBOL, PL/I, Assembler. A few back-end data stores were common – IMS/DB, DATACOM/DB, DB/2, Oracle RDBMS, IDMS, IDMS/R, VSAM. All these things came and went as various products gained or lost market share.
CRUD applications typically included a menu, functionality to browse forward and backward through the data, functionality to “drill down” from high-level data to details and back again, context-sensitive help, commit and rollback, restart after a failure to pick up where you left off, exiting the application, and the canonical CRUD operations in whatever form the selected back-end data store required.
Even if you aren’t familiar with those old technologies, you can probably imagine from this description that the overarching shape of interactive CRUD applications was the same while the coding details might be quite different. We needed a set of skeleton programs for a CICS application in COBOL accessing VSAM, another for an ISPF/DMS application in PL/I accessing DB/2, and every other permutation, all of them still pertaining to interactive CRUD applications. We needed the same sort of thing for other application patterns, too, like batch processing of sequential files, batch Extract-Transform-Load processes, online transaction processing (OLTP), and so on.
Our skeleton programs mixed and matched the various teleprocessing monitors, programming languages, back-end data stores, and menu styles. The skeletons were designed so we could drop in custom code to comply with each client’s local standards (e.g., which 3270 PF keys were assigned to which application functions).
We did a lot of things by convention that were later supported by compilers for Object-Oriented languages, to achieve results roughly similar to abstraction, encapsulation, and polymorphism, but not exactly inheritance. Well, not exactly any of those, if you want to be picky. We were not trying to invent OO design; we were just trying to keep our skeletons flexible enough to be reusable. OO-ish characteristics fell out naturally from that focus.
Standardizing and commercializing the idea
The approach was common enough that a commercial product was developed based on the idea – Telon. The product still exists and is now owned by Broadcom, who market it as Telon Application Generator. (There were a few others, too. All gone now except Telon.)
Originally, Telon was (and probably still is, although I haven’t seen the source code in decades) a set of Assembler macros that could generate COBOL, PL/I or Assembly language programs – skeleton programs – for common application patterns on mainframe systems. The resulting code had placeholders in well-defined spots where we could drop in custom logic. You could always recognize a Telon-generated COBOL program because they all had the same structure – even the same paragraph names. Today Telon is front-ended by a GUI application, so it’s easier to use (not that it was ever hard to use, really).
You’ve seen something like that in modern times. When you use Android Studio to start a new Android application, you can have the tool generate starter code for you. The starter is “skeleton” code that implements the canonical structure of an Android application and even includes a sample unit test and a sample instrumentation test (which most people ignore).
Some Webapp frameworks are similar, too. You can run “create-react-app” to jump-start a React application. Want to write an MVC app using Ruby on Rails? Run “rails g scaffold”. For Django, run “py manage.py startapp [name]”. All these tools are similar to the skeleton programs we wrote in the Olden Days insofar as they generate boilerplate code for one particular category of solutions using one particular language, and making one particular set of assumptions about what the back-end looks like.
Back to the future
James’ pattern language isn’t a Webapp framework or a code generator, of course. I mentioned that only to try and come up with an analogy the current generation might relate to. But the pattern language does feel a lot like those old skeleton programs, or products like Telon. And the pattern language appears to have been developed for Webapps initially, in the JavaScript language. It’s a reasonable analogy, I think.
Thinking about it from the perspective of developing based on a template, the process goes something like this.
You start with an overarching application design, often called an “architecture” today, to make it sound sexy. Say you want to use the A-Frame Architecture for a standalone Java application (not to be confused with A-Frame for virtual reality).
You set up some classes in a way that conforms to that architecture. Your starter code doesn’t have to work for other architectures. Just A-Frame. It doesn’t have to be the same for JavaScript/TypeScript, Ruby, Python, or any other language. Just Java. It doesn’t have to support Webapps or embedded apps or XYZ apps. Just standalone apps (sometimes called “console” apps).
Don’t overthink the word, “architecture.” It isn’t as fancy as it sounds. It’s just a convention for how to lay out the Java packages. You’d create a package for the app (the top of the A), a sub-package named “infrastructure,” and a sub-package named “logic” (the sides of the A). Then you fill in the blanks. You can add more “stuff” if you need it.
You could even set up a source repository with a lot of the boilerplate already in place, including some of the executable test cases, to facilitate re-use. You could include incomplete Java classes for Infrastructure Wrappers. Maybe you could call them “skeleton” classes. You could copy the repository to jump-start any other Java project based on the same architectural pattern. Maybe something like this. For other architectural patterns, you’d need other skeletons, of course.
We used to work that way quite often. The approach applied not only to interactive CRUD applications, but to any of the common application patterns of the era. Each application pattern called for a different set of programs written in a way that worked for that pattern, in the target execution environment, in the programming language we were using. One Pattern Language To Rule Them All wouldn’t have helped.
The main difference is that today there’s an emphasis on executable test suites. You might test-drive some of your code, or write test cases after the fact…or not. In the livestream where they demonstrate all this, James and Ted Young test-drive everything. It’s generally considered a good practice today. The pattern language doesn’t interfere with any style of TDD you might choose, or any other way of coding you might prefer besides TDD.
More old-timer tales
We didn’t have tooling for executable unit tests back in the Olden Days. In fact, the traditional mainframe languages don’t lend themselves to fine-grained testing because you can’t execute just one routine in isolation. Well, you sort of can if you try hard enough, but we didn’t try quite that hard back then. And of course you can do anything in Assembler. Anyway, all the tests we wrote were “sociable.” They weren’t necessarily fun at parties, but they were sociable. They had to be.
So, if we didn’t have unit testing libraries, and we didn’t have mock libraries, what did we do? You guessed it: We wrote production code with an “off” switch.
You would be forgiven if you thought, at first glance, the code that could be turned on for testing looked a bit like test doubles.
First glance, second glance, third glance.
I think there’s an old saying that applies here. Don’t quite remember it just now, but I’m sure it will come back. Everything does.
Suggestions for learning
If you’re interested in learning to use James’ pattern language, there’s plenty of help online. He has made his training course public in a self-guided format. It online at https://jamesshore.com/v2/courses/testing-without-mocks.
The course material includes links to more information about each concept and pattern. My first suggestion is to follow this self-guided course. Try the exercises in whatever languages you expect to be using, in addition to JavaScript (the language of the course).
My second suggestion is try not to be put off by all the new buzzwords and buzz-phrases. Almost everything in the pattern language is something that already existed. For instance, you’ll see these terms:
- Broad Tests
- Narrow Tests
- State-Based Tests
- Signature Shielding
- Zero-Impact Instantiation
- Parameterless Instantiation
- Sociable Tests
- Output Tracking
There’s nothing really new here – except Overlapping Sociable Tests that are not also Broad Tests. That seems new to me.
Finally, I suggest that once you’ve gotten a handle on the core ideas of the pattern language, don’t be afraid to adapt it to the execution environments, programming languages, and other factors relevant to your needs. Just as with the skeleton programs of old, every category of solution calls for its own application “shape”, every execution environment has its own APIs, every programming language has its own features that might make some of the patterns easier or harder to implement. Everything isn’t a Webapp written in JavaScript.