A World of Gremlins

Handle is a gremlin. Handle can make other gremlins. Handle tends to make gremlins in the likeness of what he imagines, and what he imagines always comes in pairs. Today Handle has borne two children: Reqla and Resnak. But he does not know the purpose of his children. He just exists to create them. Polys, the muse god, inspires Handle and Handle transcribes the inspiration with his creative ability. Upon creation, Handle sends them to a gremlin school, where the gremlin teacher Routin helps them find their way in the world.

Routin immediately understands why Resnak is important. He sees that Resnak’s whole purpose in life is to communicate Reqla’s purpose back to Polys. Whenever anyone tells Resnak something, he and Reqla dissolve into the aether. It’s very important that whoever talks to Resnak say the best possible thing to him, because Resnak is capable of talking to Polys directly and upon discerning Reqla’s value, will tell Polys so that Polys may experience the joy of understanding of his creation. But to him, the purpose of Reqla is less clear. He converses with Reqla to attempt to figure out how she can best serve Polys.

In general, when Routin is dealing with gremlin children, if he can determine their purpose, he will send them out to fulfill that purpose. Otherwise, he would tell the child who can speak to Polys, Resnak in this case, that he does not know the other child’s importance.

Today Routin discerned that Polys is playing a language game with himself, and, growing bored of the language in which Polys typically thinks, wanted to hear something in a different language for the joy of it. However, Routin doesn’t know anything about language, but he knows someone who does. He sends Reqla and Resnak to the one and only gremlin translation company.

When they arrive, they are met by the company foreman Logon, who knows just enough about language to know which member of the company would be the most suited. After a short conversation with Reqla, he knows just who Reqla should talk to get her message translated.

Let’s pause here and get a little more interactive.

Reqla and Resnak are currently with Logon. A translator who is capable of translating Reqla’s message is known to exist by Logon. What happens next? Should Reqla and Resnak both go to the translator? If they do, does the translator greet them, or does Reqla petition the translator? Upon translation, who tells Resnak the translation, Reqla or the translator? Should Resnak stay with Logon? If he does, who brings the translation back? Does the translator stay sequestered away (maybe he is very fat, and cannot easily move)? Upon return, does the message carrier tell Resnak directly? Or does he tell Logon who tells Resnak? Is there any value to telling the foreman first? Does the foreman need to negotiate something internally (or maybe externally, with his accounting buddy) before following through to tell Resnak? Unwinding back, maybe it’s more economical for the translator to be in the presence of all three.

No matter which question you choose to answer, they all beg the question of what line of reasoning is logically consistent with the contrived world. If you find out that translators are all kept in ivory towers and you can only shout up to them, and they back to you, then that’s going to preclude you from calling them into the foreman’s office.

If you have a background in web API development, you may realize this is a very thinly veiled anthropomorphization of a translation web service. In the jargon of what is typical of the domain: a request and response pair are generated by the underlying http server. A handler dispatches the request/response to a router which determines which sub handler to call. If it’s a translation request, the translation sub handler dispatches, based on language detection heuristic, to a translation service, which may not be colocated, but remotely accessed through sub request. Upon completion of the translation, the translation is sent back to the requesting entity through the response body.

Clearly the description of the service using jargon is much shorter than the story. And it’s tantalizing to think that the shortness is necessarily an asset.

Is it possible that the brevity is actually cryptic or opaque to the requirements of implementation?

The latter might be how I would actually describe the service to another developer if they asked about it. I certainly usually wouldn’t concoct a story like I did above (which took me like at least 30 minutes to ensure narrative consistency, and even the most basic palatability, which I am still not sure it contains, lol.)

In going through the exercise of actually making the description more verbose, and forcing my components be broken up as active sub processes (the gremlins) behaving more or less as humans might, I found it causes lateral considerations that tend to be just left out of typical “formal” systems “design.”

Narrative Reasoning

Human intuitive reasoning seems to be very closely linked to the concept of narrative, specifically narratives involving other human actors. While it is possible for humans to reason about things that seem to bear no clear resemblence to another human, such as a perfect geometric object like an isosceles triangle, this seems to be mostly secluded to the right side of the bell curve.

All more or less functional humans have the ability to reason about narrative. It’s quite literally the human way. Hidden in these myriad reasonings may be an emergent solution to a problem that can be leveraged from an otherwise average minded individual. This is similar to the difference between a strong classifier like a support vector machine (an analogy to a very smart person), and the adaptive boosted weak classifiers like chained decisions stumps (a crowd of average people). In the machine learning metaphor, the average person is not contributing a miracle solution despite his averageness, he is being leveraged to exert just the right pressure on the problem to yield a solution. This analogy in turn reminds me of the classical conceptualization of the wisdom of crowds. Some people will be smarter than the entire crowd. But the reality is that these individuals are incredibly rare, and anecdotally they seem to work well neither with others who are equally as intelligent nor with less intelligent individuals.

If we wanted to democratize reasoning about complex things, like systems, and specifically reasoning about interactions between things that give rise to this complexity, it would seem prudent to treat components directly as human agents, or carriers of intentionality, because this is something even your average person can reason about. Too often we attribute human intentionality to components of a system anyway when reasoning about them, so instead of mixing technical jargon with behavioral analogy, what if we just treated the components as little humans, or little gremlins, or lemmings or whatever. What if instead of couching speech about systems development in highly technical jargon, we told stories instead?

The Morality Argument

There exists a moral imperative concerning software and our world as well. Our world today is driven by software. Most of it is terribly written and really hard to reason about because of that, and by some miracle still works a good 80% of the time. But bugs are rampant even in solved problem domains. A lot of time these bugs come from the difficulty not in reasoning about how a single component operates, but in understanding the relationships and communication patterns between components. Breaking software into components is a good first step in eliminating bugs, and there are probably thousands of articles on why this is the case.

We need to write reliable software. People’s lives rely on it often, including my own. In order to fulfill this basic moral criterion we have to come to the realization that our current ways of reasoning about systems are inadequate.

A History of Decomposition

There are several historically prominent methods by which systems were composed to try to improve the ability to reason about them, a few examples being structured programming, 90s object-oriented programming, and the recent uptick in functional programming. Finally, cutting across all of these methods was the usage of static typing to attempt to automate some aspects of reasoning so humans did not have to do all the heavy lifting.

Previous to the 90s, the predominant way this was achieved was through the usage of structured-programming which broke down procedures into sub-procedures that followed a few heuristics in order to stay well-behaved. But sub-procedures could literally do anything, and because of this they could manipulate state that other procedures relied on in a way that was unpredictable, etc. From Wikipedia:

Structured programming is a programming paradigm aimed at improving the clarity, quality, and development time of a computer program by making extensive use of the structured control flow constructs of selection and repetition, block structures, and subroutines in contrast to using simple tests and jumps such as the go to statement, which can lead to “spaghetti code” that is potentially difficult to follow and maintain.1

In the 90s, the manner by which problems were popularly broken down was what is still called “object-oriented programming.” Unfortunately, the person (Dr. Alan Kay) who originally envisioned this concept much earlier (70s) lost it to the morass of popular opinion of something else entirely. Endemic in this approach is the anthropomorphization of things often leading to convoluted design that sounds something like “the elevator object talks to the floor object to know if it needs to stop.” Wut..? Floors don’t talk. Neither do elevators. But we were told that if we look at the requirements document and find the nouns, those can be candidates for our domain objects. There are numerous criticism of what OOP eventually became.2

This became such a travesty in design that many people opted to leave it behind entirely for the repopularized holy grail of mathematical or functional programming. Instead of floors and elevators, we had vectors whose positions represent elevators and whose values represent floors, and we pass these vectors into analysis algorithms also informed by call button state vectors. These combine to produce a description of an action that will alter which elevator will stop at which floor.

Talking about math will always make you sound smart to those who don’t know any better. Actually knowing what you’re talking about can make you admirable. If you honestly are amongst the relative few people who know what they are talking about when the now aphoristic “a monad is just a monoid in the category of endofunctors” quote gets bandied about, you know we are few.

In all of these forms of programming, types have also been included to allow for differing levels of automated reasoning about a program’s correctness. It turns out that writing types that abstract object creation (classes) or types that describe abstract mathematical properties across data structures (haskell typeclasses) is actually somewhat of a hard problem if you are looking for true simplicity. Typed languages that are used by well meaning, but under educated, people tend to end up in type declaration explosion. Type A and type B may do 50% of the same things, because they actually are ad-hoc realization of some underlying type C, but discovering the underlying abstraction C requires more than just copying and pasting the 50% of the code that is shared. This type of ad-hoc inheritance then leads to extremely brittle code when A and B are coupled in ways they should not be.

One of Alan Perlis’ famous epigrams states the following, “It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures.” This pithy statement is generally backed up by concepts that can be borrowed from abstract algebra. But finding this data structure that gives rise to 100 useful functions is sometimes very obvious (lists come to mind), and sometimes much more difficult (program I/O).

The Proposal

There is utilitarian, aesthetic, and performative value for making systems design crowd-sourceable.

There is moral value as well.

Components of a system should be structured in such a way as to be as human like as possible. By this I mean components should have a purpose or a direct responsibility they manage, their interactions should reflect or caricaturize what would be a meaningful interaction between two or more people. Breaking up a system in this manner allows any person familiar with the system to recount the systems behavior as the behaviors of individuals. Inherent in this is the ability for people outside of the technical space to reflect and reason about the system. This necessarily allows recruitment of more people to more realistically leverage something they have spent their entire lives doing: thinking about human relationships, as a way to scale solutions to system design problems. This is the democratization of systems design. This is story-oriented programming.

This is not an original idea. This is a reframing of a previous idea that allowed the idea to click in my own mind. James Coplein gave a talk3 at the GOTO 2017 conference where he asserted that the original conceptualization of OOP was that the ontology in the user’s mind was directly reflected in the ontology of systems design. If I see a dog on the screen, I should be manipulating a dog in code (to some degree). This immediately sheds light on the absurdity of some sorts of OO designs (the character walks into a wall object, and it’s the wall that tells the character it collided.) This idea started me down this train of thought.

An additional novetly of this particular reframing is its internal/external homoiconicity. In many of Alan Kays talks that have been put on YouTube, he talks about how his original conceptualization of scale was based on a model of something that has already scaled: the cellular model of the human body. The human body and brain are a stunning example of singular components in a deeply woven network working very well at very large scale. But no cell is really “smarter” than another cell in the sense we might talk about a human being smarter than another human.

This begs the question to me, if the human body is essentially a sort of distribution and democratization of life to the constituent cells, then other forms of democratization may be able to be leveraged to produce artifacts at scale. Dr. Kay proposed that democratizing tasks to components similar to cells in a computer system would give rise to scalable systems. This was bastardized into 90s OOP which has failed spectacularly in some cases. Rewinding back to that original concept, and reframing it slightly; if instead of cells, we conceptualized the system as a series of actors, gremlins, humans, whatever, engaged in a story, we leverage a reasoning technology for interaction that almost every human can contribute to. We scale out the problem of systems design in the very same way that we scale out the system itself.

Objections

In internalizing and fleshing out this argument, I came across a number of objections that I addressed internally, and would prefer to address point by point. These objections came in two varieties: aesthetic or pragramtic. Aesthetic objections are those that argue for preferential prejudice based on an arbitrary factor that can slide around (an aesthetic). Pragmatic objections are those that argue that the actual enactment of the suggestion is not possible due to social or psychological constraints.

Aesthetic

Reductio Ad Absurdem, the Slippery Slope, if we do it in one place we have to do it everywhere.

This is the argument that everything, including a value as simple as a number, should be an actor. I don’t know that I agree or disagree with this. I am presenting the aesthetic I use for discrimination.

The extant universe does seem to justify that communicative decomposition can go all the way down. We can (and do) talk about quarks exchanging color-charged gluons. Charged particles like protons and electrons exchange photons. These examples attribute human-like communication or intention onto what are otherwise extraordinarily simple phenomena.

There does exist a realm of phenomena that have no physical, semi or otherwise, referents. Products of formal thought systems, such as mathematics, do not exist in the physical world. This is the perfect line in the sand that can be drawn to figure out whether or not something should be modeled as an actor or a solid process.

Thus we would probably not say that the number 5 talks to another number 5 saying “plus.” Instead we would just use the lexical constructs of that formal realm. Much the same way that set theory has its own set of combinatorics, category theory theirs, and high school algebra theirs, these systems of thought form closed, self-consistent systems of reasoning in which no negotiation or other communication needs to happen.

A caveat to this is that care must be taken to understand that purity is easily violated. While you can talk about numbers and strings as if they are mathematical constructs (and thus do not require decomposition through communicative processes), should they come to violate their own purity (a mutable string, or a sufficiently large number such that decisions must be made for memory allocation), we once again enter the world of a mathematical notion with a physical referent, and the case can again be made for narrative decomposition.

More relevant to real-world scenarios is the example of an HTTP web server. The protocol specifies that bytes coming in conform to a specific shape. This shape is a string with further constraints on what is in the string. However, part of the specification allows for some aspects of this shape to be unbounded, such as the request body. In process, you can model a request as a string, and then treat it as if it’s a pure mathematical entity. That works until you run into the real world constraint of large request bodies that may consume all available memory, or for performance reasons, you don’t want to locate the request entirely in memory. This breaks the string abstraction and shows the request for what it really is, an interpretation of electrical signals over a wire. Treating it under this lower level view also reveals the temporal-spatial nature of a physical process. Thus, eschewing the string abstraction and instead looking at the request as an actor capable of negotiating its available bytes honors the underlying process, and more accurately allows navigation of the optimization concerns previously stated.

A fascinating union of these concepts is to first design a sub-system using narrative design, and then, if it’s internal communication concerns can be safely abstracted over, a library that treats the flows as data objects themselves can be created that populates the sytem and changes it in otherwise pure ways. This allows the programmer to flow back and forth between the option to use pure manipulations and narrative manipulations depending on the use case. Additionally, as is the case with business to business transactions: collections of subprocesses can treat other collections of subprocesses as a macro process for sake of abstraction over an agreed communication protocol.

Math is Simpler than Natural Language. Mathematical models simplify, natural discourse obfuscates.

There is an aesthetic consideration that communication should be accomplished in the simplest possible manner. Simplicity is often hard. When you discover something simple, it tends to be universal. The language of mathematics is simple, and because of its simplicity, it tends to be confusing. Yet there is room for a philosophical critique that the simplest language is the most universalizable in potential. This runs afoul of pragmatic concerns.

Humans are not born into this world understanding math. The first linguistic tool all humans learn is messy, full of exceptions, and irregular. It’s natural language: full of tone, nuance, and context. Above and beyond that, a young human must learn to strip away context and talk in terms of formal abstractions. Many simple never learn this skill above and beyond a certain level. Pragmatically speaking then, while mathematics, abstract or concrete, has the potential as a universal language, it also requires a level of commitment or intelligence to reach fluency in, and therefore precludes its candidacy for a democratic language.

A prime example of math establishing a generic language is in regards to communication and side effects in programming languages. Most of these sorts of effects, such as time, space, divergence, can be encapsulated using a notion from applied category theory called a monad. If you read anything about programming in aggregators like Hacker News, chances are you have run across this word. And maybe you’ve also run across functor, applicative, typeclass, composition, identity, densities, ends, cones, isomorphism, anamorphisms, catamorphisms, yoneda lemma, kan extensions… (where is that Kmett generator when you need it).

And here is the crux. It takes a long time for most people to be comfortable even hearing the language of category theory, let alone internalize it to the level required to make proper use of it or to communicate it to others. For a lot of people, the math is simply intimidating, language, symbols and all. They will never learn it.

But — almost morally speaking — why should they? Monads as they are applied in computer programming, are a formal mathematical construction that captures the idea of composing functions whose values are annotated with additional data. That last part is an opaque way of saying communication. Any function whose output is a monad is communicating something in addition to the result. That communication could be failure, it could be a log, it could be the description for communication with an external system. These communications can then be sequenced meaningfully using the monadic combinators. This is all to say that monads capture communication and treat it both abstractly, generically, and mathematically.

I have to explain that. I have to explain monads. Let that sink in. The majority of programmers know how to talk. They, more or less, understand that humans have relationships, and that relationships are navigated using communication. This is wired deep into us, and you don’t have to even be cognizant of it to leverage it. Why am I going out of my way to tell people, “hey, you already have a powerful technology at your disposal, let me show you how to talk about talking in a way that you can’t understand?” I don’t have an answer to this question. There is an aesthetic beauty to the abstractness of monads that appeal to analytical minded people like me, so I think I erroneously assume everyone else has the same aesthetics.

Finally, I think the true death-blow to this aesthetic desire is the argument that a mathematical representation of a problem is necessarily a premature optimization. It is an optimization away from the most accurate encoding to one that ignores what are considered irrelevant details. Murphy’s law puts a time limit on how long those details stay irrelevant though. If you get lucky the intersection of when the abstraction breaks, and when you have to further maintain the code does not exist. More often though, when you aggressively pursue casting solutions into abstractions that ignore too much, you will be bitten more quickly.

The function f(x, y) = x + y is perfect until x and y are not providable at the same time. Turns out there is a monad for that (future/stream). And are not locatable from the same source. Turns out there is a monad for that too (environment/reader). The number of people who understand monads really well are few. The number who understand monad transformers (composing these effects) are fewer. The implementation of monad transformers in languages without sophisticated type checkers is a nightmare. So when we picked this beautiful mathematical abstraction that we attempted to force into a new requirement domain suddenly it got a lot more complicated.

Pragmatic

In addition to these aesthetic considerations are a handful of pragmatic objections.

Silliness and Ego

One possible reaction to the concept of story oriented programming, that I experienced myself, was how humbling it was (I just felt silly when I relayed the gremlin story to a coworker out loud) to talk about components of a system as if I was telling a story to a child. In reflection, this is not unexpected. Components of a system at this time will not be a deep and nuanced human, so talking about them as anything more than something like “see spot run” will simply not happen unless aspects are attributed to the actors that they do not possess. But no matter how expected this seems, it is almost shocking in a way to experience a reframing of a problem that makes it sound like any child could solve it. That is exactly the point. To democratize a process requires that it be rendered into parts consumable by the lowest common denominator. A 5 year old mind makes a great exemplar.

Avoiding humility or feeling silly or stroking a big ego are not valid (moral) reasons to preclude something that may otherwise lead to progress in an area that is difficult. If you are a very intelligent person, and you find that you have a hard time letting go of the need to midas-touch everything you interact with, you may be contributing to the problem of difficult to maintain and difficult to reason about software.

Condescension

A second objection I considered is the pragmatic issue of transition. Due the democratizing nature of this decomposition technique, systems problems can be more easily explained to non technical people so they may offer their own narrative reasoning. However, it seems conceivable that in some organizations, if you were to hear someone who normally talks in jargonese suddenly tell you a story about something that sounds like it’s for children, it could come across as extremely demeaning or condescending.

This seems relatively easy to mitigate by either only practicing it with a subset of the culture who understands the value it brings, while educating outside of the culture, or just starting by espousing the benefits to non technical people (project managers, etc) and that it sounds the same no matter who is talking about the problem space. The language does not dumb down the exposition to the listener, it lowers the barrier of entry across the board.

Identity Dynamics

The final pragmatic consideration that came to mind is chained off a consideration about purity and impurity. Sometimes people defer to mathematics so they can wash their hands of intentionally human affairs. An equation may say nothing about time, but neither does it say anything about gender or race.

Prominent code bases have had contentious conversations around the usage of identifiers or concepts like “master/slave” as being exclusive of Black Americans, or people inappropriately referring to sexual concepts in code (0x8008135). It is a valid concern that departure from the extremely formal world of mathematics necessarily can encode language people take issue with into discussions about an otherwise unsentient system. In my own story above I only had one actor of female pronoun.

My response to this criticism is that these issues transcend the solution I am presenting. You are no less racist or sexist just because you practice colorblind or genderless programming. Microsoft’s racist Tay twitter bot is an indication that the systems we build extend and encode the memes of the systems that produce them. Precluding your system from these concerns implies that you preclude the concerns from the system producing it, that is, your team.

Conclusion

That is Story-Oriented Programming in a box and why you may want to try it. I will be frank though. The idea seems wildly radical to me, as I fall aesthetically in the camp of people who want something like math to be the lingua franca of computers. I have used this post to espouse a deeply counter-advocated position that indicts my own position as elitist and inherently unscalable in both the artifacts it produces, and the systems surrounding artifact production.

Maybe it resonates with you. If so, I encourage you to try it today. I will for sure be attempting to put these ideas into practice soon in my own personal projects. Then maybe I will be brave enough to bring these concepts into larger discussions at my company, or in productions systems. The implications of this approach are extremely far reaching, and so the change they would bring would be sweeping.

The human condition is narrative. It’s time to directly weave narrative into our software approach. It’s time for some story-oriented programming.