Functional Languages and the Future of Programming (Part 2)

The previous article in this series described the special features that make functional programming attractive. This article applies those concepts to programming today.

Some Important Influences of Functional Programming

Although functional languages have always been held at arm’s length by most programmers, who consider them difficult, academic, or inscrutable, the computer field has felt an irresistible gravitational attraction to their concepts. A survey of computing history (or a well-worded query to an AI chatbot, for the lazy) turns up numerous concepts that started in languages we call «functional» and are now in Python, Java, Rust, and other «mainstream» languages.

My favorite functional programming concept is higher-order functions. These specify a function that runs over a selection of variables or an array. The technique saves an enormous amount of space in the source code, produces reasonably readable code, and permits some very subtle programming effects. Dean Wampler, author of Programming Scala, points out that higher-order functions also map to the MapReduce idiom introduced in the «big data» era.

Type classes are critical for allowing many different objects to share an operation (polymorphism), such as a «resize» operation for geometric shapes or the use of a plus sign to combine objects of the same type.

Next for me comes pattern matching, a brilliant replacement for complicated nested if clauses or repeated else statements. Syntactically, pattern matching extends the concept of switch/case statements in the C language. In C, each case block specifies only a single constant. With pattern matching, you can specify ranges, groups of values, and special values such as null. You can even include blocks of different data types.

Closures are critical to functional programming, and have their value in imperative programming too. I see them as providing some of the advantages of multiple class inheritance, which was permitted in C++ but later seen as a poor programming practice.

Another neat trick is lexical scoping, where a variable’s use is limited to a particular block of code. This can reveal errors where variables are used when their meanings are not relevant.

I won’t go on listing the innovations in various languages. Some are important functional concepts but appear mostly as support for more visible programming techniques. Others, I feel, are more complex than they’re worth, and could be replaced with syntax that’s easier to work with.

Francesco Cesarini, author of Erlang Programming and Designing for Scalability with Erlang/OTP, points out that functional languages prefigured the development of microservices, a popular way to enforce modularity in imperative programming. A functional language carries out radical separation of functions in a way that works easily well on a single processor, on multicore systems, and over a network.

By exploiting multiple cores efficiently and keeping data in memory, functional programs can scale up very well on a single system and avoid relatively heavyweight network communications. The programs can also scale out, for modern Internet-facing loads.

The Daunting Gulf

But beyond the clever syntax that has spread from functional programming throughout the field of programming languages, prospective programmers have to grapple with paradigms that are very different from imperative programming. These paradigms continue to make functional languages (however hard to define) a very separate endeavor, and one that most programmers don’t want to tackle.

In imperative languages, loops represent an intuitive way to handle repetition, which is familiar from factory automation or the envelope-stuffing lines that non-profits still use to put together bulk mailings. Replacing loops with recursion (preferably tail recursion) in functional languages is a straightforward technique that can be taught and learned.

Where I think functional programming gets hard is the fundamental, iron-clad ban on side effects. Its effect is that functions can’t share variables (and certainly not global variables) to track changes in the program and its environment.

One coping mechanism is the use of cached memory for state data, as Cesarini explained to me. (This is also where Simon Peyton Jones’s transactional memory, mentioned in the first article of the series, is valuable.) The IO Monad in Haskell also helps maintain state. Another author writes, «in Haskell, we try to keep things pure until the outermost layers.»

Aside from this replacement for global variables, statements and functions in functional languages wrap around each other in ways that are much harder for most learners to grasp than the mostly hierarchical architecture of imperative languages.

Where is Programming Headed?

To help explain the trajectories of the functional programming languages, I want to offer the reader a bit of history—specifically, the history of the card game, contract bridge.

Bridge emerged in the early 20th century when innovators added a subtle bidding system to the simple game of whist. For decades, Americans everywhere used a bidding system invented by Charles H. Goren, and his books were on every player’s shelf.

While bidding was highly nuanced and rewarded sensitive thinking, Goren’s system itself was intuitive. If you had a lot of high cards and a long string of spades, you would bid «one spade.» Then your partner, if they had good cards and a long string of diamonds, would bid «two diamonds» (because each bid needed to be higher than the previous one).

There were a few filigrees hanging from this basic structure, such as «weak two bids» and other «pre-emptive» bids meant to take a deliberate loss in order to prevent the opponents from having a bigger win. But basically, the system made sense, and millions of neighbors wiled away convivial hours around their bridge tables.

But advanced players—and particularly players of duplicate bridge, which is used in bridge competitions—noticed that the weakest, lowest build of «one club» wasn’t of much use. So they thought up the idea of a «strong club,» where a bid of «one club» had nothing to do with your club suit, but just meant «I have a strong hand.» This would give you and your partner plenty of room to explore the best final bid to make.

Inventing «strong club» systems made very little difference in most bridge hands, but it was critically important in duplicate bridge because it might let a pair pull slightly ahead of players at the other tables when a hand was really strong. So «strong club» spread fast.

But once «one club» lost its natural meaning and became purely symbolic, it was necessary to add other symbolic bids to the system, and bidding often lost all connection to the intuitive meaning of bids. Furthermore, once bidding became purely symbolic, the field was opened up to endless competing «strong club» systems. To pile on yet more complexity, the bidding systems had to deal with interference from opponents using other bidding systems.

Basically, bridge was no longer fun. It became a competitive activity for professionals, not a casual way to spend a few hours with your buddies. Serious players would have to memorize several non-intuitive «strong club» bidding systems. If they didn’t, every utterance at the bridge table would have to be followed by opponents asking, «What does that mean in your bidding system?»

Now I get to the point of this game history. Many programming languages—including, I claim, functional ones—do to computing what «strong club» did to bridge. These languages called for highly complex, layered thinking. A forbidding wall has risen between the professional programmers and the amateurs who write their own small, useful programs by learning a simple language. (Think back on Basic, and later on Perl, Tcl, Ruby, and Python.)

Functional languages are not the only ones to demand professional training. If you want to write an app for an iPhone or other small Apple device, using Apple’s Swift language, you had better familiarize yourself with Swift’s puzzling memory management (which it inherited from Objective-C, the original language used for iPhone apps). Dragons also lurk if you try to learn the language that is the current darling of computer scientists: Rust.

Well, maybe the walling off of production-ready programming is ultimately good for us all, because amateur programs are buggy, hard to maintain, and too often targets for security attacks. An article on transactional memory, cited in the first of article of this series, ends: «Who knows. Maybe in 10 years, the world will be transactional after all.»

In the early 2000s, many observers thought that everyone would need to program; the movement was called «computer literacy.» The need for programming looks very different today, as software 1) moves into libraries and online services, 2) finds AI assists, 3) gets built into hardware. Programmers of the later twenty-first century may be higher paid, more highly educated, and fewer.

<< Read the first part of this article