Direct functions (dfns)

Kenneth E. Iverson, the inventor of APL, was dissatisfied with the way user functions were defined. In 1974, he devised "formal function definition" or "direct definition" for use in exposition.^[7] A direct definition has two or four parts, separated by colons:

name : expression
name : expression0 : proposition : expression1

Within a direct definition, ⍺ denotes the left argument and ⍵ the right argument. In the first instance, the result of expression is the result of the function; in the second instance, the result of the function is that of expression0 if proposition evaluates to 0, or expression1 if it evaluates to 1. Assignments within a direct definition are dynamically local. Examples of using direct definition are found in the 1979 Turing Award Lecture^[8] and in books and application papers.^{[9][10][11][12][13]}

Direct definition was too limited for use in larger systems. The ideas were further developed by multiple authors in multiple works,^{[14][15][16][17][18][19][20]} but the results were unwieldy. Of these, the "alternative APL function definition" of Bunda in 1987^[19] came closest to current facilities, but is flawed in conflicts with existing symbols and in error handling which would have caused practical difficulties, and was never implemented. The main distillates from the different proposals were that (a) the function being defined is anonymous, with subsequent naming (if required) being effected by assignment; (b) the function is denoted by a symbol and thereby enables anonymous recursion.^[13]

In 1996, Scholes invented direct functions or dfns (pronounced "dee funs"), a major distinguishing advance of early 21st century APL over prior versions.^{[21][22][23][24]} Dfns are a unique combination of array programming, higher-order functions, and functional programming. The ideas originated in 1989 when he read a special issue of The Computer Journal on functional programming.^[25] He then proceeded to study functional programming and became strongly motivated ("sick with desire", like Yeats) to bring these ideas to APL.^[23][24] He initially operated in stealth because he was concerned the changes might be judged too radical and an unnecessary complication of the language; other observers say that he operated in stealth because Dyalog colleagues were not so enamored and thought he was wasting his time and causing trouble for people. Dfns were first presented in the Dyalog Vendor Forum at the APL '96 Conference and released in Dyalog APL in early 1997.^[21] Acceptance and recognition were slow in coming. As late as 2008, in Dyalog at 25,^[6] a publication celebrating the 25th anniversary of Dyalog Ltd, dfns were barely mentioned (mentioned twice as "dynamic functions" and without elaboration). As of 2019, dfns are implemented in Dyalog APL,^[26] NARS2000,^[27] and ngn/apl.^[28] They also play a key role in efforts to exploit the computational capabilities of a GPU (graphics processing unit).^[29][13]

Dfns are illustrated here with an example. Much more extensive explanation and examples are found in the direct functions article and in the references.^[22][13][30]

Quicksort on an array ⍵ works by choosing a "pivot" at random among its major cells, then catenating the sorted major cells which strictly precede the pivot, the major cells equal to the pivot, and the sorted major cells which strictly follow the pivot, as determined by a comparison function ⍺⍺. Defined as a dop (direct operator) Q:

   Q←{1≥≢⍵:⍵ ⋄ (∇ ⍵⌿⍨0>s)⍪(⍵⌿⍨0=s)⍪∇ ⍵⌿⍨0<s←⍵ ⍺⍺ ⍵⌷⍨?≢⍵}

   ⍝ precedes            ⍝ follows            ⍝ equals
   2 (×-) 8              8 (×-) 2             8 (×-) 8
¯1                    1                    0

   x← 2 19 3 8 3 6 9 4 19 7 0 10 15 14

   (×-) Q x
0 2 3 3 4 6 7 8 9 10 14 15 19 19

Q3 is a variant that catenates the three parts enclosed by the function ⊂ instead of the parts per se. The three parts generated at each recursive step are apparent in the structure of the final result. Applying the function derived from Q3 to the same argument multiple times gives different results because the pivots are chosen at random. In-order traversal of the results does yield the same sorted array.

   Q3←{1≥≢⍵:⍵ ⋄ (⊂∇ ⍵⌿⍨0>s)⍪(⊂⍵⌿⍨0=s)⍪⊂∇ ⍵⌿⍨0<s←⍵ ⍺⍺ ⍵⌷⍨?≢⍵}

   (×-) Q3 x
┌────────────────────────────────────────────┬─────┬┐
│┌──────────────┬─┬─────────────────────────┐│19 19││
││┌──────┬───┬─┐│6│┌──────┬─┬──────────────┐││     ││
│││┌┬─┬─┐│3 3│4││ ││┌┬─┬─┐│9│┌┬──┬────────┐│││     ││
│││││0│2││   │ ││ ││││7│8││ │││10│┌──┬──┬┐││││     ││
│││└┴─┴─┘│   │ ││ ││└┴─┴─┘│ │││  ││14│15││││││     ││
││└──────┴───┴─┘│ ││      │ │││  │└──┴──┴┘││││     ││
││              │ ││      │ │└┴──┴────────┘│││     ││
││              │ │└──────┴─┴──────────────┘││     ││
│└──────────────┴─┴─────────────────────────┘│     ││
└────────────────────────────────────────────┴─────┴┘
   (×-) Q3 x
┌───────────────────────────┬─┬─────────────────────────────┐
│┌┬─┬──────────────────────┐│7│┌────────────────────┬─────┬┐│
│││0│┌┬─┬─────────────────┐││ ││┌──────┬──┬────────┐│19 19│││
│││ │││2│┌────────────┬─┬┐│││ │││┌┬─┬─┐│10│┌──┬──┬┐││     │││
│││ │││ ││┌───────┬─┬┐│6│││││ │││││8│9││  ││14│15││││     │││
│││ │││ │││┌┬───┬┐│4│││ │││││ │││└┴─┴─┘│  │└──┴──┴┘││     │││
│││ │││ │││││3 3│││ │││ │││││ ││└──────┴──┴────────┘│     │││
│││ │││ │││└┴───┴┘│ │││ │││││ │└────────────────────┴─────┴┘│
│││ │││ ││└───────┴─┴┘│ │││││ │                             │
│││ │││ │└────────────┴─┴┘│││ │                             │
│││ │└┴─┴─────────────────┘││ │                             │
│└┴─┴──────────────────────┘│ │                             │
└───────────────────────────┴─┴─────────────────────────────┘

The above formulation is not new; see for example Figure 3.7 of the classic The Design and Analysis of Computer Algorithms.^[31] However, unlike the pidgin ALGOL program in Figure 3.7, Q and Q3 are executable, and the partial order used in the sorting is an operand, the (×-) the examples above.^[13]

Articles and presentations

Wit

Scholes was well known among colleagues for his wit, sense of humor, and comic timing. His "after dinner" presentations at Dyalog conferences were highly anticipated events. A selection of them from the list above:

• 2008 A Plea for Simplicity (video)^[52]
• 2009 Complex Numbers (video)^[57]
• 2011 What is Functional Programming? (video)^[65]
• 2012 State-Free Programming (video)^[69]
• 2012 Calling Alan Turing (extract of previous item, video)^[70]
• 2013 Social Skills for Programmers (video)^[74]
• 2014 Distractions (video)^[75]

Other examples can be found in Scholisms.^[86]