A list comprehension is a syntactic construct available in some programming languages for creating a list based on existing lists. It follows the form of the mathematical set-builder notation (set comprehension) as distinct from the use of map and filter functions.

Overview

Consider the following example in mathematical set-builder notation.

${\displaystyle S=\{2\cdot x\mid x\in \mathbb {N} ,\ x^{2}>3\}}$

or often

${\displaystyle S=\{2\cdot x:x\in \mathbb {N} ,\ x^{2}>3\}}$

This can be read, "${\displaystyle S}$ is the set of all numbers "2 times ${\displaystyle x}$" SUCH THAT ${\displaystyle x}$ is an ELEMENT or MEMBER of the set of natural numbers (${\displaystyle \mathbb {N} }$), AND ${\displaystyle x}$ squared is greater than ${\displaystyle 3}$."

The smallest natural number, x = 1, fails to satisfy the condition x²>3 (the condition 1²>3 is false) so 2 ·1 is not included in S. The next natural number, 2, does satisfy the condition (2²>3) as does every other natural number. Thus x consists of 2, 3, 4, 5... Since the set S consists of all numbers "2 times x" it is given by S = {4, 6, 8, 10,...}. S is, in other words, the set of all even numbers greater than 2.

In this annotated version of the example:

${\displaystyle S=\{\underbrace {2\cdot x} _{\color {Violet}{\text{output expression}}}\mid \underbrace {x} _{\color {Violet}{\text{variable}}}\in \underbrace {\mathbb {N} } _{\color {Violet}{\text{input set}}},\ \underbrace {x^{2}>3} _{\color {Violet}{\text{predicate}}}\}}$

• ${\displaystyle x}$ is the variable representing members of an input set.
• ${\displaystyle \mathbb {N} }$ represents the input set, which in this example is the set of natural numbers
• ${\displaystyle x^{2}>3}$ is a predicate expression acting as a filter on members of the input set.
• ${\displaystyle 2\cdot x}$ is an output expression producing members of the new set from members of the input set that satisfy the predicate expression.
• ${\displaystyle \{\}}$ braces indicate that the result is a set
• ${\displaystyle \mid }$ ${\displaystyle ,}$ the vertical bar is read as "SUCH THAT". The bar and the colon ":" are used interchangeably.
• commas separate the predicates and can be read as "AND".

A list comprehension has the same syntactic components to represent generation of a list in order from an input list or iterator:

• A variable representing members of an input list.
• An input list (or iterator).
• An optional predicate expression.
• And an output expression producing members of the output list from members of the input iterable that satisfy the predicate.

The order of generation of members of the output list is based on the order of items in the input.

In Haskell's list comprehension syntax, this set-builder construct would be written similarly, as:

s = [ 2*x | x <- [0..], x^2 > 3 ]

Here, the list [0..] represents ${\displaystyle \mathbb {N} }$, x^2>3 represents the predicate, and 2*x represents the output expression.

List comprehensions give results in a defined order (unlike the members of sets); and list comprehensions may generate the members of a list in order, rather than produce the entirety of the list thus allowing, for example, the previous Haskell definition of the members of an infinite list.

History

The existence of related constructs predates the use of the term "List Comprehension". The SETL programming language (1969) has a set formation construct which is similar to list comprehensions. E.g., this code prints all prime numbers from 2 to N:

print([n in [2..N] | forall m in {2..n - 1} | n mod m > 0]);

The computer algebra system AXIOM (1973) has a similar construct that processes streams.

The first use of the term "comprehension" for such constructs was in Rod Burstall and John Darlington's description of their functional programming language NPL from 1977. In his retrospective "Some History of Functional Programming Languages",^[1] David Turner recalls:

NPL was implemented in POP2 by Burstall and used for Darlington’s work on program transformation (Burstall & Darlington 1977). The language was first order, strongly (but not polymorphically) typed, purely functional, call-by-value. It also had “set expressions” e.g.

setofeven (X)  <=  <:x : x in X & even(x):>}}

In a footnote attached to the term "list comprehension", Turner also notes

I initially called these ZF expressions, a reference to Zermelo–Fraenkel set theory — it was Phil Wadler who coined the better term list comprehension.

Burstall and Darlington's work with NPL influenced many functional programming languages during the 1980s, but not all included list comprehensions. An exception was Turner's influential, pure, lazy, functional programming language Miranda, released in 1985. The subsequently developed standard pure lazy functional language Haskell includes many of Miranda's features, including list comprehensions.

Comprehensions were proposed as a query notation for databases^[2] and were implemented in the Kleisli database query language.^[3]

Examples in different programming languages

Similar constructs

>>> s = {v for v in 'ABCDABCD' if v not in 'CB'}
>>> print(s)
{'A', 'D'}
>>> type(s)
<class 'set'>
>>>

(for/set ([v "ABCDABCD"] #:unless (member v (string->list "CB")))
         v))

>>> s = {key: val for key, val in enumerate('ABCD') if val not in 'CB'}
>>> s
{0: 'A', 3: 'D'}
>>>

(for/hash ([(val key) (in-indexed "ABCD")]
           #:unless (member val (string->list "CB")))
  (values key val))

-- regular list comprehension
a = [(x,y) | x <- [1..5], y <- [3..5]]
-- [(1,3),(1,4),(1,5),(2,3),(2,4) ...

-- zipped list comprehension
b = [(x,y) | (x,y) <- zip [1..5] [3..5]]
-- [(1,3),(2,4),(3,5)]

-- parallel list comprehension
c = [(x,y) | x <- [1..5] | y <- [3..5]]
-- [(1,3),(2,4),(3,5)]

> (for*/list ([x (in-range 1 6)] [y (in-range 3 6)]) (list x y))
'((1 3) (1 4) (1 5) (2 3) (2 4) (2 5) (3 3) (3 4) (3 5) (4 3) (4 4) (4 5) (5 3) (5 4) (5 5))
> (for/list ([x (in-range 1 6)] [y (in-range 3 6)]) (list x y))
'((1 3) (2 4) (3 5))

# regular list comprehension
>>> a = [(x, y) for x in range(1, 6) for y in range(3, 6)]
[(1, 3), (1, 4), (1, 5), (2, 3), (2, 4), ...

# parallel/zipped list comprehension
>>> b = [x for x in zip(range(1, 6), range(3, 6))]
[(1, 3), (2, 4), (3, 5)]

# regular array comprehension
>>> a = [(x, y) for x in 1:5 for y in 3:5]

# parallel/zipped array comprehension
>>> b = [x for x in zip(1:3, 3:5)]

/library/book//paragraph[@style='first-in-chapter']

for $b in //book
where $b[@pages < 400]
order by $b//title
return
  <shortBook>
    <title>{$b//title}</title>
    <firstPara>{($book//paragraph)[1]}</firstPara>
  </shortBook>

map(
  newXML(shortBook, newXML(title, $1.title), newXML(firstPara, $1...))
  filter(
    lt($1.pages, 400),
    xpath(//book)
  )
)

var s = Enumerable.Range(0, 100).Where(x => x * x > 3).Select(x => x * 2);

var s = from x in Enumerable.Range(0, 100) where x * x > 3 select x * 2;

#include <algorithm>
#include <list>
#include <numeric>

using namespace std;

template<class C, class P, class T>
C comprehend(C&& source, const P& predicate, const T& transformation)
{
  // initialize destination
  C d = forward<C>(source);

  // filter elements
  d.erase(remove_if(begin(d), end(d), predicate), end(d));

  // apply transformation
  for_each(begin(d), end(d), transformation);

  return d;
}

int main()
{
  list<int> range(10);
      // range is a list of 10 elements, all zero
  iota(begin(range), end(range), 1);
      // range now contains 1, 2, ..., 10

  list<int> result = comprehend(
      range,
      [](int x) { return x * x <= 3; },
      [](int &x) { x *= 2; });
      // result now contains 4, 6, ..., 20
}

list<int> N;
list<double> S;

for (int i = 0; i < 10; i++)
    N.push_back(i);

S << list_comprehension(3.1415 * x, x, N, x * x > 3)

bool even(int x) { return x % 2 == 0; }
bool x2(int &x) { x *= 2; return true; }

list<int> l, t;
int x, y;

for (int i = 0; i < 10; i++)
     l.push_back(i);

(x, t) = l | x2;
(t, y) = t;

t = l < 9;
t = t < 7 | even | x2;

<catalog>
  <book>
    <title>Hamlet</title>
    <price>9.99</price>
    <author>
      <name>William Shakespeare</name>
      <country>England</country>
    </author>
  </book>
  <book>...</book>
...
</catalog>

// Equivalent X-Path: "catalog/book/author/name"
std::vector<name> author_names = 
evaluate(root, catalog_ >> book_ >> author_ >> name_);

// Equivalent X-Path: "catalog//name"
std::vector<name> author_names = 
evaluate(root, catalog_ >> DescendantsOf(catalog_, name_));

// Equivalent X-Path: "catalog//author[country=="England"]"
std::vector<name> author_names = 
evaluate(root, catalog_  >> DescendantsOf(catalog_, author_)
                         >> Select(author_, [](const author & a) { return a.country() == "England"; })
                         >> name_);

See also

• Set-builder notation
• The SELECT statement together with its FROM and WHERE clauses in SQL

Notes and references

• List Comprehension in The Free On-line Dictionary of Computing, Editor Denis Howe.
• Wadler, Philip (1990). "Comprehending Monads". Proceedings of the 1990 ACM Conference on LISP and Functional Programming, Nice.

External links

• SQL-like set operations with list comprehension one-liners in the Python Cookbook
• Discussion on list comprehensions in Scheme and related constructs
• List Comprehensions across languages