Advanced D Programming Language Features

by Walter Bright, Digital Mars

Are you bored with the usual mcguffins - expressions, statements, and core strings vs library strings? Let's skip right to the naughty bits - foreach, lambdas, mixins, tuples, scope guards, lazy arguments, and static if's.

Note: All of this stuff can be done with C++, one way or another, although the result might not be worth the effort.

Foreach

Loop over an array in C:

void foo(int array[10])
{
    for (int i = 0; i < 10; i++)
    {   int value = array[i];
	... do something ...
    }
}

Issues:

i should be size_t, not int
array is not checked for overflow
10 may not be the actual array dimension may be more efficient to step through the array with pointers, rather than indices
type of array may change, but the type of value may not get updated
crashes if array is NULL
only works with arrays and pointers

In C++:

void foo(std::vector array)
{
  for (std::vector::const_iterator
       i = array.begin();
       i != array.end();
       i++)
  {
    int value = *i;
    ... do something ...
  }
}

Issues:

type of array may change, but the type of value may not get updated
too darned much typing

In D:

void foo(int[] array)
{
  foreach (value; array)
  {
    ... do something ...
  }
}

Solutions:

correct loop index type is selected based on the type of array
arrays carry with them their dimension, so foreach is guaranteed to step through the loop the correct number of times
implementation decides if pointers will do a better job than indices, based on the compilation target
type of value is inferred automatically from the type of array, so no worries if the type changes
Null arrays have 0 length, so no crashing
works with any collection type

Scope Guards

Transaction processing:

Transaction abc()
{
  Foo f;
  Bar b;

  f = dofoo();
  b = dobar();

  return Transaction(f, b);
}

Both dofoo() and dobar() must succeed, or the transaction has failed. If the transaction failed, the data must be restored to the state where neither dofoo() nor dobar() have happened. To support that, dofoo() has an unwind operation, dofoo_undo(Foo f) which will roll back the creation of a Foo. With the RAII approach:

class FooX
{
  Foo f;
  bool commit;

  this()
  {
    f = dofoo();
  }

  ~this()
  {
    if (!commit)
      dofoo_undo(f);
  }
}

Transaction abc()
{
  scope f = new FooX();
  Bar b = dobar();
  f.commit = true;
  return Transaction(f.f, b);
}

With the try-finally approach:

Transaction abc()
{
  Foo f;
  Bar b;

  f = dofoo();
  try
  {
    b = dobar();
    return Transaction(f, b);
  }
  catch (Object o)
  {
    dofoo_undo(f);
    throw o;
  }
}

These work too, but have the same problems. The RAII approach involves the creation of dummy classes, and the obtuseness of moving some of the logic out of the abc() function. The try-finally approach is wordy even with this simple example; try writing it if there are more than two components of the transaction that must succeed. It scales poorly. The scope(failure) statement solution looks like:

Transaction abc()
{
  Foo f;
  Bar b;

  f = dofoo();
  scope(failure) dofoo_undo(f);

  b = dobar();

  return Transaction(f, b);
}

The dofoo_undo(f) only is executed if the scope is exited via an exception. The unwinding code is minimal and kept aesthetically where it belongs. It scales up in a natural way to more complex transactions:

Transaction abc()
{
  Foo f;
  Bar b;
  Def d;

  f = dofoo();
  scope(failure) dofoo_undo(f);

  b = dobar();
  scope(failure) dobar_unwind(b);

  d = dodef();

  return Transaction(f, b, d);
}

There are also:

scope(exit)
scope(success)

Lazy Arguments

Consider the code:

void test(int* p)
{
  if (p && p[0])
    ...
}

The second argument is lazilly evaluated. Wouldn't it be nice to do that with function arguments? Consider a logging function:

void log(char[] message)
{
  if (logging)
    fwritefln(logfile, message);
}

Use it like:

void foo(int i)
{
  log("Entering foo() with i = "
    ~ toString(i));
}

But the argument is always evaluated. The usual solution is a macro:

#define LOG(string) \
    (logging && log(string))

The D solution uses lazy arguments:

void log(lazy char[] dg)
{
  if (logging)
    fwritefln(logfile, dg());
}

void foo(int i)
{
  log("Entering foo() with i = "
     ~ toString(i));
}

Suppose an expression is to be evaluated count times. The pattern is:

for (int i = 0; i < count; i++)
   exp;

This pattern can be encapsulated in a function using lazy arguments:

void dotimes(int count,
    lazy void exp)
{
  for (int i = 0; i < count; i++)
    exp();
}

It can be used like:

void foo()
{
  int x = 0;
  dotimes(10, writef(x++));
}

printing:

0123456789

More complex user defined control structures are possible. Here's a method to create a switch like structure:

bool scase(bool b, lazy void dg)
{
  if (b)
    dg();
  return b;
}

/* Here the variadic arguments are
   converted to delegates in this
   special case.
 */
void cond(bool delegate()[] cases ...)
{
  foreach (c; cases)
  { if (c())
      break;
  }
}

which can be used like:

void foo()
{
 int v = 2;
 cond
 (
  scase(v == 1, writefln("it's 1")),
  scase(v == 2, writefln("it's 2")),
  scase(v == 3, writefln("it's 3")),
  scase(true, writefln("the default"))
 );
}

which will print:

it's 2

Note the parallels with Lisp. There is the common pattern:

Abc p;
p = foo();
if (!p)
  throw new Exception("foo() failed");
p.bar();	// now use p

Because throw is a statement, not an expression, expressions that need to do this need to be broken up into multiple statements, and extra variables are introduced. (For a thorough treatment of this issue, see Andrei Alexandrescu and Petru Marginean's paper Enforcements). With lazy evaluation, this can all be encapsulated into a single function:

Abc Enforce(Abc p, lazy char[] msg)
{
  if (!p)
    throw new Exception(msg());
  return p;
}

and the opening example above becomes simply:

Enforce(foo(), "foo() failed").bar();

and 5 lines of code become one. Enforce can be improved by making it a template function:

T Enforce(T)(T p,  lazy char[] msg)
{
  if (!p)
    throw new Exception(msg());
  return p;
}

Lambdas

Lambdas are function literals. For example:

double foo(double[] a, double c)
{
  return apply(a,
     (double x) { return x * x * c; }
     );
}

T apply(T) (T[] a, T function(T x) f)
{
  T result;

  foreach (x; a)
    result += f(x);
  return result;
}

Tuples

To create a tuple:

template Tuple(E...)
{
  alias E Tuple;
}

Tuple!(int, long, float)
Tuple!(3, 7, 'c')
Tuple!(int, 8)

In order to symbolically refer to a tuple, use an alias:

alias Tuple!(float, float, 3) TP;
alias Tuple!(TP, 8) TR;
alias Tuple!(TP, TP) TS;

Tuples are like arrays:

TP.length       // = 3
TP[1] f = TP[2]; // float f = 3
alias TP[0..length-1] TQ;//(float,float)

Tuples are compile time creatures:

void foo(int i)
{
  TQ[i] x; // error, i is not constant
}

Head and tail:

TP[0]         // tuple head
TP[1..length] // tuple tail

To returns a tuple consisting of the trailing type arguments TL with the first occurrence of the first type argument T removed:

template Erase(T, TL...)
{
  static if (TL.length == 0)
    // 0 length tuple, return self
    alias TL Erase;
  else static if (is(T == TL[0]))
    // match with first in tuple,
    // return tail
    alias TL[1 .. length] Erase;
  else
    // no match,
    // return head concatenated with
    // recursive tail operation
    alias Tuple!(TL[0],
          Erase!(T, TL[1..length]))
              Erase;
}

Type tuples (type lists) have elements that are solely types. Function parameter lists are a list of types, a type tuple can be retrieved from them:

int foo(int x, long y);

if (is(foo P == function))
  alias P TP;
// TP is now Tuple!(int, long)

This is generalized in the template std.traits.ParameterTypeTuple:

import std.traits;

// TP is now Tuple!(int, long)
alias ParameterTypeTuple!(foo) TP;

TypeTuples can be used to declare a function:

float bar(TP); // float bar(int, long)

If implicit function template instantiation is being done, the type tuple representing the parameter types can be deduced:

int foo(int x, long y);

void Bar(R, P...)(R function(P))
{
 writefln("ret type = ", typeid(R));
 writefln("param types = ",typeid(P));
}

Bar(&foo);

Prints:

ret type = int
param types = (int,long)

Expression tuples have elements that are all expressions.

alias Tuple!(3, 7L, 6.8) ET;

...
writefln(ET);          // prints 376.8
writefln(ET[1]);        // prints 7
writefln(ET[1..length]);// prints 76.8

It can be used to create an array literal:

alias Tuple!(3, 7, 6) AT;

...
int[] a = [AT];		// same as [3,7,6]

.tupleof property turns fields into expression tuples:

struct S { int x; long y; }

void foo(int a, long b)
{
  writefln(a, b);
}

...
S s;
s.x = 7;
s.y = 8;
foo(s.x, s.y);	// prints 78
foo(s.tupleof);	// prints 78
s.tupleof[1] = 9;
s.tupleof[0] = 10;
foo(s.tupleof); // prints 109
s.tupleof[2] = 11;// err, no 3rd field

A type tuple can be created from the data fields of a struct using typeof:

// prints (int,long)
writefln(typeid(typeof(S.tupleof)));

Looping over tuple elements:

alias Tuple!(int, long, float) TL;
foreach (i, T; TL)
  writefln("TL[%d] = ", i, typeid(T));

alias Tuple!(3, 7L, 6.8) ET;
foreach (i, E; ET)
  writefln("ET[%d] = ", i, E);

Prints:

TL[0] = int
TL[1] = long
TL[2] = float
ET[0] = 3
ET[1] = 7
ET[2] = 6.8

Tuple Declarations A variable declared with a TypeTuple becomes an ExpressionTuple:

alias Tuple!(int, long) TL;

void foo(TL tl)
{
  writefln(tl, tl[1]);
}

foo(1, 6L);	// prints 166

An example:

print(7, 'a', 6.8);

should output:

7
'a'
6.8

C++:

#include 
using namespace::std;

void print()
{
}

template<class T1>
    void print(T1 a1)
{
  cout << a1 << endl;
}

template<class T1, class T2>
    void print(T1 a1, T2 a2)
{
  cout << a1 << endl;
  cout << a2 << endl;
}

template<class T1, class T2, class T3>
    void print(T1 a1, T2 a2, T3 a3)
{
  cout << a1 << endl;
  cout << a2 << endl;
  cout << a3 << endl;
}

Issues:

must decide in advance what the maximum number of arguments
logic must be cut and paste duplicated, then modified
no obvious connection between the function overloads
source code bloat

Douglas Gregor's C++ Extension

void print()
{
}

template void print(T a1, U... an)
{
  cout << a1 << newline;
  print(an...);
}

Translating the Variadic C++ Solution into D

void print()()
{
}

void print(T, A...)(T t, A a)
{
  writefln(t);
  print(a);
}

D Foreach Solution

void print(A...)(A a)
{
  foreach(t; a)
    writefln(t);
}

Mixins

Mixins add boilerplate declarations:

template Foo()
{
  int x = 5;
}

mixin Foo;

struct Bar
{
  mixin Foo;
}

void test()
{
  mixin Foo;
  writefln(x);  // prints 5
}

Mixins can be parameterized:

template Foo(T)
{
  T x = 5;
}

// create x of type int
mixin Foo!(int);

Mixins can add virtual functions to a class:

template Foo()
{
  void func()
  {
    printf("Foo.func()\n");
  }
}

class Bar
{
  mixin Foo;
}

class Code : Bar
{
  void func()
  {
    printf("Code.func()\n");
  }
}

void test()
{
  Bar b = new Bar();
  b.func(); // calls Foo.func()

  b = new Code();
  b.func(); // calls Code.func()
}

Mixins are evaluated in the scope of where they appear, not the scope of the template declaration:

int y = 3;

template Foo()
{
  int abc() { return y; }
}

void test()
{
  int y = 8;
  mixin Foo;  // local y is picked up
  assert(abc() == 8);
}

Mixins can parameterize symbols using alias parameters:

template Foo(alias b)
{
  int abc() { return b; }
}

void test()
{
  int y = 8;
  mixin Foo!(y);
  assert(abc() == 8);
}

Static Ifs

Static if's are D's replacement for #if/#endif:

#define FOO 1
void foo(int i)
{
#if FOO
  int j;
#else
  long j;
#endif
  return j + i;
}

const int FOO = 1;
void foo(int i)
{
  static if (FOO)
    int j;
  else
    long j;
  return i + j;
}

Static if really starts to shine, however, in templates:

template<int n> class factorial
{
 public:
  enum
  {
    result =
      n * factorial<n - 1>::result
  }; 
};

template<> class factorial<1>
{
 public:
  enum { result = 1 };
};

void test()
{
  // prints 24
  printf("%d\n",
    factorial<4>::result);
}

Recursion works as well in D, though with significantly less typing:

template factorial(int n)
{
  const factorial =
     n * factorial!(n-1);
}

template factorial(int n : 1)
{
  const factorial = 1;
}

void test()
{
  // prints 24 
  writefln(factorial!(4));
}

Through using the static if construct it can be done in just one template:

template factorial(int n)
{
  static if (n == 1)
    const factorial = 1;
  else
    const factorial =
      n * factorial!(n-1);
}

More things we can do with static if's:

import std.stdio;

template sqrt(real x,
              real root = x/2,
              int ntries = 0)
{
  static if (ntries == 5)
   // precision doubles with each
   // iteration, 5 should be enough
   const sqrt = root;
  else static if (root*root - x == 0)
   const sqrt = root;  // exact match
  else
   // iterate again
   const sqrt =
    sqrt!(x,(root+x/root)/2,ntries+1);
}

void main()
{
  real x = sqrt!(2);
  // 1.4142135623730950487
  writefln("%.20g", x);
}

Compute hash of string literal at compile time:

template hash(char [] s, uint sofar=0)
{
  static if (s.length == 0)
    const hash = sofar;
  else
    const hash =
      hash!(s[1..length],
            sofar*11 + s[0]);
}

uint foo()
{
  return hash!("hello world");
}