Using Variadic Templates cleanly

When one comes across examples for variadic templates, almost always recursion is used to achieve almost everything, for example like this:

// We are lucky: the author correctly used zero
// arguments instead of one as the base-case,
// thereby avoiding code-duplication:
inline void print_to_stream(std::ostream&) {}

template<typename Head, typename...Tail>
void print_to_stream(std::ostream& stream, const Head& h, const Tail&... t) {
	stream << h;
	print_to_stream(stream, t...);
}

In this article we will see what better alternatives for this rather clumsy hack exist and see how we can write a better version with less code.

Before we start, we will however need one very simple utility-function:

template<typename...Ts>
void ignore_returnvalues(Ts&&...) {}

This function really does nothing but accept any argument we might pass to it and ignores them. Now that we have that, we can start by looking at a somewhat simpler problem: We want to write a functiontemplate that accepts a variadic number of iterators and increments each of them once.

The traditionall approach to this would be to take out recursion and throw that at the problem, but for once we will think before we code: In the recursive call, we would use the elipses to expand the tail of the arguments to a list of parameters for the function (see t... above). What comes somewhat surprising to many is the fact that we may apply functions to each of the elements in the tail (like fun(t)...) which will expand to the returnvalues of those functions being used as the parameters. If we now do this with the head and the tail by not artificially splitting them up, we can call a function on each of those arguments.

In fact, not just a function: We can use almost any expression and we will still get a similar effect. This allows us to implement increment_all like this:

template<typename... Iterators>
void increment_all(Iterators&... its) {
	ignore_returnvalues((++its)...);
}

This really is everything it takes. In fact this is now so simple that there wouldn’t even be a need for a specific function in many cases, because it is so little code.

Now why do we need ignore_returnvalues here? The answer is basically that we cannot create a variadic number of statements from one expression and therefore we need to find a way to dispose of all the expressions, which ignore_returnvalues happily does.

Another way would however be to just use those returnvalues, for instance to return a tuple of the new iterators:

template<typename... Iterators>
std::tuple<Iterators...> increment_all(Iterators&... its) {
	return std::make_tuple((++its)...);
}

We could of course also use auto as the returntype here, but some people just left the stone-age (C++98) and are still unwilling to progress past the bronze-age (C++11) into a more modern age (C++14), so let’s take a look at some other possible return-types.

For that we will pick a somewhat different problem: We want to transform a variadic number of containers into a variadic number of iterator-pairs (we will use a utility-type for those pairs). The problem here is that we do not yet have the exact types of the iterators. But again, we can just treat a type-pack as if it were a single type and expand that:

template<typename Iterator>
struct range{
	Iterator first;
	Iterator last;
};
template<typename Iterator>
range<Iterator> make_range(Iterator first, Iterator last) {
	return {first, last};
}

template<typename...Containers>
std::tuple<range<typename Containers::const_iterator>...>
to_ranges(const Containers&... conts) {
	return std::make_tuple(make_range(conts.begin(), conts.end())...);
}

Again: No magic so far. But what if we don’t want to rely on the container providing a const_iterator-typedef? In that case we can still use decltype:

template<typename...Containers>
std::tuple<range<decltype(std::declval<Containers>().begin())>...>
to_ranges(const Containers&... conts) {
	return std::make_tuple(make_range(conts.begin(), conts.end())...);
}

If you don’t like std::declval, which is a undefined function that claims to return exactly it’s template-argument-type, we can also use trailing returntypes:

template<typename...Containers>
auto to_ranges(const Containers&... conts)
-> std::tuple<range<decltype(conts.begin())>...>
{
	return std::make_tuple(make_range(conts.begin(), conts.end())...);
}

When we are at it: How would you do those things with recursion?

But let’s get back to usefull stuff instead of looking at ways to support those who cannot get an even somewhat recent compiler.

The next problem we will look at, is to calculate the sum of the sums of the elements of some containers. We will assume for simplicity that all those containers contain something reasonably convertible to doubles, but it wouldn’t be overly hard to figure out the common type using std::common_type.

template<typename...Containers>
double sum_of_sums(const Containers&... conts) {
	const auto sums = std::initializer_list<double>{
		std::accumulate(conts.begin(), conts.end(), 0.0)...
	};
	return std::accumulate(sums.begin(), sums.end(), 0.0);
}

We could have used a container like std::vector here as well, but why introduce all that overhead when an initializer-list would have to be constructed anyways and it is perfectly sufficient? That aside, notice how we don’t need any control-structures that always come with an increased chance of bugs. This is perfectly linear code.

The initializer-list here comes with another very important feature that we don’t need in this example, but for many others: It guarantees that the expressions that calculate it’s arguments are executed in left-to-right order, whereas normally there are almost no guarantees at all, about the evaluation-order of function-arguments. (Not that this is a bad thing: I don’t want to have to read code that depends on something like that!)

Back to the initial problem: We wanted to print a variadic number of things to a std::ostream. Let’s introduce a helper-method for a moment that we will soon drop again:

template<typename T>
int print_to_stream(std::ostream& stream, const T& arg) {
	stream << arg;
	return 0;
}

With this we can obviously implement a first version that looks like this:

template<typename...Ts>
void print_all(std::ostream& stream, const Ts&... args) {
	(void) std::initializer_list<int>{print_to_stream(stream, args)...};
}

(The cast to void is just there to silence compiler warnings about unused temporaries.)

The reason for us to use the print_to_stream function is that we cannot create an initializer_list of ostreams from the returnvalues (lvalue-references) of the actual printing. And even if we could, we wouldn’t want, because the compiler couldn’t really optimize that away and those copies would be expensive.

A better way is however to remember the comma-operator that executes the expression to it’s left, discards it’s result (if any), executes the expresssion to the right and returns it’s result. We can use that to execute and arbitrary expression and then return 0:

template<typename...Ts>
void print_all(std::ostream& stream, const Ts&... args) {
	(void) std::initializer_list<int>{(stream << args, 0)...};
}

This works perfectly well without any further helpers, except for the case where some terrorist overloaded the comma-operator. We can however protect against that as well, by using a lambda:

template<typename...Ts>
void print_all(std::ostream& stream, const Ts&... args) {
	(void) std::initializer_list<int>{
		[&](const auto& arg){stream << arg; return 0;}(args)...
	};
}

Obviously we can generalize that further to a sequential for-each, by passing the actual function to be called on all arguments as another argument:

template<typename Fun, typename...Ts>
void sequential_foreach(Fun f, const Ts&... args) {
	(void) std::initializer_list<int>{
		[&](const auto& arg){f(arg); return 0;}(args)...
	};
}

// Now we can do this:

template<typename...Ts>
void print_all(std::ostream& stream, const Ts&... args) {
	sequential_foreach([&](const auto& arg){stream << arg;}, args...);
}

The function-object that we pass to sequential_foreach doesn’t have to be a polymorphic lambda by the way: Every function-object that is callable with all the argument-types is perfectly suitable, Even nonoverloaded functions are perfectly fine, if they can be called with all the arguments.

Another topic that we haven’t talked about really yet are the error-messages by the compiler: Since our callstack is much flatter, those also tend to be a lot shorter and to the point. If we compare what happens if we want to print ten things of which the last one is not printable, this becomes fairly obvious. Skipping the list of ignored overloads that is equal for all three solutions (the naive one, the direct one and the final one), this is what the naive solution results in:

../../src/main.cpp:13:9: error: invalid operands to binary expression
        ('std::ostream' (aka 'basic_ostream<char>') and 'const unprintable')
        stream << h;
        ~~~~~~ ^  ~
../../src/main.cpp:14:2: note: in instantiation of function template
        specialization 'print_all_naive<unprintable>' requested here
        print_all_naive(stream, tail...);
        ^
../../src/main.cpp:14:2: note: in instantiation of function template
        specialization 'print_all_naive<int, unprintable>' requested
        here
        print_all_naive(stream, tail...);
        ^
../../src/main.cpp:14:2: note: in instantiation of function template
        specialization 'print_all_naive<int, int, unprintable>'
        requested here
        print_all_naive(stream, tail...);
        ^
../../src/main.cpp:14:2: note: in instantiation of function template
        specialization 'print_all_naive<int, int, int, unprintable>'
        requested here
        print_all_naive(stream, tail...);
        ^
../../src/main.cpp:14:2: note: in instantiation of function template
        specialization 'print_all_naive<int, int, int, int, unprintable>'
        requested here
        print_all_naive(stream, tail...);
        ^
../../src/main.cpp:14:2: note: in instantiation of function template
        specialization 'print_all_naive<int, int, int, int, int,
        unprintable>' requested here
        print_all_naive(stream, tail...);
        ^
../../src/main.cpp:14:2: note: in instantiation of function template
        specialization 'print_all_naive<int, int, int, int, int, int,
        unprintable>' requested here
        print_all_naive(stream, tail...);
        ^
../../src/main.cpp:14:2: note: in instantiation of function template
        specialization 'print_all_naive<int, int, int, int, int, int,
        int, unprintable>' requested here
        print_all_naive(stream, tail...);
        ^
../../src/main.cpp:14:2: note: in instantiation of function template
        specialization 'print_all_naive<int, int, int, int, int, int,
        int, int, unprintable>' requested here
        print_all_naive(stream, tail...);
        ^
../../src/main.cpp:37:2: note: in instantiation of function template
        specialization 'print_all_naive<int, int, int, int, int, int,
        int, int, int, unprintable>' requested here
        print_all_naive(std::cout, 1,2,3,4,5,6,7,8,9, unprintable{});
        ^

The final version is already shorter, but most importantly, it focuses on the point and is of constant length with regards to the number of arguments we used:

../../src/main.cpp:31:49: error: invalid operands to binary expression
        ('std::ostream' (aka 'basic_ostream<char>') and 'const unprintable')
        sequential_foreach([&](const auto& arg){stream << arg;}, args...);
                                                ~~~~~~ ^  ~~~
../../src/main.cpp:25:24: note: in instantiation of function template
        specialization 'print_all_final(std::ostream &, const int &,
        const int &, const int &, const int &, const int &, const int &,
        const int &, const int &, const int &, const unprintable &
        )::(anonymous class)::operator()<unprintable>' requested here
                [&](const auto& arg){f(arg); return 0;}(args)...
                                     ^
../../src/main.cpp:25:3: note: in instantiation of function template
        specialization 'sequential_foreach((lambda at ../../src/main.cpp:31:21),
        const int &, const int &, const int &, const int &, const int &,
        const int &, const int &, const int &, const int &, const unprintable &
        )::(anonymous class)::operator()<unprintable>' requested here
                [&](const auto& arg){f(arg); return 0;}(args)...
                ^
../../src/main.cpp:31:2: note: in instantiation of function template
        specialization 'sequential_foreach<(lambda at ../../src/main.cpp:31:21),
        int, int, int, int, int, int, int, int, int, unprintable>' requested here
        sequential_foreach([&](const auto& arg){stream << arg;}, args...);
        ^
../../src/main.cpp:39:2: note: in instantiation of function template
        specialization 'print_all_final<int, int, int, int, int, int,
        int, int, int, unprintable>' requested here
        print_all_final(std::cout, 1,2,3,4,5,6,7,8,9, unprintable{});
        ^

Granted, we now have to look through two lambdas, which suggests that the best solution for a library that regards error-messages as very important and considers people who overload the comma-operator as so bad that they don’t deserve to be considered (libstdc++ does this btw.), might be the direct implementation which results in this very short and to the point message:

../../src/main.cpp:19:44: error: invalid operands to binary expression
        ('std::ostream' (aka 'basic_ostream<char>') and 'const unprintable')
        (void) std::initializer_list<int>{(stream << args, 0)...};
                                           ~~~~~~ ^  ~~~~
../../src/main.cpp:38:2: note: in instantiation of function template
        specialization 'print_all_direct<int, int, int, int, int, int,
        int, int, int, unprintable>' requested here
        print_all_direct(std::cout, 1,2,3,4,5,6,7,8,9, unprintable{});
        ^

(Note: These errors were generated by clang and linewrapped by hand from me)

This should be enough for today, but I would like to point out, that there is still a lot more that parameter-packs can be used for. If there is sufficient interesst, I might write one or two follow-ups that will take a look into topics like what you can do with std::index_sequence, especially since we also have std::integral_constant.

Update: As was pointed out on reddit, another way to protect agains weirdly overloaded coma-operators is to cast the expression to void before putting it on the left side, like this:

template<typename Fun, typename...Ts>
void sequential_foreach(Fun f, const Ts&... args) {
	(void) std::initializer_list<int>{
		((void) f(args), 0)...
	};
}

The way this works is that it just discards the result before it can deal any damage. In principle a normal function-call could work here as well, but it will break down in the case of the called function returning void. (I did in fact look into that when I wrote the article and didn’t include that approach because of that; and for some reason I didn’t think of void-casts at 3:30 AM).

The resulting compiler-error for the above problem would also look like this:

../../src/main.cpp:19:49: error: invalid operands to binary expression
        ('std::ostream' (aka 'basic_ostream<char>') and 'const unprintable')
        sequential_foreach([&](const auto& arg){stream << arg;}, args...);
                                                ~~~~~~ ^  ~~~
../../src/main.cpp:12:11: note: in instantiation of function template
        specialization 'print_all_new(std::ostream &, const int &, const int &,
        const int &, const int &, const int &, const int &, const int &,
        const int &, const int &, const unprintable &
        )::(anonymous class)::operator()<unprintable>' requested here
                ((void) f(args), 0)...
                        ^
../../src/main.cpp:19:2: note: in instantiation of function template
        specialization 'sequential_foreach<(lambda at ../../src/main.cpp:19:21),
        int, int, int, int, int, int, int, int, int, unprintable>' requested here
        sequential_foreach([&](const auto& arg){stream << arg;}, args...);
        ^
../../src/main.cpp:25:2: note: in instantiation of function template
        specialization 'print_all_new<int, int, int, int, int, int,
        int, int, int, unprintable>' requested here
        print_all_new(std::cout, 1,2,3,4,5,6,7,8,9, unprintable{});
        ^

Which is somewhat better than the one of the version with an additional lambda.