In computer science, tail recursion (or tail-end recursion) is a special case of recursion in which the last operation of the function, the tail call, is a recursive call. Such recursions can be easily transformed to iterations. Replacing recursion with iteration, manually or automatically, can drastically decrease the amount of stack space used and improve efficiency. This technique is commonly used with functional programming languages, where the declarative approach and explicit handling of state promote the use of recursive functions that would otherwise rapidly fill the call stack.

So, I decided to see if I could implement this in C#, and after a little bit of monkeying around I was able to get something working.

First off, I needed two helper classes, the first of which is the Ref<T> class. It makes it possible to keep track of a reference to an object across closures. There is also an implicit conversion back to the type it is referencing for convenience.

Next was the LazyRef<T>, which is the same as Ref<T> but won’t actually compute the value until it is needed.

Alright, now all that remains is the method that does all of the magic, I’ll present it piecewise then all together at the end. First, let’s look at the method’s signature,

It will return a delegate that represents the tail call optimized method, taking a T as the only parameter and returning an R. The only argument is the target method/delegate to be transformed. Note that the target takes a delegate as it’s first argument, this will be the recurse function that the method will call instead of itself when it wishes to proceed. It then takes a T, and returns a LazyRef to an R.

Now for the first part of the method, just some Ref and LazyRef variables, I’ll explain these as they are used.

Next up is setRef, it’s the function that will be the recurse function that is passed in as the first argument when target is called.

setRef first flips the calledRef flag, so that we know that target wants to do one more recursion, then we store off the value that target wanted to pass on, and we return the lazyResult. lazyResult will return currResult’s value when asked what it’s value is.

Next, is the actual function that is returned,

First off, it sets up the current parameter value as t, since it needs to be set for the first iteration. Then it enters the loop and flips the calledRef flag to false, so if target doesn’t call the recurse function the loop will not continue. After that target is called, passing in setRef as the recurse function, and the current parameter value. It’s return is stored as the currentResult which will be lazyResult until target doesn’t call the recurse function, at which point it will return something else. This goes on until target doesn’t call the recurse function and the loop exits. After that we return the value or lazyResult, which is the value or currentResult. Ok, so here is everything all together:

All of the refs end up in closures in either setRef or the return value, so the values they reference can be shared and changed. Having the current result be lazy in itself makes the whole thing possible. If we didn’t have LazyRef target would be expecting a computed value to be returned from recurse which would require actual recursion.

Time for a quick example, this will run until the x overflows:

Feel free to post questions and comments.

dynamic x = someInstance.Expose();

You may then access any field, property, or method regardless of it’s visibility level. Feel free to comment with questions if anything needs explanation .

static void Main(string[] args)
{
    dynamic foo = "bar";
    int meep = foo.Length;
}

Nice and simple, it shouldn’t look too bad… Well, the compiler generates a static container class that it uses to keep track of some things, and it names everything using names that can’t possibly collide with the programmer’s code. So, instead of showing you code that could make your eyes bleed, I took the liberty of renaming everything and inserting new lines to help readability, the result was:

static class CallSiteContainer
{
    public static CallSite<Func<CallSite, object, int>> convertSite { get; set; }
    public static CallSite<Func<CallSite, object, object>> getLengthSite { get; set; }
}

private static void Main(string[] args)
{
    object foo = "bar";

    if (CallSiteContainer.convertSite == null)
        CallSiteContainer.convertSite = CallSite<Func<CallSite, object, int>>
            .Create(new CSharpConvertBinder(typeof(int), CSharpConversionKind.ImplicitConversion, false));

    if (CallSiteContainer.getLengthSite == null)
        CallSiteContainer.getLengthSite = CallSite<Func<CallSite, object, object>>
            .Create(new CSharpGetMemberBinder("Length", typeof(Program),
                new CSharpArgumentInfo[] {
                    new CSharpArgumentInfo(CSharpArgumentInfoFlags.None, null)
                }));

    int meep = CallSiteContainer.convertSite.Target.Invoke(
        CallSiteContainer.convertSite,
        CallSiteContainer.getLengthSite.Target.Invoke(
            CallSiteContainer.getLengthSite, foo));
}

So, the first thing to notice is that the compiler is generating a static inner container class that holds the call sites for the operation so that it doesn’t have to keep creating them over and over. A call site is a point where we interact with a dynamic object, we have two here, the get on .Length and the conversion of the result of that get to an integer. Also notice that foo is actually just an object when things are all said and done, the compiler just wraps all interactions with it.

The generic parameter on the CallSite class ends up determining the signature of the call site’s Invoke method. Since the CallSiteContainer.convertSite ends up being the equivalent of the folling lambda, it takes Func<CallSite, object, int> to say that it takes a CallSite, and the object to convert as parameters, then returns an integer.

Func<object, int> convertSite = obj => (int)obj;

It just takes whatever object and tries to convert it to an int. The other call site is a bit trickier, because you cannot express it as a lambda at compile time, but you could generate one at runtime, look here for details on that. What we do know is that the CallSiteContainer.getLengthSite fetches the value of the Length property or field off of whatever object it is handed. It’s not yet clear to me why the CSharpGetMemberBinder takes the Program type and that CSharpArgumentInfo array, but it probably has something to do with how the DLR caches the code it generates. This whole thing is powered by the DLR, which does all the code generation and caching transparently on the back end. So be thankful for IronPython being around to kick all of this stuff off .

When we want to actually perform our operation, getting the Length of foo, and assigning it to meep, it just Invokes both of the call sites, passing the output of the getLengthSite to the convertSite, then taking the result of that and giving it to meep. I don’t know why each call site is passed itself as the first parameter, but after that comes what would be passed into the lambda equivalent of each call site. When the call site is Invoked, it is passed in so it can be updated to handle new cases that it hasn’t seen before, then all of the arguments that would have been passed into the lambda equivalent follow.

Well, that’s about it, all of this generated code isn’t too complex, it’s just pretty wordy. Looking at this and DynamicObject, it occurs to me that implementing Ruby style Mixins, methodMissing, and send shouldn’t be too hard. If you feel like I left anything out, please point it out in the comments.

EDIT: Incorporated information that Curt provided in the comments.

<A>
  <B X="1"/>
  <B X="2"/>
</A>

We would do the following to get the value of both the X attributes:

var root = XElement.Load("Sample.xml");

foreach (var b in root.Elements("B"))
    Console.WriteLine(b.Attribute("X").Value);

That’s not exactly the cleanest code ever, so what if we could do this instead:

var root = XElement.Load("Sample.xml");

dynamic xml = new DXElement(root);

foreach (var b in xml.B)
    Console.WriteLine(b.X);

Using the combination of C# 4.0’s new dynamic type, and the DynamicObject class, getting this working is actually pretty easy. What we need is a dynamic that will map properties to child elements and attributes. The DynamicObject base class provides an overloadable TryGetMember method that is called whenever the code tries to get a property off of a dynamic. For example, when xml.B is used above, DynamicObject.TryGetMember is called, asking the DynamicObject to return the value of the member. So the DXElement gets to decide what type and value to return, at runtime. Here is my DXElement class, in its entirety:

public class DXElement : DynamicObject
{
    public XElement BaseElement { get; set; }

    public DXElement(XElement baseElement)
    {
        this.BaseElement = baseElement;
    }

    public override bool TryGetMember(GetMemberBinder binder, out object result)
    {
        //Go ahead and try to fetch all of the elements matching the member name, and wrap them
        var elements = BaseElement.Elements(binder.Name).Select(element => new DXElement(element));

        //Get the count now, so we don't have to call it twice
        int count = elements.Count();
        if (count > 0)
        {
            if (count > 1)
                result = elements; //We have more than one matching element, so let's return the collection
            else
                result = elements.FirstOrDefault(); //There is only one matching element, so let's just return it

            return true; //return true cuz we matched
        }
        else
        {
            //Ok, so no elements matched, so lets try attributes
            var attributes = BaseElement.Attributes(binder.Name).Select(attr => attr.Value);
            count = attributes.Count();

            if (count > 0)
            {
                if (count > 1)
                    result = attributes; //more than one attribute matched, lets return the collection
                else
                    result = attributes.FirstOrDefault(); //only one attribute matched, lets just return it

                return true; // return true cuz we matched
            }
        }

        //no matches, let's let the base class handle this
        return base.TryGetMember(binder, out result);
    }
}

Not too bad right? Now, it makes a couple of assumptions about the input XML, which are most likely false, but for examples sake, we’ll just ignore that fact . As you can see, there isn’t much code needed, just the TryGetMemeber overload using XLinq against the BaseElement to find what we want to return as the result.

Let’s first take a look at the System.Linq.Queryable.Where extension method, here is the method’s signature:

public static IQueryable<TSource> Where<TSource>(
    this IQueryable<TSource> source,
    Expression<Func<TSource, bool>> predicate);

Notice that it takes a lambda expression, which we can generate dynamically. For examples sake, let’s make a Foo and a Bar class, and two methods that generate a list of each:

static IEnumerable<Foo> MakeFoos()
{
    for (int i = 0; i < 10; i++)
        yield return new Foo { A = "Foo " + i, B = i };
}

static IEnumerable<Bar> MakeBars()
{
    for (int i = 0; i < 10; i++)
        yield return new Bar { A = "Bar " + i, B = i };
}

class Foo
{
    public string A { get; set; }
    public int B { get; set; }
}

class Bar
{
    public string A { get; set; }
    public int B { get; set; }
}

So what if we want to get all of the items in a collection with a and even B? What we need is an extension method that applies our filter to an IQueryable<> of any type. First off, let’s look at the method signature:

public static IQueryable<T> FilterB<T>(this IQueryable<T> query)

It’s fairly simple, it is an extension method that takes an IQueryable<T> and returns an IQueryable<T> so that we can keep our fluent interface going. The first thing we have to do in our method is create a parameter and property expression,

var param = Expression.Parameter(typeof(T), "x");
var prop = Expression.Property(param, "B");

Then we need a binary expression that %’s our property by 2,

var two = Expression.Constant(2);
var mod = Expression.MakeBinary(ExpressionType.Modulo, prop, two);

The ConstantExpressions are pretty self explanatory, they just hold the value that they are given. BinaryExpressions can represent any kind of binary expression, which is any expression that has a left and right side, like a == b, a != b, a + b, and b – a for example. Next, we need to make another binary expression that checks to see if our first binary expression, mod, is equal to zero:

var zero = Expression.Constant(0) ;
var eqZero = Expression.MakeBinary(ExpressionType.Equal, mod, zero);

Notice that we used one binary expression as a parameter for another, MakeBinary’s second and third parameters are just Expressions which can be any type of expression. All that is left to do is create the lambda expression, where it and return it:

var lambda = Expression.Lambda<Func<T, bool>>(eqZero, param);
return query.Where(lambda);

We don’t have to compile the lambda here because query providers use it in its raw expression tree form which they translate into whatever they need to get the job done. Here is the entire method:

public static IQueryable<T> FilterB<T>(this IQueryable<T> query)
{
    var param = Expression.Parameter(typeof(T), "x");
    var prop = Expression.Property(param, "B");

    var two = Expression.Constant(2);
    var mod = Expression.MakeBinary(ExpressionType.Modulo, prop, two);

    var zero = Expression.Constant(0);
    var eqZero = Expression.MakeBinary(ExpressionType.Equal, mod, zero);

    var lambda = Expression.Lambda<Func<T, bool>>(eqZero, param);
    return query.Where(lambda);
}

This should work with any query provider including Linq to SQL and the Entity Framework. We haven’t built in any error checking so expect it to throw if you use it on a collection of a type that does not have a B property. So, given the types we defined above, let’s see how we would use it:

var foos = MakeFoos().AsQueryable();
var bars = MakeBars().AsQueryable();

foreach (var foo in foos.FilterB())
    Console.WriteLine(foo.A);

foreach (var bar in bars.FilterB())
    Console.WriteLine(bar.A);

We are first getting collections of both Foos and Bars, and then changing them into IQueryable collections. we then just iterate over each collection after calling FilterB() on them. We could chain other calls after the FilterB call if we wanted. The output will be:

Foo 0

Foo 2

Foo 4

Foo 6

Foo 8

Bar 0

Bar 2

Bar 4

Bar 6

Bar 8

Yay! It works!

First off, let’s take a look at a simple lambda, and then try to build it dynamically. Say we want to grab the length of a string:

Func<string, int> getStrLen = x => x.Length;

The lambda expression that we are assigning to the delegate has 2 main parts, the x parameter, and the x.Length property access. Let’s go ahead and create these two smaller expressions. Expressions use a factory pattern and can only be instantiated through static methods of off Expression so here is the code to build those two parts:

var param = Expression.Parameter(typeof(string), "x");
var prop = Expression.Property(param, "Length");

Expression.Parameter take a Type as the first parameter, which is the type of the parameter, and a string as the second parameter which is only used when you .ToString() the expression. Expression.Property takes another Expression as the first parameter, in our case we are using the ParameterExpression we created on the line before, but it can be a few other things like other PropertyExpressions, ConstantExpressions, CallExpressions, etc. The second parameter of Expression.Property is a string, it is just the name of the property you want to get. All that is left to do now is to build the lambda expression itself, and compile it:

var lambda = Expression.Lambda<Func<string, int>>(prop, param);
Func<string, int> result = lambda.Compile();

Expression.Lambda has both regular and generic versions, we want to use the generic because we want to be able to call it as directly as possible. The first parameter it takes is the body of the lambda, and in our case that is x.Length, which is our prop variable. Every parameter after the first should be ParameterExpressions representing parameters that you are passing to the lambda, and should match the order and number of the generic parameters on the delegate type that is used as the generic for Expression.Lambda. The call to lambda.Compile() is where the real magic happens, the library compiles our expression tree down to executable code. So, we’ve dynamically built the same lambda as we have assigned to getStrLen. if you are looking for more proof, check lambda.ToString(), which also comes in handy when you are trying to debug your expression building code. Alright, now let’s build a nice generic method using the same pieces as above, starting off with the method signature:

Func<T,R> BuildGetProperty<T, R>(string propName)

Our method will have two generic parameters, T which is the type containing the property, and R which is the type of the property, and one string that will be the name of the property to get. The body of our method is going to be almost verbatim of what is outlined above, except that we are going to substitute our generic types and property name. The end result is:

public static Func<T,R> BuildGetProperty<T, R>(string propName)
{
    var param = Expression.Parameter(typeof(T), "x");
    var prop = Expression.Property(param, propName);

    var lambda = Expression.Lambda<Func<T, R>>(prop, param);
    return lambda.Compile();
}

So, we could replace our getStrLen lambda above with:

var getStrLen = BuildGetProperty<string, int>("Length");

And to use it:

var len = getStrLen("woot");

Which would yield 4 as the result. I’ll be covering more on Expressions in further posts, so stay tuned. Also, comments are more than welcome since this is my first post.