IDE Generated Code

Smalltalk - a highly dynamic language - has always had a superb IDE, in which little things like automatic refactoring, SUnit (the grandparent of all *Unit frameworks), "who sends this message?", "who implements this method?" and the like were pioneered. So yes, IDEs can indeed support dynamic languages to a level that, until very recently, far surpassed those of statically typed languages.

S. Lott says that "Dynamic languages cannot have the same kind of code completion that static languages do. It's essentially impossible."

"Essentially impossible" in a theoretical, absolutist sense? Sure. In a practical sense? Not really. There are many ways to infer type for dynamic languages, statically (k-CFA, RoelTyper), at run time (PICs), and other methods, like using a programmer's recent history - "How Program History Can Improve Code Completion".

Certainly the Pharo and Squeak communities are very happy with their code completion tools.

I had a chance to look at your actual code. It looks like you're most of the way there but this "tree of tokens" concept is getting in the way. It turns out you probably don't need it at all (specifically, I'm referring to the stuff in the Expressions/ folder).

For example, in Join, you have a method called GetDeclarationExpression that returns an IExpressionItem that is a tree of tokens. What I'm suggesting is that you can simplify this and instead directly return a String which is the actual SQL generated from the join. In concrete terms, something like the following:

    String IJoinItem.GetDeclarationExpression(CommandOptions options)
    {
        // [ "(" ] <Left> <Combiner> <Right> [ "ON" <Filter> ] [ ")" ]
        StringBuilder expression = new StringBuilder();
        if (WrapInParentheses ?? options.WrapJoinsInParentheses)
        {
            expression.Append("(");
        }
        expression.Append(_leftHand.GetDeclarationExpression(options));
        expression.Append(" ")
        expression.Append(GetJoinNameExpression(options));
        expression.Append(" ")
        expression.Append(_rightHand.GetDeclarationExpression(options));
        expression.Append(" ")
        expression.Append(GetOnExpression(options));
        if (WrapInParentheses ?? options.WrapJoinsInParentheses)
        {
            expression.Append(")");
        }
        return expression.ToString();
    }

Of course, all other methods used during the AST traversal would also need to be modified to return String instead of IExpressionItem as needed. Each method that is called during the tree traversal would be responsible for generating the actual SQL string, which will all eventually get concatenated together.