Writing a new programming language – when and how to bootstrap datastructures

If you want this to maintainable then not really. I've seen a compiler that was literally an overgrown sed script. It worked of course, but then we decided we wanted to add something to the language..

However, if you take the more or less standard route of

1. Lex
2. Parse
3. Compile superset to vanilla C# AST
4. Pretty print AST

you can almost certainly use an existing library for 4, and if you decide to grow your compiler then you'll have a far easier time. If you want to do anything vaguely serious with this compiler than the initial overhead is well worth it.

It might be worth your time to look into some nicer tools for parsing/lexing. I don't think it'd be impossible to find/modify an existing C# grammar to deal with 1 and 2.

There's a number of inter-related questions here, I'll try to separate them as best I can.

Why do other languages build on LLVM IR and not clang AST?

This is simply because clang is a C/C++ front end and the AST it produces is tightly coupled to C/C++. Another language could use it but it would need near identical semantics to some subset of C/C++ which is very limiting. As you point out, parsing to an AST is fairly straightforward so restricting your semantic choices is unlikely to be worth the small saving.

However, if you're writing tooling for C/C++ e.g. static analysers, then re-using the AST makes a lot of sense as it's a lot easier to work with the AST than the raw text iff you're working with C/C++.

Why is LLVM IR the form it is?

LLVM IR was chosen as an appropriate form to write compiler optimisations. As such, it's primary feature is that it's in SSA form. It's quite a low level IR so that it is applicable to a wide range of languages e.g. it doesn't type memory as this varies a lot across languages.

Now, it happens to be the case that writing compiler optimisations is quite a specialist task and is often orthogonal to language feature design. However, having a compiled language run fast is a fairly general requirement. Also, the conversion from LLVM IR to ASM is fairly mechanical and not generally interesting to language designers either.

Therefore, lowering a language to LLVM IR gives a language designer a lot of "free stuff" that is very useful in practice leaving them to concentrate on the language itself.

Would a different IR be useful (OK, not asked but sort of implied)?

Absolutely! ASTs are quite good for certain transformations on the program structure but are very hard to use if you want to transform program flow. An SSA form is generally better. However, LLVM IR is very low level so a lot of the high level structure is lost (on purpose so it's more generally applicable). Having an IR between the AST and the low level IR can be beneficial here. Both Rust and Swift take this approach and have a high level IR between the two.