We had a similar problem (without the technical details, of course) in the company I work about two years ago.
You just need to do it one step at a time. Don't try to adopt the agile software development in a rush. There's a lot of stuff to learn and apply.
Don't let the lack of expertise to bring you down either.
Build slowly (but as fast as you can :P), steadily and surely.
I would recommend the next steps (to do this, you might switch from management to development for a while, but that should be fine)
- Learn a good version control system, and learn it well. Personally I would recommend git or mercurial. There is a lot of documentation on both.
- Build a solid core on practices and patterns. Read books, read blogs, watch screencasts with the team members. This will give a new air to the development.
- Learn TDD/BDD and try to apply it in the new code, as well as in the old code that you might touch when doing a new feature.
- Do pair programming. Two heads think better than one, and also 4 eyes are better than 2 :).
- Find about the latest and most common used tools in the community of the language you are currently developing. Learn about them and try to include some of them in the project. See how these were built and learn.
- Use scrum. Iterations, stories, story points, impediments are all concepts you should get familiar with. For me, scrum has proven to be the best workflow for software development and management. Apply it and learn from each day experience.
- Teach by example. Most of beginner developers are eager to learn new stuff, but also some of them are very lazy. Anyways, show them the new stuff you've been learning and applying and hopefully that will tickle their brains.
Also, if possible, hire a consultant just so that he can check out the process and give better advice.
Don't get lazy or discouraged. Just learn from your mistakes and try different approaches. This is just the beginning!
Edit:
Here are some of the links and books that I've been reading/using lately...
Learning git: Pro Git
These are some of the blogs that I would recommend (most of them are .NET oriented):
For books, you can see Buiding A Solid Programming Core list on amazon.
I would also recommend these:
When I have to return to a section of code I've not worked on for a
while, I have difficulty remembering how it worked. I spend a lot of
time reviewing the header files for the relevant classes and reading
the comments I placed along the way in the source files.
Imagine how the poor guy who comes after you is going to feel -- he doesn't even have the benefit of having once known how your code works. Instead of trying to decipher your code, you should be reviewing the documentation that you wrote for the module in question. That documentation should offer a reasonably accurate view of what the module does any why. It's not "the module starts by initializing three arrays using a triple for loop...", but instead: "this module retrieves the data collected by the Fabulotron's main sensor, rearranges it into standard Neopolitan (chocolate, vanilla, strawberry) format, and delivers it to the Analysis module."
In a perfect world, you'd have a design document that sets out the various modules in the system and describes their respective responsibilities, and each of your modules could just refer back to that document to explain what they do: "This module provides the Fabulotron data collection service as detailed in section 4.8 of the design document: http://fabulotron.org/design/section4-8.html." If you don't have something like that, start writing down an overview of each module as you work on it. You don't need to write a book -- a few paragraphs are often enough to get you oriented.
When I introduce changes, sometimes I realize half-way that what I'm
trying to do will break things somewhere else (or worse, it shows up
only at runtime as a surprise). I revert and start doing it
differently, only to find out I neglected the influence on some other
component.
That might be an indication that your modules are too interconnected. The more independent you can make your modules/classes/units, the less likely it will be that you'll run into this kind of problem. Try to make the interfaces between modules as explicit as possible, and try to limit them to just what needs to be there. An interface is a contract -- if you know that some module lives up to its obligations as specified in the interface, you don't need to know anything else about it. That prevents changes in the module you're working on from affecting other modules.
I wish there was some "architecture diagram" where I could see how things get done
Using standard parts can help in this respect. C++ provides standard parts in the form of the Standard Template Library, and using those parts where appropriate lets you work at a higher level of abstraction. If you've written your own code to manage data structures like lists, you and those that follow you will constantly have to read the source to figure out what's going on. If you instead use standard parts provided by the STL, any C++ programmer will quickly be able to tell what your code is doing without digging into your data management routines.
Another kind of standard part comes from design patterns. They're standard concepts that can be used as a shorthand to explain how the relationship between two objects works.
Best Answer
Getting the original investment
According to my experience, the most difficult part is to get the original investment, because investors may be reluctant to give you money with no commitment from your side. What if your research leads to a dead end? What if it requires much more money than they can give you? Ordinary business projects, even Agile ones, are somehow more certain, for two reasons:
While the customer can't know in advance if it will cost $40 000 or $50 000, he probably knows that it will not cost $800 000 nor $1 000. In research-oriented projects, we often have no idea whatsoever how much would it cost.
In business projects, if a twelve-months project is canceled after four months, there should be a “working something” which may not meet the needs of the customer, but which can still be deployed in production and bring some money in.
In research-oriented projects, if after several months or years the project is canceled, there often will be nothing the investors will be able to use to get part of their money back.
This is very similar to Waterfall projects: unless the project is completed, time and money is simply wasted.
Note that this is a generalization. Some R&D projects can be very specific and predictable. Another example: I'm currently working on a project which is unrelated to research, but which didn't have a precise scope at the beginning, but rather a “we'll do whatever appears to be actually valuable and will release it whenever we'll finish it” spirit. Totally acceptable for me—I'm acting as both the investor and the implementer here, but not doable when you need to attract external investors.
Tracking project progress
Once you've convinced the investors to give you money, the next thing is to track project progress in order to manage risks and give precise feedback to your investors.
Here, R&D project is not much different from business projects with no up-front requirements, so any Agile methodology should work. With short iterations, you can track your progress while not knowing exactly your final destination. The difference compared to ordinary non-research projects is that the uncertainty doesn't come from customers who are not sure what they needs are, but the research aspect of the project.
You may also want to read about evolutionary prototyping (including the comments). The difference between evolutionary prototyping and ordinary Agile is that you don't have to deliver a fully working product at each iteration. Instead, every prototype you deliver can also contain some more or less broken features.
This makes it more attractive for R&D projects where you can be stuck for long periods of time on a particular aspect of your product, and not being able to deliver anything fully functional during several iterations. Given the uncertainty of the project, you'd rather show some progress, even inconclusive, than remain silent for several months.