development-process
When creating a new file format for an application, you can write the code or the documentation first.
When writing the documentation first, you have a better idea of what you should implement.
However, you might need to change a lot of the documentation you have written afterwards when writing the code, because you may discover certain things aren't very efficient to implement that way.
So: When creating a new file format, what should you write first: The code or the documentation?
You sound like a Software Engineer.
A Project Manager is more focused on the status of the overall project and allocating resources in an effective way to ensure that milestones are being met and in proper time. They also remove roadblocks and help the project resources focus on their specific job functions.
Writing and maintaining design and technical documentation is an important part of being a software engineer and is appropriate for your role. Too much of a good thing can be bad though (See Analysis Paraylsis ).
He considers the design to be paramount, coding is "just writing the design down"
If the project manager considers the design documents to be paramount then he expects the documents as a deliverable in the process. It is not a wasted time on your part if he feels they are important enough to allocate 80% of your time to it.
it shouldn't take too long, and "all the code should be written before the hardware is ready".
This is wishful thinking and quite an antiquated Waterfall style view of how software development works. It invariably is never that clean no matter how much design and preparation that you devote. On that note however, you should have clear hardware specs and proper test environments and mock hardware simulations for your code to interact with but again, the real world is messy.
Project management is incredibly easy in a perfect world. The world is not perfect however no matter how much you wish it to be so making real project management incredibly difficult. This is why they are paid the big bucks.
(2) Doesn't understand the difference between a Central & Distributed Version control.
Why should he care? How does it affect the milestones? It doesn't.
3) Doesn't understand code, and wants to understand every bug and its proposed solution
His goal is for working software and yours should be too. He does not need to understand the code but he does need to understand the issues that are preventing working software. Once the two of you see eye to eye on this basic level then your mutual goal will help you work together more effectively.
(4) Believes verification should be done by developer, and validation by the Tester.
What is wrong with this sentiment? Testers in the quality assurance role should be concerned with validating features and requirements. Developers should make every effort to verify and test their work before it goes to testing.
I don't really like creating documents, I want to solve problems and write code.
If you would rather be a simple programmer then perhaps you should talk to your boss about this and see if there is a better role for you on the project. Somebody needs to write and maintain the technical documentation, so perhaps one of the other developers can help you with some of these tasks so you aren't spending 80% of your time writing documentation.
In my experience, creating design documents only helps to an extent, its never the solution to better or faster code.
This is mostly true... but only if all ten developers are spending 80% of their time writing technical documentation that nobody will ever read. This is an enormous management anti-pattern that I have lived in before. If you find that you are doing most of the documentation for the team then it hardly seems fair that you are denied the right to perform more coding work.
The fact of the matter is that the best technical documentation is the code itself.
I feel the boss doesn't really care about making better products, but only about being a good manager in the eyes of the management
I feel that he does care because the product is his ward and if projects and features are not completed and clients aren't happy then upper management will not care for him very much. The problem is that your perspective on the necessary quality improvements don't match with his, and upper management, and the clients perception of what they feel is important.
Try to understand what is truly important for the product, because while you can increase the efficiency of a process 3 fold, if you spend an entire week doing this then it is at the cost of another important feature that the client is demanding.
We all want to look good to the eyes of our superior. There is nothing wrong with that, it is human nature to be self serving. This is a fact of life.
You already have a hint: look at how Java API is documented.
More generally, there is no unique set of rules which apply to every project. When I work on business-critical large-scale projects, the documentation has nothing to do with the one I would write for a small open source library, which, in turn, has nothing to do with the documentation of my medium-scale personal project.
Because most open source projects are made by people who contribute to those projects because it's fun. Most programmers and developers consider that writing documentation is not fun enough to be done for free.
Because it costs a huge amount of money to (1) write good documentation and to (2) maintain it.
The immediate cost (cost of writing the documentation) is clearly visible to the stakeholders: if your team asks to spend the next two months documenting the project, it's two additional months of salary to pay.
The long term cost (cost of maintaining the documentation) becomes noticeable pretty easy too to the managers, and is often the first target when they must lower the cost or shorten the delays. This causes an additional problem of outdated documentation which quickly becomes useless, and is extremely expensive to update.
The long term savings (savings from not having to waste a few days exploring the legacy code just to understand a basic thing which should have been documented years ago) are, on the other hand, difficult to measure, which confirms the feeling of some managers that writing and maintaining documentation is a waste of time.
What I often observe is that:
At the beginning, the team is willing to document a lot.
Over time, pressure of deadlines and lack of interest make it more and more difficult to maintain the documentation.
A few months later, a new person who joins the project practically can't use the documentation, because it doesn't correspond at all to the actual system.
Noticing that, management blames developers for not maintaining the documentation; developers ask to spend a few weeks updating it.
If management grants a few weeks for that, the cycle repeats.
If management refuses, based on previous experience, it only increases the bad experience, since the product lacks documentation, but a few months were spent writing and maintaining it.
Documentation should be a continuous process, just like testing. Spend a week simply coding a few thousands of LOC, and getting back to tests and documentation is very, very painful.
Similarly to the ways to encourage people to write clean code, to do regular refactoring, to use design patterns or to add enough unit tests.
Lead by example. If you write good documentation, your pairs might start doing it too.
Do systematic code reviews, including formal code reviews targeted at inspecting the documentation.
If some members of the team are particularly antipathetic to good documentation (or documentation at all), discuss the subject with them privately, to understand what are the impediments which prevent them from writing better documentation. If they blame the lack of time, you see the source of the problems.
Make the presence or the lack of documentation measurable for a few weeks or months, but don't focus on that. For example, you may measure the number of lines of comments per LOC, but don't make it a permanent measure, otherwise, developers will start writing long but meaningless comments just to get rid of low scores.
Use gamification. This comes together with the previous point.
Use positive/negative reinforcement.
(See the comment by SJuan76) Treat the lack of comments as errors. For example, in Visual Studio, you can check an option to generate XML documentation. If you also check that all warnings are treated as errors, the lack of a comment at the top of a class or a method will halt the compilation.
As for the three previous points, this one should be used with caution. I used it for a while with a particularly tough team of beginner programmers, and it ended up with StyleCop-compliant comments like that:
/// <summary>
/// Gets or sets the PrimaryHandling.
/// </summary>
public Workflow PrimaryHandling { get; set; }
which were, hm..., not particularly helpful.
Remember: nothing automated can help you pinpoint bad comments when programmers wants to screw with you. Only code reviews and other human tasks will help.
Documentation explaining the architecture and the design is not needed:
For a prototype,
For a personal project written in a few hours to accomplish a task, while being pretty sure this project won't be maintained any longer,
For any project where it is obvious, given the small size of it, coupled with the particularly clean code, that you will spend more time writing documentation than all the future maintainers exploring the code.
In-code documentation (code comments) is not needed according to some developers when the code is self-documenting. For them, the presence of comments is, except in the rare cases, not a good sign, but a sign that the code wasn't refactored enough to be clear without the need for comments.
I feel that we should have a documentation review after a project is delivered.
If your project is delivered at least once per week, it's the way to go. If your project is not agile and is delivered at the intervals of six months, then do more regular reviews.
Best Answer
You are confusing documentation with specification.
Specification is the process (usually collaborative) through which the definition of what the format should do and how a hypothetical application should handle it, render it, or parse it.
It's a design process like the one W3C does when creating the HTML or CSS specifications way before browsers implement it.
Or course the product of this process can be understood as "documentation", but it's not the documentation of an application.
If you are creating a new file format, discuss and design the specification.
Then code the application that uses that file format ( or have others code it ).
But to create a new file format you don't have to write a single line of code. The specification may never be implemented, yet it exists as an specification.
Obviously when an specification does get implemented, it is revised and extended with the feedback of implementers.