Alright, let's reply by parts
1) Private addresses
ipv6 has different "scopes" so you can have a local scope and a global scope, ipv6 is smart enough to know who's what and to regulate traffic accordingly so you can have a local non-routable network on ipv6 without any problem at all, actually it comes by default as that
2) Dump ipv4 and run only ipv6
All ipv6 implementations so far are dual stack so you can comfortably run both, and I would definitely recommend you to run both, there's no damage in doing that and ipv4 is not going away for a long time, although ipv6 is very cool completely dropping ipv4 is not something I would do.
3) Short questions
a) No technical disadvantages, on the contrary! Lots of cool stuff, automatic assignation of addresses, anycast, native ipsec, it's quite cool
b) Firewalls should be good, but there's some specific firewall rules that you should pay attention to like allowing local-link scope traffic, allow multicast on ipv6 and disable processing of RH0 packets, also have in mind that icmpv6 is a completely new protocol and ipv6 is a lot more dependant on it than icmp on ipv4 so filtering it is not a good idea
c) As far as I know most of linux services support ipv6 without any problem, dual stack ftw!
Also it's not bad to get yourself familiar with all the ipv6 new specs, have a look at http://en.wikipedia.org/wiki/IPv6 for starters
The first thing that should be mentioned about IPv6 subnetting is that a different mode of thought is called for. In IPv4 you usually think about how many addresses you have available and how you can allocate enough of them to each end user. In IPv6 you usually think about how many /64-subnets you have available and how you can allocate them to end users. You almost never worry about how many IP addresses will be used in a given subnet. Except for some special cases like point to point links, each subnet just simply has far more addresses available than it will ever require, so instead you worry only about allocating subnets, not hosts inside them.
IPv6 subnets are usually /64 because that is required in order for SLAAC (stateless address auto-configuration) to work. Even where SLAAC is not in use, there may be other reasons to use /64. For example, there might be some end user devices out there that just assume /64, or else routing subnets narrower than /64 might be inefficient on some routers because the router implementer has optimized the case of /64 or wider routes in order to save routing table memory.
Why is it recommended to use /127 for point to point links?
For the specific case of point-to-point links, /127 is recommended instead of /64 in order to avoid a vulnerability where packets addressed to any one of the quadrillions of unused addresses on the subnet cause unwanted neighbour solicitation requests and table entries that could drown a router. Such misaddressed packets may be malicious or accidental. But even if you actually configure a point-to-point link as /127, some people advocate assigning a whole /64 anyway just to be consistent.
Why would virtual machines be provisioned with subnets narrower than /64?
I don't know specifically why virtual machines would be provisioned with subnets narrower than /64. Perhaps because a hosting provider assumed that a server was like an end-user and required only a single /64 subnet, not anticipating that the server would actually be a collection of VMs requiring an internal routing topology? It could be done also simply as a matter of making the addressing plan easier to memorize: the host gets PREFIX::/64, then each VM gets PREFIX:0:NNNN::/96 where NNNN is unique to the VM and the VM can allocate PREFIX:0:NNNN:XXXX:YYYY as it pleases.
Can I map directly from IPv4 subnets to IPv6 subnets? For instance, does an IPv4 /24 correspond directly to an IPv6 /56 or /120?
From a low-level perspective of how addressing and routing works, the prefix length has the same meaning in IPv6 and IPv4. On that level, you can make an analogy such as "an IPv4 /16 uses half the bits for the network address and half the bits for the host address, that's like a /64 in IPv6". But this comparison is not really apt. Strong conventions have emerged in IPv6 which make the divisions of network sizes look somewhat more like the old world of classful networks in IPv4. To be sure, IPv6 didn't reintroduce classful addressing in which the most significant few bits of the address force a particular netmask, but what IPv6 does have is certain [de facto/conventional] standard network sizes:
/64: the basic size of a single subnet: LAN, WAN, block of addresses for web virtual hosts, etc... "Normal" subnets are never expected to be any narrower (longer prefix) than /64. No subnets are ever expected to be wider (shorter prefix) than /64 since a /64's worth of host addresses is much more than we can imagine needing./56: a block of 256 basic subnets. Even though current policies permit ISPs to hand out blocks as large as /48 to every end user and still consider their address utilisation well justified, some ISPs may (and already do) choose to allocate a /56 to consumer-grade customers as a compromise between allocation lots of subnets for them and address economy./48: a block of 65536 basic subnets and the recommended size of block that every ISP customer end site should receive./32: the default size of block that most ISPs will receive each time they request more addresses from a regional address registry.
Inside service provider and enterprise networks, many more prefix lengths than these 4 can be seen. When looking at the routing tables of routers inside these networks, IPv4 and IPv6 have much in common including most of the way routing works: routes for longer prefixes override covering routes for shorter prefixes, so it is possible to aggregate (make shorter) and drill down (make longer) routes. Like in IPv4, routes can be aggregated or summarized to larger blocks with shorter prefixes in order to minimize the size of routing tables.
A different question of mapping between IPv4 and IPv6 would be how to harmonize IPv4 and IPv6 assignments on dual-stack machines so that addressing plans can be readily understood. Far that, there are certainly conventions in common use to do this: embed the IPv4 "subnet number" into a portion of the IPv6 prefix, either with BCD (e.g. 10.0.234.0/24 becomes 2001:db8:abcd:234::/64) or binary (10.0.234.0/24 becomes 2001:db8:abcd:ea::/64).
My interfaces have several IPv6 addresses. Must the subnet be the same for all of them?
Absolutely not! IPv6 hosts are expected to be able to be multihomed by having several IP addresses simultaneously that come from different subnets, just like IPv4. If they are autoconfigured with SLAAC then the different subnets might have come from router advertisements from different routers.
Why do I sometimes see a % rather than a / in an IPv6 address and what does it mean?
You would not see one instead of the other. They have different meanings. A slash denotes a prefix (subnet), meaning a block of addresses that all start with the same n bits. An address without a slash is a host address. You may think of such an address as having an implied /128 at the end, meaning all 128 bits are specified.
The percent sign accompanies a link-local address. In IPv6, every interface has a link-local address in addition to any other IP addresses it might have. But the thing is, link-local addresses are always, without exception, in the fe80::/10 block. But if we attempt to talk to a peer using a link local address and the local host has multiple interfaces, how are we to know which interface to use to talk to this peer? Normally the routing table tells us which interface to use for a particular prefix, but here it will tell us than fe80::/10 is reachable via every interface.
The answer is that we must tell it which interface to use using the syntax address%interface. For example, fe80::1234:5678:8765:4321%eth0.
Am I wasting too many subnets? Aren't we just going to run out again?
Nobody knows. Who can tell the future?
But consider this. In IPv6 the number of available subnets is the square of the number of available individual addresses in IPv4. That's really quite a lot. No, I mean really quite a lot!
But still: we are automatically handing out a /32 to any ISP who requests one, we are handing out a /48 to every single ISP customer. Perhaps we're exaggerating and we will squander IPv6 after all. But there is a provision for this: Only one eighth of the IPv6 space has been made available for use so far: 2000::/3. The idea is that if we make a horrible mess of the first eighth and we have to drastically revise the liberal allocation policies, we get to try 7 more times before we're in trouble.
And finally: IPv6 doesn't have to last forever. Perhaps it will have a longer lifetime than IPv4 (an impressive lifetime already and it's not over) but like every technology it will someday stop mattering. We only need to make it until then.
Best Answer
Inside your network, I guess, technically, you can do anything you like. You can assign /120's if you really want to. But you shouldn't.
But is there actually a problem you're trying to solve here? There should be zero issue assigning multiple IPv6 addresses from the same /64 to a single server. In fact you can assign an entire /64 or larger if you want to (and there are reasons why you would want to do this).
Now, that all said and done, it appears that you are mostly concerned with intra-server comunication. Here's the thing though - even if you assign IP addresses from different networks intra-server communication is still going to work. Because a server will only route to a default gateway (and thus pass through a network level firewall) if it has no route to the target - but in your case it has a direct route to the target because the IP addresses are all on the same machine.
Secondly, addressing this network communication is only one small part of the pie. If you're concerned about security in depth, what's to stop an attacker (or a rogue application) from just reading and writing directly to the memory space of the other application? Or the disk?
The most common way to enforce that sort of segregation is via Virtual Machines. Or containers/jails.