There are several reasons. One: power loss in a wire is I^2 * R. Therefore it is better to transmit power at very high voltage and low current. AC is much more easily boosted to high voltage (no electronics are needed). To boost industrial loads using silicon electronics is not practical.
Another is ease of switching under load. If you turn off a load connected to DC, the arcing at the switch due to wire inductance and load inductance becomes problematic. This forces DC switches to be more robust.
The 60 Hz noise created by transformers is much less than the switching noise that would be created by all the electronics required to buck and boost DC and then convert it to AC at point of load as you propose.
I can't speak to American/ANSI standards, but in Australia we use AS/NZS standard 60076.5-2012 Power Transformers - Ability to withstand short-circuit as a guideline for the absolute minimum impedance of power transformers. Note AS/NZS 60076.5 is equivalent to IEC 60076.5.
Table 1 of that standard gives absolute minimum percent impedances for various transformer sizes. I cannot reproduce the entire table, but the relevant part for you is:
Table 1 - Recognised minimum values of short circuit impedance for
transformers with two separate windings
Short circuit impedance at rated current
Rated Power (kVA) | Minimum short circuit impedance (%)
------------------+------------------------------------
25 - 630 | 4 %
631 - 1,250 | 5 %
1,251 - 2,500 | 6 %
... | ...
Noting that most residential transformers will be in the 200kVA - 2,500 kVA range. (Pole top transformers can be as large as 500 kVA; past that, up to 2,500 kVA, they tend to be pad-mount on the ground.)
Why are these the typical values?
The reason this information is found in the standard about "ability to withstand short circuit", which is an odd place to find it, is because the transformer impedance is important in limiting the current through the transformer under fault conditions.
A minimum impedance limit implies a upper limit on the through-fault current, hence a limit on the maximum energy dissipation and dynamic force under fault conditions.
The maximum energy dissipation and dynamic forces directly influence the design of the transformer. For instance, AS60076.5 mandates that the transformer must be able to withstand two seconds at maximum through-fault current without sustaining damage from heating, so the conductor thicknesses and so forth must be chosen to accomplish this.
At a guess, the exact values found in Table 1 were chosen because it was found (experimentally) that these were the lowest impedances it was possible to specify, while still having a sufficiently reliable and robust transformer.
Can transformers be ordered with "non standard" impedances?
Transformers can be ordered with a different impedance than the minimum set forth in AS 60076.5, which is only a suggestion. It is common to order transformers with a higher impedance, so the fault levels on the LV system are reduced. I have seen 2,500 kVA transformers ordered with impedance of 12%, which is double the minimum standard impedance, for fault limiting purposes.
It is not common to ask for a transformer with less than the standard impedance, as such a transformer will have a very high LV fault level, which is bad for equipment and personnel safety. Additionally, the high fault level will tend to make the transformer self-destruct under fault. As such, transformers with less than minimum impedance would only be ordered if you really knew what you were doing, and you were willing to waive some of the fault-withstand requirements set forth in AS60076.5.
Best Answer
Without any more details than what's in your question, here's what I believe happened: (It might be counterintuitive, so to avoid confusion: When a breaker is closed, current can go through it. When a breaker is open, current can not go through it. Also, when a relay trips, it will eventually open one or several breakers (thus cutting the power)).
The flickering:
For some (unknown) reason, the transformer substation exploded. This might have caused a bunch of different faults that may trip nearby relays. My guess would be a three-phase fault, as such faults often result in the highest currents (dependent on grounding). Normally you would only want the closest relay to trip, thus keeping the rest of the grid intact. However, this time the relay is probably of little use, as the substation is blown to pieces. So, other nearby relays will trip breakers in order to isolate the fault.
The relays will normally try to close the breakers again to get the power back up within a matter of (milli) seconds. (Note that even though a relay may trip immediately, it will take about 100 ms for the breaker to actually break the current.)
This is most likely what caused the initial flickering.
So, what causes the power to go some unknown time later?
Practically all power systems are operated after the N-1 criterion (or in some cases N-2, N-k). "The N-1 criterion expresses the ability of the transmission system to lose a linkage without causing an overload failure elsewhere." [1] It is however impossible for the transmission system operators (TSO) to comply to the N-1 criterion at all times.
Transformers, lines, cables etc. can handle more than what they are rated for. Transformers can often operate at 50% overload for as much as one hour without taking any damage. Transmission lines can actually be loaded as much as you want. However, as you don't want to risk damage to the equipment, relays are designed to cut the power if the overcurrent lasts too long or gets too high.
The figure above shows a typical relay trip characteristic on a log-log scale. You can find the trip time of the breaker if you know the current. You do that by finding the current of the x-axis, go up and see which value the green curve corresponds to on the y-axis. On the far right the current is very high, 10-1000 x In, where In is the nominal current of the equipment. The horizontal line of the far right is typically at approx 0-100 ms.
The dashed line to the left shows the lowest pick-up value for the relay. This line if typically at 1.2 x In. Since the trip curve is vertical here, any current less than 1.2 x In will never cause a trip.
Between 1.2 and 10 times In, the trip time varies according to the curve shown between the two dashed lines. The rightmost part of the inverse curve is typically at 300 ms, whereas the leftmost part of the curve might be as much as minutes (remember the scale is logarithmic).
Hypothesis:
The failure of the substation causes an overload of (at least one of) the remaining substations feeding Manhattan with power. In this case, the current has probably been slightly above 1.2 x In for one component, thus causing a trip, but with a large time delay. When the first relay trips, another connection will be even more overloaded thus causing another trip, and another one, and another one, eventually cutting all the power to the city.