Creating your own libraries is quite simple. Their register spec documentation is pretty good, most if not all of the peripherals are easy to setup. I find it much more painful to use their libraries. but perhaps that is just me. This is true for st, nxp, ti, atmel to name a few (not so much for intel and microchip).
Why do they change libraries, could be any number of reasons, some new boss took over, some division got shut down another took over. Marketing wanted a new image for the product. As ElectronS mentioned, could be an attempt to abstract away from the hardware more to attract users not willing or able to do bare metal. I would go further on that and say they are probably trying to compete with the Arduino phenomenon. Which mbed and everyone else has always tried to do and failed (even before Arduino).
In any case the further away from the hardware it gets the more bloated and slower, so the more you have to spend per unit for rom, ram and mhz. Just so you could spend the same amount of time programming? Just doing it differently?
You say you come from the PIC world, now they did do an okay job with tools, their chip docs were dreadful though, some of the worst. they compensated with libraries and sandboxes.
At the end of the day, try the various options, try the competing products to see how their tools compare. A lot of that you can do for free just to see if it makes sense and you can compile stuff. Maybe even use an instruction set simulator. Find the one that matches for you.
Note, the without canned libraries option is ALWAYS available to you. You are not limited as to what toolchain you can use, what host operating system, what ide, editor, etc. They might stick it to you on programming the parts, if their options are extremely limited in that respect move on to some other chip or vendor if you can.
To sell a chip product like this they have to provide a development environment whether it is all theirs or free stuff they glued together. And they tend to put a library of some kind together. It only has to look just good enough and the blink the led example work just well enough to get either your management or your hardware team to design in their product, then when your board product is tossed over the wall to software, is when the pain does or doesnt arrive. If it almost works but not quite is a big win for the chip vendor as you will now pay for tech support for that last little bit. So it is in their best interest to be almost there but not quite.
The chip vendors only have to look good enough to get the design win. They have to keep improving (? changing) the product to attract new and old customers. So they will have do overs, how far apart and how many prior libraries the continue to support, varies. So just about any library you get used to will go away eventually. So learn to adapt (or dont use their stuff and go your own, which you can support indefinitely). Granted, ideally, you only need to develop the application once per product, make your firmware perfect (good luck if using third party libraries), and you wont need to go back and find a computer that will load their toolchain if you can find a copy of it, and remember how to use that old library. Remember not only should you save your source code, but you should save all of their tools and docs.
Their libraries are only supported on usually one toolchain, under one maybe two IDEs and sometimes only on Windows, and certain versions. Again you have none of those limitations, most definitely not for ARM, if you do your own thing. You can always read any/all of their libraries to see how they do things. But that is often very scary, they dont use their A team developers for libraries, I have extracted a few lines of code to ask interview candidates what is wrong with this code.
to save time and effort both on the silicon side and the software side they very often recycle the same ip, so once you see how the peripheral works on one of their chips it often works the same way on many other of their chips. Yes the clock systems can be tricky with or without their libraries. High chance of bricking the chip, that is where most of my chip/board bricking has happened. Helps to understand how their chips work for example the AVR's, most if not all, can be reprogrammed while the chip is in reset, so any bad code that messes up the pins needed to re-program, or hangs logic needed to reprogrammed, doesnt matter, you can reprogram those chips. Some of these vendors (st is one) has an internal bootloader which you can select using a strap (BOOT0 for example in the st world), you could easily put your own strap in while developing and should whether or not you use a library.
One size fits all fits no-one well. Particularly true for software. So any attempt to abstract away the hardware, just makes it slow and bloated. Might as well get a bigger chip and run linux on it, if that is what you are truly after. A lot of this is as a result of the developers though, not wanting to get their hands dirty, so we have basically asked for this, and they are trying to supply it.
Again, dont lock yourself into st or any one vendor (unless it is too late and management and or the hardware team has stuck it to you, note the stm32 products are nice and easy to use). Shop around. TI is putting a lot of eggs in the cortex-m4 basket. You can do the mbed thing on a number of these arm products as well as the vendor supported solutions.
One thing you can always rely on, is that they will change libraries from time to time and eventually stop supporting the one you got used to.
Best Answer
Definitely the CMSIS. It is not exactly a library, it mostly contains definitions for the various registers.
It is exactly what one needs to access the microcontroller's registers easy, so as to implement his/her own HAL. It has no overhead, since you just access the registers.
Keep in mind that CMSIS, unlike the other two, is defined by ARM and not ST. This means that the various CMSIS libraries out there for the various microcontrollers are quite similar, which greatly aids in portability.
Furthermore, CMSIS is the simpler one so it is (IMO) the most versatile, and most reliable, with possibly fewer (or no) bugs. Some hal libraries for the various mcu's that I've used are quite infamous for their bugs.
On the other hand, CMSIS needs quite more work from you. It is however my personal choice, since I prefer to invest my time creating quality libraries, that suit my needs, and understanding how the chip works, than just spending time to learn another, new library.