The public folder replicas and the default public folder database for a mailbox server are two different things.
The public folder tree is an organization-level entity, unrelated to any physical database; each folder can have one or more replicas, residing on various mailbox servers.
When a user needs to actually access a PF, Outlook connects to the nearest-available public folder database which contains a replica of that folder; this is an automated process, and doesn't need any user or admin intervention. So, if each and every folder is replicated to both of your mailbox servers, then your public folder databases are already higly available: should one of the servers fail, your clients would automatically connect to the other one.
The default public folder database is a configuration setting of every mailbox database; it specifies to which PF database users whose mailboxes reside on that mailbox database will first try to connect in order to access public folders, and where new folders will be physically created when they are created by one of those users. This actually doesn't have any importance at all if all of your folders are replicated to all of your servers: as I said above, failover will be handled automatically, and folders that get created on one server will be very quickly replicated to the other one.
If your public folders databases have been created and mounted and everything is replicated to both of them, you shouldn't have to worry at all about their availability; PF replication has been around for quite a while, and has been very thoroughly tested. So much, actually, that there was no need to include PF databases into DAGs exactly because they already have built-in redundancy.
Best Answer
After opening a support ticket with Veeam, we've discovered that the System Public Folders are stored in the Public Folder Database, but for some reason their software doesn't back it up. There is a size discrepancy between the PFDB in my environment and the PFDB I can restore from after a backup. I'm missing 130MB in the backed up version.
We discovered that the path of the System Public Folders root can be found in EMC by going to the Public Folder Management Console and left-click System Public Folders, then EFORMS REGISTRY. In the right pane, right-click Organization Forms and go to properties. It shows on the General tab the location of the object (Public Folder Database:).
So I know it's in the database, but Veeam doesn't process it during backups. I made a manual backup of my PFDB and used StorageCraft GRE to open the database and the System Public Folders were there.