Magento 2 – How to Create Multiselect with Source Model in UI Component

Since Magento 2.1 you have to define a UI component for custom category attributes (See Magento 2.1: how to create category attrbute programmatically?)

I have one multiselect attribute with a custom source model and cannot get it to work. I took available_sort_by as an example and managed to show the attribute with a multiselect input but it has no options, i.e. it is not using my source model.

This is the relevant part of category_form.xml:

    <field name="solr_remove_filters">
        <argument name="data" xsi:type="array">
            <item name="config" xsi:type="array">
                <item name="sortOrder" xsi:type="number">20</item>
                <item name="additionalClasses" xsi:type="string">admin__field-default</item>
                <item name="formElement" xsi:type="string">multiselect</item>
                <item name="source" xsi:type="string">category</item>
                <item name="label" xsi:type="string" translate="true">Remove Filters</item>
                <item name="tooltip" xsi:type="array">
                    <item name="link" xsi:type="string">#</item>
                    <item name="description" xsi:type="string" translate="true">Hold the CTRL key to select multiple filters</item>
                </item>
                <item name="scopeLabel" xsi:type="string">[STORE VIEW]</item>
            </item>
        </argument>
    </field>

The attribute solr_remove_filters has been added with the following install script:

    $eavSetup->addAttribute(
        Category::ENTITY,
        'solr_remove_filters',
        [
            'type'              => 'text',
            'input'             => 'multiselect',
            'source'            => FilterableProductAttributeSource::class,
            'backend'           => FilterableProductAttributeBackend::class,
            'label'             => 'Remove Filters',
            'note'              => 'Hold the CTRL key to select multiple filters',
            'required'          => 0,
            'user_defined'      => 0,
            'group'             => 'Solr',
            'global'            => 0,
            'visible'           => 1,
        ]
    );

I have set a debug breakpoint in FilterableProductAttributeSource::getAllOptions() and see that it is executed and returns an array in the form

[
    ['value' => 'value1', 'label' => 'label1'],
    ['value' => 'value2', 'label' => 'label2'],
    ['value' => 'value3', 'label' => 'label3'],
]

which confirms that the source model is instantiated and works.

What do I need to do to populate the UI component with these values?