Complex UI component, using dynamic rows in admin, for a product attribute, is not saving when removing all rows

I created a complex product attribute entry in product admin, which uses dynamic rows.

Same idea as this core one:

enter image description here

UI works fine, but if you remove all rows, and save, the attribute does not update, and reloads back to the rows prior to removed. So, basically, you cannot clear out the rows/entries.

However, this example core one works 100%.

My entry is adjusted using UI Components modifiers, exactly like this core attribute is done.

https://github.com/magento/magento2/blob/2.3-develop/app/code/Magento/Catalog/Ui/DataProvider/Product/Form/Modifier/AdvancedPricing.php#L433

(they used a derived UI component called Magento_Catalog/js/components/dynamic-rows-tier-price in place of Magento_Catalog/js/components/dynamic-rows, but they are essentially the same. Tested as well by making my component use the same, which did not solve the issue (just in case)

So, essentially, we have ability to build these complex input components, but they cannot be cleared.