Functionality is missing from UI Toolkit ListView, but planned for the 23.2 release. In the mean time, users could potentially add code directly in a callback on selectionChanged by querying the "-" button (look for unity-list-view__remove-button) and disable it when tag is not selected.