Supplement feeds in Google Merchant Center are great for adding or editing data and allowing you to set up strategies or enhance your Google Shopping setup.
But how do you remove supplement feeds? It does not allow me to do this?

That is because you are still linking the supplement feed to a primary feed.
Step 1
Go to Google Merchant Center > Cog Icon (top right corner) > Data Sources
Now click on your primary feed where the supplemental feed is connected to.
From the tabs select Attribute Rules and click on the feed rule that says All other attributes Default rule
Now click on the bin icon next to the supplemental rule you want to remove.
You can now save as draft.
Now you will be redirected back to the attribute rules, and you can apply the changes and click on update.
After a minute you can go back to the supplemental feed overview, you can now delete your supplement feed.