I can use the IModelDoc2.IGet3rdPartyStorage (in tandem with IRelease3rdPartyStorage) to write custom data into the SolidWorks file.

But how do I remove it?

I could not find any deletion method in the API docs.