Deleting referenced layers

How can the layers already being used by other entities, be deleted?