You can permanently delete a store using the Delete This Store option in the Store menu.

How to delete a store

1. After selecting a store, go to Store > Delete This Store; a dialog box will appear.
2. In the Delete This Store dialog box, verify the name of the store you are deleting.
3. Click [Delete] to confirm the action and delete the store or click [Cancel] if you change your mind and want to keep the store intact.