This might fall more under disaster recovery but an issue I've found is that if one of the servers goes offline (even if it has no sites but has roles) you can't remove it from the control panel.
I feel like if a server is offline there should be some way to remove it even if sites "rely" on it.