There is no way to do this in the UI but it could be done manually. If you send a support ticket, I can give you some database queries to run to remove the backup server assignment which will allow you to delete it, leaving only the S3 backups in place.
