So the upgrade is now complete and we are on v12 š„³ - it did go smoothly except for the 1 server that hung after the "v12-upgrade upgrade-app-server" command.
I believe that hang was down to the upgrade script waiting for all of the running PHP processes to stop but there was 1 PHP process that would not quit, we even tried force killing it (kill -9 xxxx) when we noticed it was still there. Rebooting the server (had to use ctrl+alt+del as normal shutdown was also waiting for that process to stop) and rerunning the command worked but we had to disable that PHP process as it poppped back up.
We know exactly what the PHP process was as its called from a cronjob by Magento and it's always running. We disabled the cronjob, killed it and then reran the upgrade command.
After that, the upgrade went fine but our backups servers (they are dedicated backup servers) still had Docker with Apache, filerd and bkupd running on them. We tried shutting down docker but the enhance controld started them back up again. We didn't originally run the "v12-upgrade upgrade-app-server" command on the backup servers because it wasn't in the instructions, these servers do not have the Application role or email role, they only have the Backup role.
Anyway, we ran that command on the backup servers as well and that appears to have sorted it.
It's now been nearly 2 hours since the phone lines opened up and we have not had a single call, or email, about any issues so we are very happy š
