Rebooting even monthly just to fix memory usage? That's problematic if you are serious about uptime.
https://askubuntu.com/questions/184217/why-most-people-recommend-to-reduce-swappiness-to-10-20 the accepted answer explains how linux handles memory management in a very easy to understand way.
Also https://www.linuxatemyram.com/
Basically Linux likes to use available memory for caching but if a process needs said memory, it will release the memory. This can make available memory look MUCH lower than it really is. Also if you disable or cripple swapping, you can end up with high memory usage just because inactive processes are using memory that could be moved to swap without performance impact. Adds up with time.
Mysql could also be happily eating up all memory slowly but surely and not releasing it until its restarted.
docker stats --no-stream
Could shed light on memory usage of the mysql container or if its a webserver container using the memory.
As adam said if you have tuned mysql for max performance it will be very memory hungry. What a busy woocommerce store needs and what a landing page needs in terms of mysql configs is drastically different.