Use apt-get everywhere and cleanup post install

The current docker build uses a mix of apt and apt-get.

From online resource, the best practice seems to be using apt-get everywhere (see for instance https://protsenko.dev/infrastructure-security/avoid-using-apt/).
Also, intermediate artifacts are currently not cleanup after each install, making the size of intermediate layers larger than it could be.

Note: while this is a best practice, this is not actually impacting the overall image size (3.6 GB at the time of writing).