Better support the manipulation of private keys when a cluster node is down

Private keys are stored on disk. This means that it needs to be duplicated on each cluster member. It is the case, but this only works when all nodes are up and running. This makes initial setup a bit fragile and also makes it hard to add another node to a cluster later.