Larger textures look sharper, but they also add significantly more memory usage. So for this reason, the best practice is to use only the largest size that you need.
Unless the camera is super close to a material, 3K is usually more than enough for environments like architecture.
But in situations where the camera is close to the subject (like a muddy ground), you may need 6K or higher.