The judge of the probate court shall keep his office and all things belonging thereto at the county site and at the courthouse unless impracticable from any cause, in which case the office may be kept at some other designated place not more than two miles therefrom, of which public notice must be given.