All persons within the groups enumerated in Code Section 43-12-1 seeking a certificate of exemption from the payment of occupation taxes, administrative fees, or regulatory fees for peddling, conducting a business, or practicing a profession or semiprofession must first make application to the judge of the probate court of the county in which he or she resides for a certificate of eligibility. Each applicant shall make an affidavit before the judge of the probate court that he or she is not subject to payment of any income taxes to this state. Upon receipt of the evidence required in Code Section 43-12-2 and the execution of the affidavit required by this Code section, the judge of the probate court shall issue a certificate of eligibility stating that the applicant has furnished the proof required for the issuance of a certificate of exemption required by the commissioner of veterans service.