Once you have mustered out, you are no longer in the service (except the Scouts). Its a one-time deal when leaving active duty.
If you are doing a campaign where the characters are still in the service, they do not muster out until it is time for them to leave. Anything they need (weapons, transport, etc.) is going to be provided by the service, and will be expected by that service to be returned in good order when the mission is complete. Keeping a particular weapon might be feasable if it is of use for the serviceperson in their continued duty assignments.
If they have mustered out, they may recieve that weapon as a token of appreciation for thier service. Note that once they have left the service, they can no longer gain levels in their service class (again, except Scouts), which means they will need to multiclass into another class (such as mercinary if they wish to still be a soldier of some sort).