Taxpayers lost an estimated $23 billion to stolen tax returns last year, affecting approximately 2. 4 million people, according to a report by the Treasury Inspector General for Tax Administration.