What You Need To Know About The IRS Crackdown On Tax Evasion

Lauren: Thank you for having me, Toby.

Toby: You get the joyous task of talking about tax evasion and the 89,000 new agents that are set there all armed and ready to take the task, but only if you make over $400,000.

Lauren: That’s what they say. Wink, wink.

Toby: Yeah, we’ll get into all that. I want to hit on a few topics. You’re kind of laying the land for people as to the division of the IRS because there are different portions, there’s civil and there’s criminal, obviously, and where this 89,000 agent comes into play and whether somebody really has to worry.

Then if you’re rich, do you really need to worry? Are they really lining them out in the street and auditing the crap out of them? Or is this more just politicking? You can dive into those at your leisure. Which ones do you want to hit first?

Lauren: Let’s talk about that last point. What’s actually going to happen, what’s real, and what is just the politics, what’s the fluff.

Toby: Tell us.

Lauren: Ultimately, it seems like a partisan issue. It seems like the left is coming in and Biden’s saying, all right, we’re going to crack down on those millionaires that aren’t paying their fair share of taxes. That’s what the media is telling you. That’s what everyone’s freaking out about. But is that really what’s going to happen? I don’t think so.

Just with the workforce that the IRS has currently, I don’t see that happening. Just that their audit rates for the very rich might go up a little bit compared to the almost zero that it is now. It’ll increase a little bit, but all in all, it’s a lot of hot air. It doesn’t really seem like the IRS has the workforce backing up a claim like this.

Toby: Let’s hit on this. What’s happened to the IRS workforce in the last 10 years?

Lauren: One, they’ve been dropping like flies left and right. Two, the IRS has been defunded and understaffed since 2010, so they aren’t getting enough funding. The people that they have that are doing the audits don’t have the right training or really they’re overwhelmed with work. They don’t have enough accountants and lawyers to be doing what they’re supposed to be doing right now.

Toby: Realistically, this shouldn’t be a partisan issue. We should all be saying, this agency that we have that’s tasked with enforcing our laws and making sure people comply fairly with the law. You don’t have one person paying nothing because they’ve decided to thumb their nose to it and somebody else who’s paying, having money taken out of their paycheck. Everybody’s treated fairly. We should all agree that it would be good if that agency was efficient and was able to handle the workload that it has. But it didn’t. It got demonized.

I saw people talking about armed agents and there are 89,000, an army of agents, that you suck on the conservatives. I think that the actual budget numbers show that a lot of that money is going to customer service, which is picking up. It’s getting better with the IRS.

They’re actually answering their phones now. It used to be like one out of every call gets answered. It was just a […]. What are you seeing? You deal with this on a day to day basis. What are you seeing?

Lauren: I’m seeing that overall this idea or the announcement is creating a lot of fear in people, but we need to take a look at what this really means. Yeah, they’re going to hire 80,000-something, but that’s over the next decade, at least.

Toby: It’s not at once. They’re not hiring 89,000 this year.

Lauren: That’s not possible.

Toby: They gave us a schedule. Is that what you are using? It’s over a decade.

Lauren: Yeah, according to the IRS website, when they were going into a bit more detail on these announcements and what this rollout really means. They were saying this is over the next 10 years that we’re looking to hire this many people. If you want to do a little bit of math there, that’s 8000–9000-ish a year maybe.

Toby: It’s almost enough to replace the people that are leaving because I don’t know what the number is. I should Google that and look it up.

Lauren: A lot of what they’re wanting to do is take a lot of the funding to improve taxpayer services. Improving the online features that are available, the response times, actually picking up the phones, and also working on creating more taxpayer assistance centers that are in lower income areas so that they can help people file their taxes. For people that don’t have a ton of education or experience with filing their taxes, that could be a helpful thing.

Toby: I’m looking at some of these. It’s been dropping for a number of years, but you’re going from over 85,000 agents. We dropped down below 75,000 over a period of, it looks like 8 years. You’re losing people pretty quick. I guess this would replenish some of that and maybe give them some more, but I don’t think it’s certainly not like hey, here we go. It’s all going to come on board.

Lauren: Yeah, they don’t have an army of armed CPAs and attorneys raring to go that are going to come knocking on your door. That’s not what’s going to happen. They’re claiming that they’re going to be focusing more than they have been on people that are wealthy, that have either not been paying at all, so not filing at all, or people that are…

Toby: Should, they absolutely should. They actually should be coming after folks that aren’t filing. I know that there are a couple of things. They’re cracking down on people that are playing the Puerto Rican game. There are people that legitimately moved to Puerto Rico and meet Act 60. I think this is where they’re establishing a Puerto Rican qualified entity and then they get tax benefits as a result if they’re doing business.

Lauren: If you’re actually doing business in Puerto Rico, you’re fine. No issues. If you’re doing it fraudulently or you’re claiming to be in Puerto Rico or doing business there, and you’re not, that’s when you have something to worry about.

Toby: Yeah, there are a bunch of folks that they’ve already nabbed. I think they audited a hundred of them this season. Some of them get criminal referrals, so we’ll talk about that in a second.

I’m looking here, the Federal News Network said that 52,000 IRS employees are expected to retire within the next six years. We can just do simple math. You have maybe 80,000 employees in the whole place. You’re losing people.

I believe, and I’m just going off the top of my head, that there were over 115,000 employees at one time. Yes, you can be more efficient, but a lot of the investment needs to be made in their computers. I think they’re still using COBOL from like the 80s as their system. It’s horrible.

Lauren: It’s laughable.

Toby: Let’s get them some new computers maybe. I don’t know.

Lauren: If we’re looking at the number of people that are going to be retiring, it’s 50,000-something and they’re planning on hiring 80,000 over the next decade, they’re barely replacing the staff that’s retiring.

Toby: Definitely the audit. We saw when the staff rate drops, the audit rate drops.

Lauren: Yeah, because there are not enough people to do that.

Toby: What happens is that the audits are done half-cocked. We actually do all of us as Americans, I’m not a big fan of a big brother or beating on people unfairly, but there are folks out there that take unreasonable positions and just flat out lie or just don’t file taxes. Those people should be held to account. But then there’s the other side. Let’s talk about the other side. What is the difference between civil enforcement and criminal enforcement?

Lauren: Civil enforcement is really just you’ll be paying a fine ultimately. If you’re convicted of criminal tax evasion, then you’re looking at some potential prison time and a fine. But again, unless you’re doing some serious Al Capone stuff, that’s not something that everyone needs to be worried about.

Toby: The way I always think about it is just as Rehnquist once said, there’s nothing wrong with a strategy to avoid taxes, which is there’s the tax code. Probably 10% of it is here’s the taxes you have to pay and then the rest of it, 90% of it is how do you avoid it? Here’s the circumstances in which you can avoid it.

It’s like, hey, you have to pay tax on all your money, but here’s a standard deduction and here’s things that you can write off under certain circumstances. You get a state and local tax deduction. You have some medical deduction. Here’s HSAs, IRAs, and 401(k)s. Here’s a whole bunch of things that you can use to reduce it or charitable donations. There are other things built into the code. Avoidance is okay.

Evasion is when you know you owe a tax and then you just flat out lie, like write a deduction that didn’t occur. Where you see people get in trouble is when they don’t think their tax is fair and they take matters into their own hands. Instead of looking at the code for incentives or taking legitimate expenses, they think it’s unfair that they’re paying tax and then they can cop things. They do like Enron where they start creating businesses that don’t exist and start writing things off, or they just flat out say you know what? I’m not going to file it. You can come get me at some point.

Those folks may get the criminal referral. A lot of folks don’t realize that when a civil sees evidence of a crime, they’re going to refer it over to CID and CID is going to take it from there. Now CID, those guys do have guns. That’s criminal enforcement because you said Al Capone. These are the guys dealing with the bad. If you end up dealing with them, don’t move. Get your lawyer.

Lauren: Don’t resist. Get a lawyer.

Toby: Don’t try to tuck your way out of it. What else is going on in that world? What is the IRS communicating to folks? What are you hearing? What are the stress points that you’re hearing from clients especially?

Lauren: The stress point that I’m hearing from clients is I’m afraid to try and do the 280A method, trying to do that because I’m afraid that they’ll say that that wasn’t real and I’ll get a penalty.

All right, that’s a valid concern. But one, are they even going to be looking at someone like you? Let’s look overall at what your income and deductions are, and what all of that looks like. Also just have the documentation to show that you did have a meeting at your home for your business, that you and maybe your spouse are involved in.

Just have your meeting minutes and have invoices. If everything’s running as it’s supposed to, if it’s running like a real business, you have nothing to worry about because even if there was an audit, you have all of your evidence to show.

Toby: What you have is a level of comfort with the aggressiveness that they’re going to display. If they have a business, they don’t understand, hey, I can reimburse myself for certain things for example. A lot of people will just say, I don’t feel comfortable reimbursing my phone even though I’m using it for my business, because I only use it partially for my business and they can’t get over the mental leap that there’s a distinction when you have an accountable plan or something that you can write these things off.

280A is just called the Augusta Exclusion, but if I could pay somebody else to do something, chances are I can pay myself for it so long as it’s arm’s length and it’s of equal or greater value. Getting over that little mental hurdle, but I’m negotiating with myself. I know, that’s why you use fair market value and you go out there and get quotes from somebody else. Yeah, you’re allowed to do this.

The IRS has already established that you can. In fact, the IRS doesn’t even write the laws. They just enforce it and there’s already a law on this. It’s not even up for debate, but somebody will say, it just seems like I’m getting too good of a deal. I get it.

Lauren: Like, all right. We’re wary of things that seem too good to be true, but if you’re just looking at what the tax code allows for and it allows you to do that, do it.

Toby: I just had a client this morning. It was a defined benefit planner where they were able to put the range from the actuary was between $300,000–$400,000 and they were looking to shelter $200,000. They couldn’t believe that that actually existed and you’re saying you’re going to pay tax on it.

See, here’s the thing. A lot of these incentives are there. Somebody is getting a benefit and the IRS or the Congress when they write these laws is the idea that, hey, if we don’t take too much from people, there’ll be more to take later.

In the case of retirement plans, a lot of times, the government collects even more taxes because you’re deferring it and now they’re going to tax the growth and the deferral at some point in the future at your ordinary rate. They’re just willing to say just keep it for now.

Again, there are some funky little things out there that people don’t realize are incentives. Just because you weren’t aware of it doesn’t make it illegal. It just means you weren’t aware of it, and there are a lot of incentives that are rolling around out there that will benefit you.

What else are you seeing and hearing from people? Because every time there’s an article written or people are like hey, the IRS is going to go after people making $400,000 and above. All of a sudden you’re like, I make $400,000 and above. Are they expressing anxiety? Do they feel like there’s a bullseye on their back?

Lauren: A lot of clients have been expressing that they’re worried that they’re going to see an increase in audits, or they’ll get audited multiple years in a row, or something really crazy like that. Just looking at the logistics and at the numbers, the IRS doesn’t have the manpower to even do that and they’re not going to. The audit rates are still going to be proportional, but it’s just going to be a slight increase.

Toby: You just said something really important, Lauren, when you said proportional because this is the game they keep playing is that the 89,000 agents and your audit rates proportionally is not going to go up. They always sneak in that proportionally. Well, okay, so let’s just look at the numbers.

Syracuse University does a study every year and I think it was 13.2 out of 1000 returns get audited for poor folks and 2.2 out of a 1000 get audited for wealthy folks. They’re saying that proportionality is not going to change. It doesn’t change the fact that the poor get audited way more than the rich, probably because they can’t defend themselves or they can’t hire a lawyer.

Lauren: Yeah, they can’t hire attorneys to defend them in an audit so they just pay up.

Toby: Proportionately, that’s not changing. Whenever they say things like oh, we’re going after the rich so that they pay their fair share. What they’re saying is we’re going to actually fund this division, but let’s be real. The people they audit, the highest audit rates are still the poor. It’s not even close. Even then you’re looking at the wealthy, they generally have multiple arguments. Even if they take an unreasonable position, you’re going to fight it and negotiate it.

Lauren: You get whatever that amount owed, get it negotiated way down.

Toby: Nobody really wants to go to tax court unless it’s a novel case and we don’t know the answer then sometimes you go to tax court, but otherwise the judge doesn’t want to. It’s a tax court, not an actual place. It moves around. When it comes to your town, you better have a legitimate beef or they’re going to be ticked off both the government and the taxpayer saying what are you guys doing here? But if it’s illegitimate, hey, we don’t know what the answer is. We need the judge to give us some direction and it’s never a bad thing. Sometimes you have to do that.

All right, I feel like what I find reading tea leaves with you is that there’s some fear mongering out there.

Lauren: There’s a lot.

Toby: It doesn’t have a basis in reality.

Lauren: No, it doesn’t. If you’re delinquent on your taxes, just get them filed and you’ll be fine. If you’re evading taxes, maybe be a little bit worried.

Toby: You’re a crook if you’re evading it. Let’s just be real. Hope they catch you. […] even though I have plenty of strategy, I still pay a lot of taxes. You’re screwing the rest of us, so please stop that. If you’re an evader, just pay your taxes or just file your taxes better yet. Even if you can’t pay them, there’s a payment plan. There are arrangements. They can’t squeeze you that hard. But if you don’t file and you don’t pay the penalty—

Lauren: They don’t like that. They don’t love that.

Toby: They’ll hit you with a stick. If you file your taxes, even if you owe them and say hey, I owe this, but I can’t pay this. You can even bankrupt away taxes. People don’t realize the IRS isn’t in the position. There’s no debtors’ jail. There are none of these things. Just don’t lie to them and follow the rules. Then if you do do it, the IRS, don’t ever talk to them. Just get an attorney or an accountant to represent you. I know it’s easier said than done, but trust me, you never want to be sitting there getting free discovery, which is when they can ask you all those questions.

Lauren: Never snitch on yourself.

Toby: Don’t snitch on yourself. I’m going to take that as a quote. Lauren Robins, don’t snitch on yourself.

Lauren: If you’re ever talking to someone in government authority. I do not recall. I need my attorney every time.

Toby: It’s easy and all you’re doing is because the accountant can say, I don’t know. I’ll look and find that out and then review, then they review everything else. When they ask you, what was this for? I don’t have a receipt do you? You start trying to talk your way around out of it.

Lauren: Yeah, don’t do that. Call a lawyer.

Toby: Easy peasy. All right, Lauren, thanks for joining us. It’s an interesting topic and there’s a lot of anxiety and fear around taxes. Hopefully, this lets people feel a little bit better. I just want to say thanks for joining us, and until next time.

Lauren: Absolutely. Thank you again for having me.