Real Estate, Retirement Income, and the IRS Problem

In this episode of the Divorce the IRS Podcast, Jimmy Miller continues the conversation from last week’s episode on the FIRE movement by taking a closer look at real estate investing and how it fits into the Divorce the IRS framework.

Real estate is one of the most common investment topics Jimmy gets asked about. How does it compare to the stock market? Can it work as a retirement income strategy? Does it help or hurt when the goal is to reduce taxes in retirement?

Jimmy gives his honest perspective on the appeal of real estate, including the ability to own something tangible and understandable, while also explaining why he does not view rental properties as an ideal primary investment strategy for retirement.

In this episode, Jimmy discusses:

• How real estate compares to long-term stock market investing
• Why home values and investment returns are not always the same thing
• The hidden costs of rental properties, including repairs, vacancies, insurance, HOA fees, and management
• Why being a landlord may not fit the retirement lifestyle many people actually want
• How rental income can affect the taxation of Social Security benefits
• Why provisional income matters in the Divorce the IRS framework
• How depreciation recapture works when a rental property is sold
• Why some investors underestimate the tax bill that can come at the end
• The difference between owning rental properties directly and investing in REITs
• How REITs can offer real estate exposure inside a diversified portfolio

Jimmy also explains why real estate investment trusts, or REITs, may be a more practical way to include real estate in a retirement portfolio, especially when used inside Roth accounts where income and growth can potentially avoid creating provisional income.

In the next episode, Jimmy will discuss another popular retirement philosophy: the desire to die broke.

• Visit Divorce-the-IRS.com
• Visit Baobab Wealth
• Visit Baobab Wealth Abroad
• Buy a copy of Jimmy's book, Divorce the IRS
• Follow us on Facebook
• Subscribe to us on YouTube
• Connect with us on LinkedIn

Transcript

SPEAKER_00 0:01

Welcome to the Divorce the IRS Podcast, the retirement income planning podcast designed specifically for those who want to pay the least amount of taxes possible and build a retirement income that lasts. Inspired by the best-selling book, Divorce the IRS, you get to go behind the scenes with financial planner, author, and speaker Jimmy Miller. Learn how to set yourself up to pay the least amount of taxes in retirement when you'll need your money the most. And now, here's your host, Jimmy Miller.

SPEAKER_01 0:27

Welcome. Welcome to episode 28 of the Divorce the IRS podcast. Following our discussion last week about the fire movement, this week we will dive into real estate as an investment and specifically how it fits into the Divorce the IRS framework. Real estate investing is something I get asked about a lot. The questions usually fall into three main categories. How does it compare to investing in the stock market? Does it work within the concepts of divorce the IRS? And does it work well as a retirement income strategy? We'll take an admittedly biased look at all three in this episode. And my goal here is to help you see the issue from another angle and to give you some basic facts to consider. As I tell my clients all the time, it's never my job to tell you what to do with your money, only to give you the answers needed so that you can make the best decision for yourself. Now I understand the appeal of real estate, like owning something tangible that is also understandable. I also want to admit up front that I am not a proponent of investing in real estate as a primary investment strategy, either while saving for retirement or for income during retirement. The reasons are simple and many. I don't really like being a landlord, and it's hard to do while you're away from your rentals, while you're traveling in retirement. It's labor intensive. The income can fluctuate. There's lots of liabilities, costs, and fees. Real estate is very illiquid, it's non-diversified, and so on. I don't intend to dissuade people from real estate investing. I believe that everyone should try it, and most do I find, and then they can draw their own conclusions about it. Let's start though by looking at how it compares to investing in the greatest companies in the world, the stock market. Now in 1975, the average house in America cost around $38,000 to purchase. Today the average is about $420,000. That comes out to a compounded growth rate of 4.8% over those 50 years. If you back out the average inflation rate, which was 3.8% over the past 50 years, you get your real rate of return. That's how much your money is actually rising in value as defined by what you can actually buy with it. Your real rate becomes about 1%. Even though real estate can be tax-efficient investments while owning it, once you back out inflation and then back out your taxes, you're often left with a negative real return in real estate investing. Now we don't need to discuss the number of upgrades, repairs, vacancies, HOA fees, insurance costs, etc., that come along with the rental property for you to see my point. Homes are designed to house families, not to be an investment. I like to joke with people when discussing real estate as an investment by stating that I've never had to put a new roof on my S P 500 index fund. Now, I've heard all the counter-arguments, using other people's money, the bank to leverage the investment, renters pay the cost of the loan for you, buying in higher than average growth areas, etc. None of these will change my mind or would make me want to do all the work involved with real estate, or to pay someone else most of my profits to do it for me. Had I put that same $38,000 in 1975 into an S P 500 index fund and left it alone for the past 50 years, it would be worth more than $12 million today. That's a 12.8% annualized compound of return with dividends reinvested. Back inflation out of that return, and you're still left with a real rate of return in the 9% range. Back taxes out now, which are the favorable long-term capital gains rates, and you're still way ahead and actually growing your wealth, what your money can actually purchase. If you would only put $1,500 into the SP $500 in 1975, you would have over $500,000 today, more than the average house value. You probably would have put down more than $1,500 on that house in 1975 as well. I think that if every home in America had a price ticker, just like stocks do, over the front door, showing second to second what the value of the home was worth, most Americans would come totally unglued. The truth is that homes and real estate in general are volatile and the prices go up and down daily, just like the values of companies. People just don't have that price ticker over their front door, so they don't think about what their home is worth day to day. Rental homes or Airbnbs don't work particularly well for retirement income or within the divorcey IRS concept. Maybe you don't mind being a landlord while you're in your working years, but I don't sit down with many couples to discuss what they want their retirement to look like and hear that they can't wait to run over and fix a broken toilet or a water heater at the rental unit. I usually hear things about world travel and spending time with the kids and grandkids. That rental unit or many units will usually only detract from these retirement goals. Now, all rental income is going to be considered provisional income in the eyes of the IRS. And if you remember from tax time bomb four, which was episode 13, you will recall that provisional income is what the IRS uses to determine the taxability of your Social Security benefits. The provisional income thresholds to keep the IRS from taxing up to 85% of your Social Security are not that high. Any rental income at all usually pushes retirees above that threshold. And thus the rental income can be the cause of your Social Security becoming taxable. Factor that into the return of the rental, and you will find that it often makes the real rate of return on a rental negative. The IRS will love you though. The other issue with real estate that I find people don't know or understand that well is the recapture of depreciation. Most people love the depreciation write-offs on their taxes while owning rental income properties. But this is just like borrowing money from the IRS, as all the depreciation will have to be recaptured when the property is ultimately sold. And that's usually when you're too old to manage the property or properties any longer, and or you get sick of the tax bills. Depreciation recapture on investment real estate is taxed at your ordinary income tax rate and capped at 25%. You will only pay long-term capital gains tax on the actual gain in value of the property, if there is any, and ordinary income tax up to 25% on the amount of depreciation you took over the years. This tax at the end of the investment, on top of the real estate commissions and fees paid to sell the property, paid in a lump sum, ends up making the rental's real rate of return not so great, if there is any return at all. People often come back at me about this with, well, I'll just keep the properties till I die, and my kids will inherit them with a step up and cost basis, and the depreciation recapture will die with me. And that's a great thought, and I have actually seen it happen, but rarely. There are few 90 plus year olds that still like owning and managing rental properties so that they can pass them along to the kids who have moved away and are busy with their own lives and are not around to help any longer. Now I don't want to beat up on real estate too much here. This is something that I have invested in myself in the past, and I've had to learn it the hard way, which is sometimes the best way to learn things. And I do still invest in real estate actually. I use real estate investment trusts, or REITs for short, within the context of a well-diversified, high-quality investment portfolio. REITs offer an easy and liquid way for everyday investors to invest in commercial and residential real estate at a scale only available through REITs or REIT mutual funds and ETFs. This is real diversification, as opposed to owning a few units in your hometown. It's tax efficient, and it's an easy way to put real estate inside your Roth IRA accounts so that all the income and growth is tax-free for you and doesn't count towards provisional income. Now there's a lot of ways to invest and to make money and create residual streams of income. Real estate, outside of REITs, is just not the way for me. It doesn't lend itself well to growing real wealth over the long term. It's too much effort, and it doesn't work well if you really want to divorce the IRS one day. That's about all I want to say today about real estate investing. In the next episode, we'll discuss another retirement philosophy that has become very popular. What to do if you have the desire to die broke. So stay tuned for that discussion next week.

SPEAKER_00 9:26

Want even more ideas, tools, and resources on how to navigate your financial life? Check out all the resources on the Divorce the IRS website at divorce-b-IRS.com or the Bayob Wealth website at bayablewealth.com and subscribe to the blog to stay up to date on issues affecting retirement income planning. Don't forget to subscribe to the podcast so you never miss an episode. Bayob Wealth and Bayabut Wealth Abroad are DBAs of Bayob Wealth LLC, a Florida registered investment advisor. This podcast is designed for general education purposes only and shouldn't be taken as legal investment or tax advice. You should seek out a qualified tax professional or licensed financial advisor to determine what is best for your personal situation.