Is It a Smart Idea to Pay Off the Rental Property?

Is It a Smart Idea to Pay Off the Rental Property?

I’ve been salivating over these insanely low mortgage rates out there right now. But I never even thought that another option might be to just pay off the rental property.

Today I want to tell you about how I went from unsuccessfully trying to refinance to considering that maybe I just pay off the rental property. It even involves a discussion with the one and only Clark Howard!


There’s always discussion in the real estate world about whether it makes sense to buy property out-of-state or other areas where the costs are lower and the rents are higher (in perspective to the costs).

Lucky for us, that was never an issue because northeastern Ohio is one of those good rental areas and we lived there at the time we started investing.

We bought our duplex back in 2015 during a quest for a second rental property. We were able to purchase it for $98,000 – less than $100k for a nice ready-to-rent two-unit property! It was built in 1967 and it’s located in an area where duplexes are the norm and solid renters are common.

I put down about $25k and was able to secure a loan for $73,500 at 4.75% for 30 years. The property is now worth around $150,000. I’m not in the game for the appreciation, but that’s always nice to see!

In the meantime, our tenants are paying $800 each in rent. That’s $1,600/month with a mortgage payment (including taxes and insurance) of only $750/month – and that’s with me adding some extra to pay it off a little faster.

Currently, we owe $63,739.63 on the duplex. So after 5 years of owning the place, the tenants have paid off just under $10k on it… not too shabby! In other words, not only am I making money every month on the place, but the tenants are also helping to build the appreciation on the property.

I’m not saying everything’s perfect. We’re in the process of evicting a tenant for not paying rent and we get the occasional repair headaches, but overall, we’re in good shape. And knowing that one side of the duplex can cover the whole mortgage payment each month keeps a lot of the stress down.

As a side note, this is the only rental property we still hold. We sold our first rental house in 2018, which was becoming a real pain in the butt. Unlike the duplex, the house is in a rough area (there was a shooting incident directly in front of the house!) and it’s over 100 years old. It was one problem after another… I really learned from my mistakes when we bought the duplex!

The refinance problem

The start of this entire rabbit hole was that I wanted to refinance the mortgage on the duplex. It’s hard to complain about a fixed rate of 4.75% but with rates as low as they are now, I could do a lot better.

Per Bankrate:

On Thursday, August 06, 2020, according to Bankrate’s latest survey of the nation’s largest refinance lenders, the benchmark 30-year fixed refinance rate is 3.080% with an APR of 3.270%. The average 15-year fixed refinance rate is 2.650% with an APR of 2.860%. The 5/1 adjustable-rate refinance (ARM) rate is 3.300% with an APR of 3.990%.

While poking around, I’m seeing rates of 2.775% for a 20-year loan. Plugging in my current and proposed numbers into a calculator like this one on NerdWallet gives me these results:

Is It a Smart Idea to Pay Off the Rental Property? - NerdWallet Refinance Calculator Results 2.775%

That looks fantastic, right? I’d shave about 5 years off my loan and would drop my payment down a few bucks every month.

Unfortunately, that’s only half the story. If this was our personal residence, done deal. But with a rental property, you’re generally not going to get the same rates. More commonly, you can expect to pay one extra percentage point for that type of mortgage.

To put it simply, we could probably expect to find a loan for the property for around 3.775% instead of 2.775%. While not as exciting at that rate, the results still aren’t too bad:

Is It a Smart Idea to Pay Off the Rental Property? - NerdWallet Refinance Calculator Results 3.775%

Even though my loan payment would increase by $17/month, I’d still shave 5 years off my loan. Depending on the actual rate I could get, it may or may not be worth pulling the trigger on it.

Here’s the problem – getting a bank or mortgage broker to help me was going from one dead-end to another. First of all, refinances are hot right now. A lot of the institutions are struggling to even keep up with all the applicants. So who would want to deal with the measly $64k I still owe on the property? Not many.

But the other issue I’m running into is that I don’t have a regular job. I’m retired… happily retired. Yeah, I make a little money blogging, but not enough for them to want to give me a new mortgage.

Even though our net worth is currently over $1.3 million and we probably have a steadier paycheck than a lot of folks, that doesn’t mean much. You’d also think that our rental income being twice as much as our current mortgage payment (taxes and insurance included) would help green flag this for a loan.

Nope.

I knew this going into it, but early retirement brings along some issues like trying to buy or refinance a property.

A whole different perspective from Clark Howard

I used to listen to a ton of personal finance podcasts all the time. But now, I’m a lot pickier. Even though I have my favorites, I’ll only listen to an episode here or there if it truly seems interesting.

However, there’s one that I listen to every single day… The Clark Howard Show. I’ve been listening to him for years and I usually learn something new in every episode. I find him entertaining and his advice is generally spot-on. Plus, he touches on other topics like technology that I enjoy as well.

So I decided to write to him about my refinance problem a few weeks ago. Would you believe that I got a reply later that same day asking if I wanted to be on a call with Clark Howard the next day? Of course, I would!

We were actually on our way to go camping at the time the call was scheduled for so we pulled off to a store parking lot and I did the call with him.

And, of course, camping with friends was wonderful, too! Here’s Faith finishing her job of pounding in the tent stakes…

I explained my situation to Clark about how I was struggling to find a loan provider because of my early retirement and the smaller loan amount. I was hoping to find an alternative avenue where I could make the refinance possible.

Here’s why I love the guy. He listened to what I was saying, took a step back, and said something along the lines of, “So, I’m thinking of something a little different. With you being an early retiree, you’re probably sitting on a lot of cash right now, right?”

When I replied “yes” with it being part of our withdrawal strategy to help protect against the sequence of returns risk, he mentioned that I’m probably not earning much in savings on it. That’s correct. My online savings account with Ally is only earning about 1% right now.

His thought… what about just paying off the property completely?

Hmm, that’s an interesting idea that I just hadn’t even thought of before. I had always just kept my blinders on with letting the tenant pay off the mortgage.

But this really was an eye-opening thought. If I’m earning a paltry 1% on my money in savings, wouldn’t it make more sense to put it toward a debt of 4.75%? That would be a much better return on my money. It would even be a better return on my money than a refinance, too!

There was more information and back and forth, but I found this to be considerably helpful – a fresh and valuable perspective.

And just so you know, I never heard my call on his show… I must be pretty boring to talk to on the air! Doesn’t he know that we were on an episode of House Hunters International?! Guess that isn’t very helpful either. 😉

UPDATE: What are the chances my appearance on The Clark Howard Show aired on the same day that my post came out?! If you’re interested, you can check it out on the Clark Howard Podcast page. It’s episode 8.10.20 and I’m on the air at about the 13:24 mark. Thanks for the heads up, Thomas!

Why we should pay off the rental property

Though the notion to pay off the rental property wasn’t something I had considered before, it sure was a reasonable idea.

Mathematically, it just makes sense. Paying off debt with a 4.75% interest rate using money only earning 1% is a much better return on my money. That’s about as simple as it can get.

Additionally, my net profit on rental income from the duplex would also go up substantially. That would be a nice check every month from the property management company after they take out their cut. The money we get could then either go toward our own monthly living expenses or be used to put back into our cash bucket for use a few years down the line.

And then there’s the psychological aspect. This mortgage is our only debt. I consider it to be good debt since our tenants are paying for it and we’re making extra money on it as well, but it’s still debt. And obviously, it’s much easier to sleep without a mortgage over our shoulders. I don’t have much stress since one tenant can cover the payment anyway, but it would still be nice not to have to think about it at all.

Why we shouldn’t pay off the rental property

Ok, so it seems like a no-brainer to pay off the rental, right? Not so fast! After thinking it over and talking with a friend who’s very much on top of this stuff, there were a few realizations.

First off, there’s a small nuance I didn’t realize. We get an interest deduction on our taxes for having the mortgage on our rental. I’m not an expert in taxes, but I thought that this went away for those filing a standard deduction which is what we’re now doing.

But that’s only true on a personal residence – we still get the deduction for a rental property on Schedule E. Last year, we had $1,309 in interest. With the Roth IRA conversions we’re doing every year now, that deduction gives us the breathing room to convert that amount at the target tax rate. But it’s not a ton of dough… we could just convert a little less each year to accommodate for that – not that big of a deal.

Another possible issue to remember is that if we pay off the rental property… well, that money’s gone. When you have money in the bank or the stock market, you can always move it around or take it out if needed. When you pay off a debt like this with a chunk of money, you can’t get it back. Sure, a way around that might be to get a home equity loan or line of credit but that’s a whole separate project.

Finally, the biggest obstacle is that the numbers aren’t as straightforward as they seem. Even if I could get a mortgage at the 3.775% I mentioned (or lower), I can’t just compare that to the 1% interest rate I’m getting from Ally for two reasons:

  1. That interest rate from Ally is going to change over the years… and it’s probably going to go up. Over 20 years, it could absolutely go higher – maybe even a lot higher. Will it go higher than the 4.75% I’m paying right now? I have no idea.
  2. The money we would be using to pay off the loan wouldn’t just be from Ally at the 1% rate. In fact, most of it would come from the latter money we have in our bucket strategy. And that means it would come from selling off some BulletShare bond ETFs to pay off the mortgage. When I checked, I was surprised to see that BSCO has returned 5.32% YTD and 8.06% over the past year… that’s a dramatically better return than the 4.75% I’m paying in interest on my loan. That said, interest rates are crazy low right now. As rates go up (which they will), the return will inevitably go down. When will that be? I have no idea.

After looking at the pros and cons, this is not that easy of a decision to make. It’d be nice to pay off the rental, but I don’t think it’s going to really make that much of a difference over the long haul.

So if that’s the case, I might be back to square one of just trying to refinance the property, which is probably pretty unlikely. Personal finance is fun, my friends… good times!

What do you guys think? Pay off the rental property, try harder to refinance it, or just leave it alone?

Thanks for reading!!

— Jim

PS Shout-out to my friend for the great information on both sides of this equation – very much appreciated!

34 thoughts on “Is It a Smart Idea to Pay Off the Rental Property?”

  1. Good Morning Jim,

    Agreed it’s not a straightforward answer. The $64K you would need to pull equates to about $2560 per year in cash flow using the 4% rule. Then, there is the element of market timing, which I know we’re not supposed to talk about. 🙂 However, PE’s on US stocks are high right now. If I’m in your shoes, I would harvest the gains from the last 4 months and eliminate your last long term obligation.

    1. Thanks for your thoughts… such a tough call. It would be nice to be done with the debt but once that money’s gone, I can’t get it back. I don’t plan on needing it anytime soon, but you just never know. I really have some thinking to do!

  2. Hi Jim

    Absolutely, without equivocation, pay off the place. 🙂

    What all the spreadsheets don’t show is the absolute, warm, fuzzy, deep-down feeling of content, when you go to bed at night knowing you are completely debt free. 🙂

    My wife had been after me for years to pay off our primary residence. I resisted because, unlike you, our mortgage rate was less than the return we were seeing from our investments. From that perspective, it simply didn’t make sense to pay it off. However, there is an intangible, absolute feeling of content, freedom, etc., knowing you are completely debt free. Maybe that feeling was amplified because this was our primary residence and we were still living there, but still, it’s a wicked-cool feeling.

    I suspect you won’t even notice the $68k gone from your balance sheet (other than the brief pain of actually writing the check!) but seeing the large bump each month in your income will bring a smile to your face every month.

    I say, go for it!
    Jim

    1. Love it, Jim! That’s got to be a great feeling knowing you don’t have to make another mortgage payment again. On the flip side, I really don’t notice the mortgage payment since my tenants are paying for it. The property manager deposits my money in our business checking account and then the mortgage payment is paid automatically from it. Usually that account just continues to build up with excess until I move money from it and put it to use elsewhere.

      That said, getting rid of the payment completely would be nice regardless… tough decision!

  3. Hey Jim, I was listening to Clark this morning (8-11) & heard you on his podcast about half way through. Sounding good…. stay safe, stay positive & keep writing. I really enjoy reading your blog & following along with your wild early retirement journey.

  4. At this point in your journey, I think you gotta do what helps you sleep better at night.

    Revisit your original plan for the rental. Why alter that plan now?

    Does the annual savings on mortgage interest out weight the “cost” of your time and money (security) to refi?

    What would you save in interest by paying it off now? Is that worth the “cost”?

    How long would it take to replenish your “bucket” if you paid it off?

    With all your uncertainty right now, is it the right time to do anything? You might be looking for a home back in the U.S., which will increase your COL, and you might need to use money from your “bucket” for it.

    1. You’re so logical, Amy… I love it! It’s very true that it just might not be worth the time and effort to refinance. Paying it off might be a different story though with some good savings – that would difficult to calculate with so many variables. But either way, you bring up a great point about our COL increasing in the very likely scenario of moving back to the U.S. – having that safety net in place can be invaluable.

  5. Hi Jim, I’m with Jim G above re: placing a high value on that warm, fuzzy, mom-was-right-I-am-a-good-guy feeling you’ll get from paying off the loan. Plus, that would take that debt off your minds and balance sheet for any future considerations that might involve raising your daughter (college costs, etc.). Amy also has a point about possibly needing the cash to buy a US-based house, but I would think that you all are sufficiently flexible and informed that you would be able to leap that hurdle should it ever arrive.

    Buena suerte either way!

  6. My vote is for eliminating the debt. My husband and I are on the cusp of paying off one of our rental properties currently at a 3.99% rate.

    We live on my salary and use his for investing and eliminating debt. Since his pay is varied and unpredictable, it’s hard to make plans for it. In the current economic climate – even though I work in vaccines and should be safe – we’re choosing to knock out as much debt as possible. Our only debt is our primary home and two rental properties. One down. Two to go. Woot!

    1. Nice job, Lisa!! Congrats and double woot-woot!

      That’s interesting that you’re paying off a rental property having such a low rate, but it does make sense with the reasons you mentioned. You guys are going to be really rocking it debt-free before you know it!

  7. I think I would stay the course for the interim. Your plan is working despite all the variables of the past year.
    What I would not do is refinance. The calculator savings assume you will pay off the loan over the term – if you pay it faster the difference of 1% becomes minimal. Plus it is a hassle. We went above our target on our new home purchase and I chose to finance $150 k for 15 years (locked at 2.125% – not bad). Preserving cash allows us to do some upgrades. But trying to prove I can draw revenue from retirement savings has been a hassle – plus they still insisted on an overpriced appraisal despite only 30% LTV. If I entered escrow today I would be more inclined to burn our cash cushion and be debt free. Maybe once I’ve actually seen the impact of living off savings for a couple years I’ll pay it off. Not for the financial sense – I expect investments to earn more – but for the peace of mind and independence that being debt free entails.

    1. I think we’re on the same page about the refi… seems like too much of a hassle to be worth it. Sorry you had to go through a bunch of crap to find out what a pain-in-the-butt it can be.

      Not dealing with a refi is good because I don’t have to worry about missing out on rates. I can actually take my time to think about whether or not it makes sense to pay off the debt. Who knows – maybe that’ll be a Christmas present for ourselves! 😉

  8. Cash is king, I would keep it in your bank account!! Once you hold the money in the house it would be hard to take out a loan or obtain a HELOC if you ever needed the money in the future.

  9. Hi Jim, here’s a 3rd alternative maybe you haven’t considered. Since you guys are stuck in the U.S. for a bit, and you are in the process of evicting a tenant from the duplex, why not “move in” to that half? Then your property will be owner occupied, and you might be able to refinance at owner occupied rates and get the deal done. With your cash on hand, you can always pay it off at any time, but this will preserve your cash during these uncertain times. When Panama opens up, if you decide to return, you can just rent it out again.

    1. That’s an interesting thought, JennyMac! I guess the first hurdle would be deciding if we want to live in Ohio or not. But if we do, that could be a nice way to make the magic happen. It wouldn’t be as cheap as living in my in-laws basement but it still wouldn’t be too shabby! 😉

  10. Hi Jim –

    You might also take a look at recasting the loan, where you make a lump sum principal payment and reduce the monthly payment amount for a small fee – Quicken charges $250 and requires at least $10,000 lump pay down. Kind of a middle of the road option. I too have been noodling on paying off rental mortgages or not, and just recently learned of the recasting option! I’m having a hard time letting go of the liquidity though – I don’t really need it, but what other great things could I use that money for? Great to have options, so here’s another one for your toolbox. 🙂

      1. Jim, I don’t think so. I have conventional 30-year mortgages on my rentals and Quicken immediately gave me their parameters for re-amortizing after a pay down. I read the big banks do it, but smaller ones may not.

  11. Hello Jim,
    Glad you’re getting time to spend with family and friends ☺️
    We paid off our rental, (3.75%)a couple years ago, even though financial advisers told us to invest the money. Since we were preparing to retire (2020), we didn’t want to gamble. It gave us tremendous peace of mind. Maybe we could have “netted” more in market, but at 2 years away from retirement we would have invested conservatively. I would have been sick if we lost money we set aside for payoff.
    Personally if I were you I wouldn’t do anything for a couple months, until you decide where you’re going to reside. However, the one thing I WOULDN’T do is refinance. As you have found it’s not that easy to get a loan without a job😜. You probably won’t get the “best” rate. When you add in the junk fees banks always charge, appraisal, survey, other closing related charges you’re just throwing money away. In my opinion the only reason to hang onto the cash is if you may want to pay for a new residence outright and avoid going through the hassles you’re currently experiencing (also maybe better purchase price if paying cash). Those are just my thoughts for what it’s worth. It’s so nice to be in a position to have choices 🙌
    Take care & safe travels 🤗

    1. Glad to hear from someone who’s gone through this same scenario, Kathy! I think I’m on the same page with not refinancing – it just doesn’t seem worth the time, effort, and headaches for the small amount of potential savings.

      I also agree with your thought on for waiting a while to decide if I want to pay it off… especially since that’s something you can’t take back once you hand over your money. Considering that paying it off is irrelevant of interest rates like refinancing would be, I’ll take my good old time to think this one through.

  12. Last year, we moved into our duplex. Refinanced it and paid off the mortgage on our rental condo. We owe about the same amount, but at a lower rate and only one mortgage instead of 2. It’s easier to deal with.
    Maybe you can do something like that. Go live in the duplex for a while and refinance it.
    Having no steady paycheck will still be a problem, though. Maybe you can look at some local lenders.
    I would not pay off the mortgage. That will deplete your cash savings. Nobody knows how long this recession will last. It could take a few years to get back to normal. The stock market can’t keep this up.

    1. That’s not a bad idea… if we decide to stay in Ohio. There are definitely some nice incentives to living in a multi-unit property and I’m glad it’s working out well for you. I’m probably not going to refinance – it just doesn’t seem worth the time, effort, and headaches involved. As far as paying it off goes, I think I just need to think more about it. The nice thing is that I don’t have to try to jump on it now since it’s not dependent on something like interest rates like a refi would be.

  13. I worked as a mortgage underwriter for 20 years at a nationwide bank. Unless you can show dividends/interest for the past 24 mos to use as qualifying income or enough self employed income for 24 mos, I don’t know how you would qualify. The only other way I can think of is to call yourself a self employed landlord and use your bottom line rental income to qualify (along with dividends/interest for 24 mos), if that covers all your debts. You have to disclose the rent you are paying in Panama so it doesn’t sound like your debt-to-income ratio would be low enough to qualify. I would just pay it off.

  14. Hi Jim,
    I would pay off the rental personally, but I also have this conundrum because I seriously consider recasting our mortgage. The problem is I only have six years left at 2.875% (only 97k left). But if I recast to a 30 year it would cost me more in interest, but I would get an extra $900 in cash flow. I could put that money in brokerage account and make better returns. Not sure what to do, but if I had more cash I would put it to work.

    1. That really is a fun one, Jason… the nice thing is that these are good problems to have. That’s a fantastic rate to have scored on the rental regardless – congrats on that one! Not sure what I would do in your situation… I bet I’d probably lean toward just leaving it alone and being done with it in six years. I’ll be curious to hear what your decide to do.

  15. I would pay it off. I did that at double the amount about a year before I was downsized at only a 2.85% rate/10 yr. I have no regrets. As Dave Ramsey says “If you really want your mortgage back you can always go get one.” You can view it as your own annuity. You put in the $60k and you get back “X” extra every month. Not having to deal with a mortgage is a good feeling. As I get older I find that simplifying things makes my life better.

    1. Yeah, that really does sound nice and much simpler. I’m leaning toward following suit but I’ll probably sit on it for a few months just to think about it. It won’t hurt to ponder it a little more before pulling the trigger but that bigger paycheck every month does sound appealing. And like you mentioned, it’s one less account on the books to worry about.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.