Beneath-market rates of interest may inspire Uncle Sam to look to your loved ones for taxes on the curiosity you'll have had to pay a conventional lender. Bankers' resentment of the lenders was longstanding: When their clients take out property tax loans, it impairs the collateral value of the property and might have an effect on the borrower's ability to make mortgage payments on time.