Prepayment penalties on your existing mortgage could make refinancing more costly. Check the details of your current loan agreement and be sure to factor in the cost of any prepayment penalty when you consider the benefits of refinancing.