A $20,000 withdrawal could have a far bigger impact on your future than you might expect.