[squiffy]

Two issues.
Is now the best time to buy the lot? Is it possible you might get a better deal in a year or two, if real estate prices are inflated now where you live, and will decline?

Buying raw land can be costly because you may be paying taxes on it, with no rental income to offset those costs.

But if you want to build your dream home there, I guess it is worth the cost and the risk.

Also, stock market returns, over long periods of 16-20 years tend to be superior. But for intermediate and shorter periods, returns can be downright bad. (Stocks for the Long Run, Siegel)

So it's not clear that my assumption is correct that keeping your 401 K money in stocks is a good thing.

But having said that, I still think the best move is to borrow against the equity in your home, rather than drain your 401K.