Plans to demolish an empty office building in Wycombe have been given the go-ahead.