Modified critical behavior and crossover in the site impure Baxter-Wu model