Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following