On holographic Rényi entropy in some modified theories of gravity