Median algebras acting on sets