Polygon Simplification by Minimizing Convex Corners

Let P be a polygon with \(r>0\) reflex vertices and possibly with holes. A subsuming polygon of P is a polygon \(P'\) such that \(P \subseteq P'\), each connected component \(R'\) of \(P'\) subsumes a distinct component R of P, i.e., \(R\subseteq R'\), and the reflex corners of R coincide with the reflex corners of \(R'\). A subsuming chain of \(P'\) is a minimal path on the boundary of \(P'\) whose two end edges coincide with two edges of P. Aichholzer et al. proved that every polygon P has a subsuming polygon with O(r) vertices. Let \(\mathcal {A}_e(P)\) (resp., \(\mathcal {A}_v(P)\)) be the arrangement of lines determined by the edges (resp., pairs of vertices) of P. Aichholzer et al. observed that a challenge of computing an optimal subsuming polygon \(P'_{min}\), i.e., a subsuming polygon with minimum number of convex vertices, is that it may not always lie on \(\mathcal {A}_e(P)\). We prove that in some settings, one can find an optimal subsuming polygon for a given simple polygon in polynomial time, i.e., when \(\mathcal {A}_e(P'_{min}) = \mathcal {A}_e(P)\) and the subsuming chains are of constant length. In contrast, we prove the problem to be NP-hard for polygons with holes, even if there exists some \(P'_{min}\) with \(\mathcal {A}_e(P'_{min}) = \mathcal {A}_e(P)\) and subsuming chains are of length three. Both results extend to the scenario when \(\mathcal {A}_v(P'_{min}) = \mathcal {A}_v(P)\).