Waiting algorithms for synchronization in large-scale multiprocessors

Through analysis and experiments, this paper investigates two-phase waiting algorithms to minimize the cost of waiting for synchronization in large-scale multiprocessors. In a two-phase algorithm, a thread first waits by polling a synchronization variable. If the cost of polling reaches a limit <italic>L<subscrpt>poll</subscrpt></italic> and further waiting is necessary, the thread is blocked, incurring an additional fixed cost, <italic>B</italic>. The choice of <italic>L<subscrpt>poll</subscrpt></italic> is a critical determinant of the performance of two-phase algorithms. We focus on methods for statically determining <italic>L<subscrpt>poll</subscrpt></italic> because the run-time overhead of dynamically determining <italic>L<subscrpt>poll</subscrpt></italic> can be comparable to the cost of blocking in large-scale multiprocessor systems with lightweight threads. Our experiments show that <italic>always-block</italic> (<italic>L<subscrpt>poll</subscrpt></italic> = 0) is a good waiting algorithm with performance that is usually close to the best of the algorithms compared. We show that even better performance can be achieved with a static choice of <italic>L<subscrpt>poll</subscrpt></italic> based on knowledge of likely wait-time distributions. Motivated by the observation that different synchronization types exhibit different wait-time distributions, we prove that a static choice of <italic>L<subscrpt>poll</subscrpt></italic> can yield close to optimal on-line performance against an adversary that is restricted to choosing wait times from a fixed family of probability distributions. This result allows us to make an optimal static choice of <italic>L<subscrpt>poll</subscrpt></italic> based on synchronization type. For exponentially distributed wait times, we prove that setting <italic>L<subscrpt>poll</subscrpt></italic> = 1n(e-1)<italic>B</italic> results in a waiting cost that is no more than <italic>e/(e-1)</italic> times the cost of an optimal off-line algorithm. For uniformly distributed wait times, we prove that setting <italic>L</italic><subscrpt>poll</subscrpt>=1/2(square root of 5 -1)<italic>B</italic> results in a waiting cost that is no more than (square root of 5 + 1)/2 (the golden ratio) times the cost of an optimal off-line algorithm. Experimental measurements of several parallel applications on the Alewife multiprocessor simulator corroborate our theoretical findings.