Coordinate Descent Algorithms With Coupling Constraints : Lessons Learned

Coordinate descent methods are enjoying renewed interest due to their simplicity and success in many machine learning applications. Given recent theoretical results on random coordinate descent with linear coupling constraints, we develop a software architecture for this class of algorithms. A software architecture has to (1) maintain solution feasibility, (2) be applicable to different execution environments, whether local or distributed and (3) decouple problem-specific logic from the execution environment. We demonstrate that due to the nature of these algorithms, these requirements raise some issues that are not present in many other classes of machine learning algorithms and thus can be overlooked when designing a generic machine learning system.