Structured Matrices and the Algebra of Displacement Operators