Unified Picture of the Local Interstellar Magnetic Field from Voyager and IBEX