Quantum computers hold promise as accelerators onto which some classically-intractable problems may be offloaded, necessitating hybrid quantum-classical workflows. Understanding how these two computing paradigms can work in tandem is critical for identifying where such workflows could provide an advantage over strictly classical ones. In this work, we study such workflows in the context of quantum optimization, using an implementation of the Variational Quantum Factoring(VQF) algorithm as a prototypical example of QAOA-based quantum optimization algorithms. We execute experimental demonstrations using a superconducting quantum processor, and investigate the trade-off between quantum resources (number of qubits and circuit depth) and the probability that a given integer is successfully factored. In our experiments, the integers 1,099,551,473,989 and6,557 are factored with 3 and 5 qubits, respectively, using a QAOA ansatz with up to 8 layers. Our results empirically demonstrate the impact of different noise sources and reveal a residual ZZ-coupling between qubits as a dominant source of error. Additionally, we are able to identify the optimal number of circuit layers for a given instance to maximize success probability.