An androgen receptor (AR) is an intensively studied treatment target for castration-resistant prostate cancer that is irresponsive to conventional antiandrogen therapeutics. Binding function 3 (BF3) inhibitors with alternative modes of action have emerged as a promising approach to overcoming antiandrogen resistance. However, how these BF3 inhibitors modulate AR function remains elusive, hindering the development of BF3-targeting agents. Here, we performed an integrated computational study to interrogate the binding mechanism of several known BF3 inhibitors with ARs. Our results show that the inhibitory effect of the BF3 antagonists arises from their allosteric modulation of the activation function (AF2) site, which alters the dynamic coupling between the BF3 and AF2 sites as well as the AF2-coactivator (SRC2-3) interaction. Moreover, the per-residue binding energy analyses reveal the "anchor" role of the linker connecting the phenyl ring and benzimidazole/indole in these BF3 inhibitors. Furthermore, the allosteric driver-interacting residues are found to include both "positive", e.g., Phe673 and Asn833, and "negative" ones, e.g., Phe826, and the differential interactions with these residues provide an explanation why stronger binding does not necessarily result in higher inhibitory activities. Finally, our allosteric communication pathway analyses delineate how the allosteric signals triggered by BF3 binding are propagated to the AF2 pocket through multiple short- and/or long-ranged transmission pathways. Collectively, our combined computational study provides a comprehensive structural mechanism underlying how the selected set of BF3 inhibitors modulate AR function, which will help guide future development of BF3 antagonists.