Compared with the acoustic source localization based on single array, the multi-microphone arrays have the higher robustness and accuracy in complex environment. However, it is still one of the challenges to obtain an optimal result by fusing multiple groups of the position information from multiple sub-arrays. To solve this problem, this paper introduced the Brayden Fletcher Goldfarb Shanno (BFGS) method into the localization method based on multi-microphone arrays. The BFGS method is used to an optimal objective function, which is related to the direction of the acoustic source. In addition, several initial value selection schemes of BFGS method are proposed and compared to select a best initial value. The experiments were carried out on the self-developed platform with four four-element sub-microphone arrays.