In this journal, Cheng has proposed a backpropagation (BP) procedure calledBPFCC for deep L-stage fully connected cascaded (FCC) neural network learning in comparisonwith a neuron-by-neuron (NBN) algorithm of Wilamowski and Yu; yet, Cheng apparentlyneglected two major points of the NBN algorithm: (1) the merit of NBN is emphasizedon the multiple q-output (q > 1) case; and (2) NBN is described for a q-outputFCC different in structure from Cheng’s FCC type. In nonlinear regression (for minimizingthe sum of squared residuals), NBN employs forward passes only on the q-output FCC toevaluate the Gauss-Newton (approximate Hessian) matrix ∇rT∇r, the cross product of theJacobian matrix ∇r of the residual vector r. Notably, both BPFCC and NBN are designedto reduce the cost for evaluating (q rows per datum of) the Jacobian matrix ∇r rather thanthe dominant cost for forming the cross product ∇rT∇r by rank updates. The purpose ofthis paper is to present a new BP procedure that exploits the special FCC network structureof Cheng’s type for reducing the dominant cost (with no rows of ∇r explicitly), evaluatingthe Gauss-Newton matrix efficiently in a block arrow matrix form when q > 1.