To address the problems of scientific theory of brain and mind, common technology and engineering application of nature-inspired intelligence, this paper is focused on research in the semantic-oriented computing framework design for multimedia and multimodal information. The multimedia neural cognitive computing model of brain-inspired computing was designed based on the brain mechanism of nervous system and mind architecture of cognitive system. Furthermore, the semantic-oriented hierarchical cross-modal cognitive neural computing framework of brain-like computing was proposed based on multimedia neural cognitive computing model. Furthermore, the formal description and analysis of cross-modal cognitive neural computing framework was given. It would effectively improve the performance of semantic processing of multimedia and cross-modal information such as target detection, classification and recognition in high-resolution remote sensing image, and has far-reaching significance for exploration and realization brain and mind inspired computing.