Building a system for extracting information from the scientific literature is an important research topic in the field of inorganic materials science. However, conventional extracting systems have a limitation in that they do not extract characteristic values from non-textual components, such as charts, diagrams, and tables, which provide key information in many scientific documents. Although there have been several studies on identifying the characteristic values of graphs in the literature, there is no general method that classifies graphs according to the property conditions of the values in the field of materials science. Therefore, in this study, we focus on the graphs that are figures representing graphically numerical data, such as a bar graph and line graph, as the first step towards developing a framework for extracting material property information from such non-contextual components. We propose deep-learning-based classification models for identifying the types of graph properties, such as temperature and time, by combining graph images, text in graphs, and captions in neural networks. To train and evaluate the models, we construct a material graph dataset with different types of material properties from a large collection of data from journals in the field of materials science. By using cloud sourcing, we annotated 16,668 images in about 3 days. Our experimental results demonstrate that the best model can achieve high performance with a micro-averaged F-score of 0.961.