The never-ending journey of material scientists exploring materials from the vast expanse of chemical space has now found a powerful computational guide called deep learning. In material research driven by deep learning, an artificial neural network (ANN) is trained by structure-property datasets to develop predictive capability of properties based on the observed structure. Such ANNs have been used to discover new materials with targeted properties [1,2] in the field of chemical compounds [3–5], solid-state inorganic materials [6–8], and materials with periodic structures [9,10]. For the ANN to function, the material structure must be precisely defined, and an expansive set of structure-property data, usually provided by public databases, must be available for training.
However, the structures of tangible materials most prevalent in our daily life, such as plastic composites, metal alloys, and rubbers are too complex to be defined, i.e., they are not periodic and may have multiple levels of structural hierarchy. As such, structure-property databases do not exist, which makes it impractical to apply deep learning to these classes of materials. However, recent advances in deep learning technologies might provide a rational route to overcome this limitation. The generative adversarial network (GAN), developed by Ian Goodfellow in 2014 [11], is a new class of ANN that can computationally generate fake images of abstract objects, such as human faces, cats, and dogs. This technology has the potential to carry out deep learning research on a very wide range of materials, and, in recent years, substantial efforts have been placed along this direction [12–17]. For example, B. Kim et al. have created zeolites with a user-desired methane heat of adsorption [16]. Y. Mao et. al. have proposed an experience-free approach to design configurations of architected materials with GANs that reach the Hashin-Shtrikman upper bounds of isotropic elasticity and strain energy storage [17]. L. Banko et. al. have extracted the synthesis-composition vs microstructure of thin films [13]. Although these studies represent important advances, to move forward and possibly surpass existing technologies requires the ability to implement “virtual” experiments on tangible materials, where the processing of materials into a tangible form, characterization of the structure, and measurement of properties are all reproduced on computers.
Here we propose a deep-learning-based computational framework that is able to implement “virtual experiments” on tangible materials with hierarchal, nondefinable, and nonperiodic structure (Fig. 1a). Central to our work reported here is the use of multiple GANs that create fake images of the structures of the form at different scales with various compositions. These images were merged into a “tiled” image that includes information of the hierarchal structure, from which an ANN predicted multiple properties (Fig. 1a).
Although our computational framework can be applied to various tangible material systems, in this work we featured carbon nanotube (CNT) films (Fig. 1b). The CNT films were fabricated through vacuum filtration of CNTs dispersed in solution, and the surface structure was characterized by scanning electron microscopy (SEM) at multiple magnifications. In addition, the electrical conductivity and specific surface area of the films were measured (Fig. 1a). Films of CNTs represent excellent examples of hierarchal structures because CNTs spontaneously entangle by strong Van der Waals interaction and form complex networks from the nanometer to the millimeter length-scales due to their one-dimensional structure and exceptionally high aspect ratios. The structural hierarchy of the CNT films is shown by a series of SEM images taken at increasing magnifications (Fig. 1c), i.e. at the lowest magnification, a “superhighway” network of large CNT bundles; at higher magnification, mid-sized bundles with visible internal structure; and at highest magnification, a random network of fine CNT bundles. We would like to note that the structural hierarchy, such as the porosity of the fine network structure and tortuosity of the dendritic-like structure governs the electrical conductivity. Although referenced identically by the name, “CNTs,” their structures vary widely in such ways as length, diameter, crystallinity, and wall number [18,19,20]. This diversity makes CNTs an excellent test-bed material for this work because of the unique opportunity to fabricate an assortment of CNT films with different structures and properties. Here, we chose seven commercial CNTs (eDips, SG, Tuball, JEIO, Knano, Nanocyl, Cnano) that encompass a wide range of structures with wall numbers varying from 1 to 8.9, diameters varying from 1.8 to 15 nm, crystallinities (estimated by the Raman G-band/D-band ratio) varying from 0.5 to 59, and lengths varying from less than 1 μm to more than 100 μm (Supplementary Table S1).
Our approach to predict properties of tangible materials by virtual experiments is based on training multiple GANs from real SEM images to make possible the creation of fake images of CNT films at various scales and compositions. Generation of fake images by a GAN includes the effects of dispersion, filtration, and SEM observation executed on the computer. Each GAN is made from two ANNs, i.e., a generator and a discriminator, that are simultaneously trained by an adversarial learning process. The generator generates fake SEM images of CNT films, and the discriminator differentiates fake from real [11,23].
The training protocol of the GAN was decided by the balance among conflicting criteria. First As images on the order of 10,000s are required for each training point, cropped real SEM images were used to increase image number. In addition, the pixel resolution of images must be minimized to reduce memory usage and enable learning convergence within a reasonable time. Furthermore, the pixel resolution must be sufficiently high to capture the characteristic structural features of the CNT film for accurate prediction of properties. Taking these factors into account, for 17 CNT films (seven types of CNTs and ten mixtures of CNTs) we took approximately 300 SEM images (960x1280 pixels) at a fixed magnification and divided them into 64x64 pixels, rotated 90 degrees for four times, and thus obtained 17 sets of ~12,000 training data. For each of the 17 CNT films, we implemented 50,000 iteration steps of the adversarial learning process and snapshots (Supplementary Figure S2) at different iteration levels show how the fake image of the generator evolved from random noise to structures that resemble the training data. This process was repeated to train four GANs at the four different SEM magnifications (x2k, x20k, x50k and x100k). The potential of the multi-scale GANs framework was shown by a side-by-side comparison of the (Fig. 2a) fake and real images of the 28 CNT films (7 types of CNTs at 4 different scales). Although the real CNT film images showed a wide variation in structure exhibiting the hierarchy as well as the difference in the CNT structures, exceptional similarity can be seen in all 28 pairs of fake and real images. This demonstrated the versatility of GANs to process the wide range of structures of these tangible materials.
Image analysis was implemented to quantitively assess the similarity between the fake and real CNT film images. Two descriptors (Fig. 2b), i.e., the width of CNT bundle structure and void diameter were chosen to characterize the structure of the CNT films. Specifically, approximately 200 real and fake images of CNT films for each of the 21 cases (7 types and 3 scales) [18] were binarized (Fig. 2b) where white/black pixels represent CNTs/voids, respectively. The frequency histograms of the width of CNTs and void diameter were calculated from the binary images, and two examples (Cnano at x50k, JEIO at x20k) are shown in Fig. 2c. While the frequency histograms differed significantly between Cnano at 50k and JEIO at x20k, the agreement between the fake and real images was excellent for both cases, which illustrates the ability of a GAN to reproduce highly diverse structures among CNT films. Furthermore, correlation plots (Fig. 2d) between the fake and real mean values for the width of CNTs and void diameter, calculated from the histograms, provided correlation coefficients of 0.78 and 0.84, respectively, demonstrating the potential of GAN to accurately create fake images of various CNT films.
The properties of the CNT film were predicted by a structure-property ANN, however our approach differs substantially from previous efforts, because training and prediction processes were entirely accomplished on fake images created by GANs, and fake images representing different scales were tiled and merged into one to provide information about the hierarchal structure.(Fig. 2e) By this approach, computational prediction of properties of tangible materials possessing structural hierarchy was made possible. Our structure-property ANN was trained to predict the electrical conductivity and specific surface area of CNT films. Specifically, the training was carried out on the 17 CNT films (seven types and ten mixtures of CNTs), and for each training point 256 images (128x128 pixels) were created from each of the four GANs representing different magnifications, divided into four 64x64 pixels, rotated 90 degrees four times, providing 4000 images, and combination of tiling further increased the dataset to 12,000, of which 90% was used for training and 10% for validation. Once trained, the values of 256 predictions were averaged to provide the properties of a specific film.
Predicted versus experimental values of the electrical conductivity and specific surface area (Fig. 2f), for the validation and test sets, spanned a wide range, with prediction reliabilities (R2) of 0.99/0.85 for electrical conductivity and 0.99/0.42 for specific surface area, respectively. These results demonstrated the potential of our deep-learning based-computational framework to implement virtual experiments. Overall, this level of accuracy was sufficient to predict the properties of the CNT films to study relationship among structures and properties, as demonstrated later. It should be mentioned here that previous researches have elucidated that the electrical conductivity/specific surface of CNTs are strongly correlated to their length, crystallinity, and wall number [19,20], however the resolution of SEM is unable to resolve all of these CNT features even at the highest resolution, and thus not directly included here all. The influence of these undetected important structures of individual CNTs on properties must have been reflected indirectly through the difference in the network structure and hierarchy of the CNT films. The significance of using the “tiling” technique (Fig. 2f) for accurate property prediction is demonstrated by the comparison of the prediction reliabilities of the electrical conductivity from “tiled” multiple-magnification images against that of a single magnification for the 17 CNT films (Fig. 2g). Whereas the reliabilities show a large variation among the films and magnifications, the prediction reliabilities based on tiled images were consistently high.
The potential of predicting properties of tangible materials by virtual experiments cannot be overstated. To highlight this aspect, we predicted the electrical conductivities and specific surface areas of 1865 CNT films composed of diverse compositions (Supplementary Table S3), and plotted the results into an Ashby map of surface area and electrical conductivity (Fig. 3a). We note that the 1865 virtual experiments were executed independently. However, plots in the Ashby map shows a smooth transitions in color, which means continuous changes in composition with property, thus indicating that our naïve deep learning framework was capable of capturing the property trends on material the composition. We also stress that constructing this map experimentally would require years, however, in our case, once trained, these virtual experiments were completed within an hour. This speed underscores the great advantage of deep learning.
The results of the 1865 virtual experiments can serve as a database for research on tangible materials, analogous to databases of molecules and solids used in cheminformatics. For this, we present several examples. First, the predicted properties of CNT films formed a triangular domain that covers a very wide region within the Ashby map. This originates from the wide structural variation of the CNTs. Moreover, an inverse relationship between the specific surface area and the electrical conductivity was observed, which means no single CNT film possesses both the highest levels of surface area and electrical conductivity.
Second, the plots in the Ashby map were recolored (Fig. 3b) to show the weighted average wall-number of the CNT films, from which a wall-number phase diagram (Fig. 3c) was constructed. The phase diagram showed that both properties rapidly decrease with increase in wall number, implying wall number as a crucial structural parameter of CNT films. Films exhibiting the highest conductivities and specific surface areas were both composed from single-walled carbon nanotubes (SWNTs), but from different types, one with higher crystallinity and smaller diameter giving higher conductivity, and the other with lower crystallinity and larger diameter. The former/latter SWNTs formed a tightly/loosely packed bundle resulting in low/high specific surface areas. This is the origin of the inverse relationship between electrical conductivity and specific surface area. Moreover, considerable overlap among the wall-number domains was observed, e.g., the upper boundary of the double-walled carbon nanotube (DWNT) is nearly equivalent to that of the SWNTs. This means that the influence of a small addition of multi-walled carbon nanotubes (MWNTs) in a SWNT film is negligible and the properties are dominantly determined by the SWNT network.
Third, the database is not only useful for science but is also invaluable to develop practical applications. Frequently, the mission of developing real applications is to pursue the most cost-effective mixture of materials while satisfying the target properties. Currently, highly functional SWNTs are hundreds of times more expensive than MWNTs, and to examine economical solutions, we defined (Supplementary Method 1) and calculated the economic performance of the CNT films (larger value = more cost-effective), and plotted them against the electrical conductivity (Fig. 3d) and specific surface area (Fig. 3e), respectively. Both figures clearly show that an increase in properties result in a drop in economic performance. The CNT films at the upper boundary (highlighted by yellow stars) represents the economic Pareto optimal solutions, i.e., the most economical CNT film at that specific property. The compositions of the Pareto optimal solutions vary with electrical conductivity level (Fig. 3f). For example, it changes from large diameter MWNT mixtures (Cnano-Nanocyl (-10 S/cm), Cnano-Knano (-30 S/cm)), to medium diameter, fewer walled mixtures (Tuball-Knano (-120 S/cm)), to smaller diameter, longer length mixtures (Tuball-JEIO (-240 S/cm)), to small diameter, highly crystalline mixtures (eDips-Tuball (-330 S/cm), eDips (-360 S/cm)). In contrast, the compositions of Pareto optimal solutions follow a different course for specific surface area (Fig. 3g) as it begins from large diameter, MWNT mixtures (Cnano-Nanocyl (-240 m2/g), JEIO-Knano (-300 m2/g)), to large diameter, few-wall CNTs (JEIO-Nanocyl (-580 m2/g), JEIO (-660 m2/g)), to longer, SWNT-rich mixtures (SG-JEIO (-970 m2/g), SG (-1020 m2/g)). The compositions of optimal solutions differ for electrical conductivity and specific surface area. These predicted trends are valuable in developing real applications and would be difficult to obtain experimentally.
The database can be used to seek the compositions of CNT films that possess an intended property, and this ability would open opportunities for the inverse design of applications. For example, CNTs films are known to be well-suited for supercapacitors electrodes due to their high surface area and electrical conductivity [21,22]. From the Ashby map we selected several CNT films with varying and targeted properties (Supplementary Figure 4) and constructed two-electrode electric double-layer capacitor (EDLC) cells using a H2SO4 electrolyte. The galvanic discharge curves and impedance spectroscopies (Supplementary Figure 5) were measured from which the energy densities and relaxation time constants (a measure of the speed of operation) were calculated and plotted against the specific surface area (Fig. 4a) and electrical conductivity (Fig. 4b). The specific surface area/electrical conductivity showed a strong and monotonical correlation with energy/density/relaxation time constant. These results mean one can determine the mixture of CNTs for EDLC cell electrodes that meets a target energy density and relaxation time constant, and by combing the economic Pareto optimal solutions, even can address to the most economical electrode.
In summary, we proposed a general deep learning based computational framework that predict properties by using GAN to create fake structural images (SEM) of materials (CNT films) with hierarchal, non-definable, and non-periodic structures. Our approach is not only limited to CNT films but is readily extendable to other tangible material provided structural information is experimentally available and the properties have enough correlation with the structure, and thus we believe our approach represents an important step forward in expanding the scope of materials artificial intelligence can be applied.