Classification of iPSC-Derived Cultures Using Convolutional Neural Networks to Identify Single Differentiated Neurons for Isolation or Measurement

doi:10.21203/rs.3.rs-4849357/v1

Download PDF

Research Article

Classification of iPSC-Derived Cultures Using Convolutional Neural Networks to Identify Single Differentiated Neurons for Isolation or Measurement

https://doi.org/10.21203/rs.3.rs-4849357/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Understanding neurodegenerative disease pathology requires a close examination of neurons and their processes. However, image-based single-cell analyses of neurons often require laborious and time-consuming manual classification tasks. Here, we present a machine learning (ML) approach leveraging convolutional neural network (CNN) classifiers capable of accurately identifying various classes of neuronal images, including single neurons. We developed the Single Neuron Identification Model 20-Class (SNIM20) which was trained on a dataset of induced pluripotent stem cell (iPSC)-derived motor neurons, containing over 12,000 images from 20 distinct classes. SNIM20 is built in TensorFlow and trained on images of neurons differentiated from iPSC cultures that were stained for nuclei and microtubules. This classifier demonstrated high predictive accuracy (AUC = 0.99) for distinguishing single neurons. Additionally, the 2-stage training framework can be used more broadly for cellular classification tasks. A variation was successfully trained on images of a human osteosarcoma cell line (U2OS) for single-cell classification (AUC = 0.99). While this framework was primarily designed for single-cell microraft-based identification and capture, it also works with cells in standard plate formats. We additionally explore the impact of fluorescent channels and brightfield images, class groupings, and transfer learning on the quality of the classification. This framework can both assist in high throughput neuronal or cellular identification and be used to train a custom classifier for the user’s specific needs.

iPSCs

neurons

machine learning

CNN

microrafts

image classification

Research on neurodegenerative diseases such as Parkinson’s Disease (PD), Amyotrophic Lateral Sclerosis (ALS) and neuropathies centers on deep cellular and molecular investigations of neurons in their diseased state. Given the intricacies of studying single-cell neuronal phenotypes and challenges posed by complex neurite entanglement, researchers often rely on meticulous efforts in manual cellular annotation [1]. However, for high throughput pooled assays aimed at investigating the impact of genetic variants on neuronal morphology and organellar phenotypes, single-cell evaluation and nuclear identification are essential. One massively parallel approach employs microraft arrays containing induced pluripotent stem cell (iPSC)-derived single neurons enabling discrete phenotyping and single-cell isolation [2]. Each raft, which acts as a region-of-interest that can be captured, can be annotated to determine its contents. These contents may include a single neuron, multiple neurons, a dead cell, an empty raft etc. Here, we outline a pipeline and present pretrained machine learning (ML) models that enable rapid classification and automated annotation of microscopic images.

The implementation of artificial neural networks (ANNs) for cell classification has been simplified by the availability of manually annotated datasets containing biological and medical images, and platforms like the Allen Cell Explorer [3, 4]. However, it remains a challenge to identify microscopic images of individual neurons, since cultures contain a heterogeneous mixture of cell types (neurons, undifferentiated cells, dead cells). Additionally, under normal cell culture conditions, neurons form complex networks and clusters that facilitate neuronal growth but make image segmentation difficult. While there are image-based algorithms and machine learning tools that enable object detection and segmentation of clusters of neurons, the task of single neuron detection remains poorly addressed [5–10]. ANNs make an ideal choice to overcome limitations in the existing tools and have been implemented in similar contexts [11–15].

Here, we built “single neuron imaging model 20-class” (SNIM20), a convolutional neural network (CNN) that automates manual annotation of images in a high throughput setting. We apply and share a set of single iPSC-derived human motor neuron images and ANNs (see Methods) to train a neuronal classifier and present a pipeline that can enable others to quickly train custom ANNs for regional annotations. SNIM20 can identify and classify images of single iPSC-derived motor neurons with high accuracy, which can then be taken into a variety of downstream applications, such as organelle quantification within the cell or the distal neurite. We conducted a comparative analysis of the SNIM20 classifier against a variety of related classifiers that varied in their fluorescent wavelengths, class composition, and model architecture. Moreover, we also applied a similar CNN-based architecture to a non-neuronal cell type (U2OS) to evaluate the model’s performance and versatility across different experimental paradigms. This approach highlights the adaptable nature of image-based CNN classifiers for annotation of varied cellular images.

Overview of the Imaging and Training Pipeline

To enable pooled, image-based functional genomics pipelines, we seek to image motor neurons in a high-throughput manner. We differentiate iPS cells into motor neurons (Fig. S1) and then seed them onto a microraft surface. On the day of the experiment, we image the differentiated neuronal cultures to visualize the nucleus and the tubulin cytoskeleton (Fig. 1a). From here, the system is adaptable to different situations. Training images are hand-annotated using FIVTools (an in-house software which enables multi-user image annotation) (Fig. 1b, Fig. S2), and are exported to sub-folders named with the class label. Next, the system independently trains several CNN classifiers, and then the user selects the best model, based on its accuracy, as the initial model (designated as ‘v1’ in Fig. 1a). The second training round starts by using the ‘v1’ classifier for automated classification of an unseen/unlabeled set of images. Images with high model confidence are then exported. Since the ‘v1’ classifier has minimal training, it can make mistakes, necessitating some manual re-sorting prior to a new round of training and selecting a new ‘v2’ classifier which is much more accurate (Fig. 1c). This process allows for a relatively small initial training set, as the v1 classifier enables us to get a much larger training set which is used for subsequent training, resulting in a highly tuned classifier.

Differentiated Neurons and Class Definitions

Experiments with differentiated neurons on rafts led to many potentially segregable differences between images (Fig. 2a and Fig. S3). We broadly classified the training set images into four groupings: “No nuclei,” “One nuclei,” “Two nuclei,” and “Multiple nuclei” (three or more). We focused on identifying high-quality single and double neurons with distinct nuclei and neurites. These groupings guided our selection of rafts for isolation and downstream genotyping – owing to our interest in single-cell phenotypic analyses. However, due to considerable variation within the "One nuclei” grouping, especially among neurons with very short or disconnected neurites, as well as those with poor focus, they were later categorized into distinct classes. We also introduced multiple classes within the “Two nuclei” grouping (Fig. 2b, Fig. S4). In total, there were 12,689 images in the training set (Table S1). Similar to human annotators, our CNN classifiers fallibly discriminated between the class “2 Nuclei 1 Soma” and these one-neuron classes: “1 Cell and Disconnected Neurite,” “1 Neuron,” and “1 Neuron Poor Focus.” (Fig. 2c). For our experimental purposes, achieving the aforementioned class distinctions was vital. Confusion between one and two neurons per raft could lead to misinterpretations of downstream molecular results derived from sequencing. Figure 2b shows the four groupings and demonstrates how each class trained in the creation of SNIM20 classifier fits within them. After the training, SNIM20 consisted of varying filter sizes within its convolutional layers to capture diverse image features at different scales (Fig. 2d). While a different biological question would have different classes, the details of the classes are presented to show the possible subtlety that can be captured by these classifiers. Moreover, the architecture shown is the one that worked best, but additional optimization or a different training set would result in variations of the parameters and architecture that are likely to perform similarly.

Neuronal Classifier Performance

The training and testing of SNIM20 consisted of 12,689 images across the 20 classes with a train-test ratio of 80:20. We observed that SNIM20 accurately distinguished single neurons ("1 Neuron”) from other classes (AUC = 0.99) (Fig. 3a). In that same set of images, SNIM20 showed high sensitivity and precision in identifying “1 Neuron” images with a max F1 score of 0.92 (Fig. 3b). We evaluated the classifier’s performance using a confusion matrix, which further highlighted its ability to call the “1 Neuron” class with few errors, although sometimes it was confused with the class “1 Neuron Very Short,”. Overall, the classifier demonstrated a multiclass accuracy of 73.3% (Fig. 3c). SNIM20 training history shows that by epoch 40, the best weights were found, and the accuracy was maintained across 220 epochs (Fig. S5). We next took the prediction score for each of the 20 classes on each of the cells and generated a UMAP representation of the dataset (Fig. 3d). Clusters emerged, reflecting the true labels (although several individual images appeared incongruous within the grouping).

We performed further validation to assess the accuracy of SNIM20’s predictions in identifying the single neuron class. This involved evaluating SNIM20’s prediction on an independent hand-curated set of 52 distinctly single-neuron images, which were not part of the original tranche of data. The classifier scored an accuracy of 81% in correctly predicting raft images within the “1 Neuron” class (Fig. 3e). 12% (6) of the images were classified as "2 Nuclei 1 Soma" and 8% (4) of the images were unscored. The performance of SNIM20, trained exclusively with neurons on rafts, was further validated in a non-raft setting, i.e. out-of-context. On a 96-well plate, SNIM20 identified the neuron or neuron clusters stained for the microtubule-associated marker MAP2 and nuclei (Fig. 3f). The classifier’s ability to accurately detect many single- and double-neuron ‘regions’ shows its flexible applicability on images acquired from a different staining and non-microraft plate, even though it was not trained on it.

For comparison, we also evaluated an alternative model, SNIM16, which consisted of 16 classes (Fig. S6). SNIM16 exhibited relatively high distinguishability of “1 Neuron” class images when compared to “1 Cell Tub no Neurite” and “2 Nuc + 1 Neuron” classes, with an AUC of 0.96 and an F1 score of 0.90. This shows that SNIM20 effectively captures subtle features of single neuron images and illustrates how the expansion of classes impacts a classifier’s performance. Overall, a CNN classifier (such as SNIM20) can effectively annotate regions for a variety of classes and has particularly good metrics for identifying single neurons, even in contexts in which it was not trained.

Nuclei, Tubulin, and Brightfield Images can each be used to Classify Neurons

Next, we sought to study the relative importance of a set of markers (in different wavelengths) on the ability to create an effective 20-class model. To achieve this, we trained new models on a set of 11,308 images that contained different wavelengths (Fig. 4a) following the pipeline outlined in Fig. 1. Instead of solely evaluating the top classifier for each wavelength, we examined collective performance of all the classifiers associated with that wavelength to compare their relative performance. Classifiers trained with only one wavelength, either Hoechst (nuclei) or tubulin (neurites), had the highest validation accuracy. Those trained on Brightfield-only images performed relatively poorly, likely owing to the less specific information provided in that channel (Fig. 4b). We also trained classifiers on multi-channel images, so each classifier had access to a combination of two of the channels, or all three (Fig. 4c). Classifiers trained on images with both Hoechst and tubulin (similar to SNIM20) reached the highest overall accuracy, nearing 80%. Interestingly, adding brightfield to the other two channels did not increase performance. As expected, compared with the single-channel training images in Fig. 4b, the multi-channel images had an overall higher accuracy.

The intriguing possibility of classifying neurons without staining or marking the nuclei and neurites, prompted us to explore the brightfield-only classifiers in more detail. While the majority failed to successfully classify, the highest-performing brightfield-only model achieved over 50% multiclass accuracy. When comparing between the “1 Neuron” class and other classes, the brightfield-only classifier performed reasonably well (AUC = 0.83, max F1 = 0.46) (Fig. 4d). Similarly, another model trained on brightfield-only images also exhibited reasonable discrimination (AUC = 0.78, Fig. S7). It is likely that a larger training set could enhance the brightfield-only performance, making it suitable for specific applications where stains or markers are unavailable.

Coarse and Fine Class Definitions and Model Performance

We further investigated whether a coarser set of classes would change the abilities of the classifiers. We trained hundreds of individual classifiers by regrouping the same set of 11,308 images (multi-channel Hoechst and Tubulin) into varying numbers of classes. Each classifier had the same number of total training images, but the class definitions were different (Fig. 5a). For example, in the three-class versions, the “0 Empty” and “1 Neuron” classes were comprised of the same set of images used in SNIM20, and the remaining images were grouped into a third class. For each classifier, we evaluated the AUC of the ROC and the max F1 score comparing the “1 Neuron” class to all the other classes, treating the different classifiers equivalently (Fig. 5b). We did not evaluate the multiclass mean accuracy, as it expectedly decreases with an increasing number of classes. The high metrics attained with fewer classes is prominent not only in the 3-class versions but also in the 6- and 8-class versions. This is expected since each class now has more training data per class and the total number of images is not divided amongst as many subtly different classes. However, when examining the individual confusion matrices (Fig. 5c, d) it becomes apparent that even though the fraction of images confused with “1 Neuron” is lower, the absolute number of confused images is higher. This establishes that various granularities (coarse or fine) of class definitions can lead to reliable image-based classifiers. It also demonstrates that the knowledge provided by the classes should influence their definitions and groupings, and not necessarily the overall numbers of classes.

Custom CNN versus Existing Pre-Trained Transfer Learning

Our training pipeline includes the ability to randomly sample hyperparameters and neural architectures and replace the initial CNN component of the classifier with existing pretrained models. We aimed to optimize the model architecture to assess potential accuracy improvements through transfer learning. We trained hundreds of classifiers, each with the set of 11,308 images, and each with the original 20 classes from Fig. 2. All images contained both Hoechst and tubulin channels, so the results could be easily compared to the SNIM20 classifier. The transfer learning models use pretrained weights learned from different datasets [16]. We employed seven existing pretrained architectures (ResNet, EfficientNet, MobileNet, NASNet, Inception, Xception, and ConvNexT) as well as our custom CNNs with (AR) and without (NR) residuals. Generally, the use of residuals is known to improve the model performance [17]. Over two dozen different classifiers were trained with each architecture, and the multiclass validation accuracy varied widely between them (Fig. 6a, b). The results showed that the CNN AR classifier performed comparably to classifiers that incorporated Xception and NASNet transfer learning layers (medians around 50%, Fig. 6a). Some Inception-based classifiers performed very well while others struggled. MobileNet, ResNet, EfficientNet, and ConvNexT had trouble with the fluorescent cellular image data, perhaps because they were trained on mesoscale photographs instead of microscopic images. We also examined the distribution of different classifiers with these architectures (Fig. 6b). The custom CNN AR classifier, including SNIM20, outperformed others, achieving the highest accuracy in this set (AUC = 0.98 and F1 score = 0.84). Metrics for that CNN, and some of the best Inception, NASNet, and Xception classifiers produced high AUC and max F1 scores (Fig. S8).

Our next objective was to understand how the complexity of the model, measured by its size, affects its accuracy in predicting images from the “1 Neuron” class. CNNs with or without the residuals and NASNet Mobile classifiers both had the highest accuracy while maintaining the smallest size (Fig. 6c). While Xception had high accuracy (~ 57%), the trained model was on average five times larger. A small model size is advantageous for the lower computational costs associated with training and deploying the classifier. Also, the pretrained models are ‘generalists’, whereas our custom CNNs are specialized for single-cell neuronal classification. Hence, starting with NASNet or CNNs with residuals are both appropriate options for these cellular classification tasks.

Classification Task in Non-Neurons

The training pipeline and model architectures presented are not restricted to neurons on microrafts; they can also be used more broadly. Therefore, we sought to develop and test a classifier for a non-neuronal cell type, specifically, human osteosarcoma (U2OS) cells. We followed the pipeline outlined in Fig. 1 to identify regions that had single or multiple U2OS cells in images featuring nuclear and mitochondrial staining (Fig. 7a). In this paradigm, the total number of nuclei in each region (raft) was also classified to distinguish high-quality single-cell rafts (‘1 Cell’) from other cases using 12 classes total (Fig. 7b). After the refinement round with just 2,007 training images, we selected a “SCIM12” classifier that had high accuracy (89.1%) and minimal confusion between classes (Fig. 7c). When examining the model’s performance in differentiating “1 Cell” with all the other classes, the metrics were very high (AUC = 0.99, Max F1 = 0.95, Fig. S9). As in Fig. 4, we trained classifiers using solely Hoechst or mitochondrial channels, and then compared their performance with classifiers trained using images including both the channels (Fig. 7d). The models with both nuclear and mitochondrial channels showed superior performance, with accuracies approaching 90%. Therefore, the pipeline presented in this manuscript can provide high-quality annotations suitable for both neuronal and non-neuronal contexts.

Neuronal classification and segmentation tasks pose numerous challenges owing to inherent complexities of neuronal cultures, characterized by their tendency to cluster, thrive in high density, and exhibit complex morphologies. In high throughput or massively parallel approaches which require single cell capture and phenotypic analyses, such factors present significant hinderances for the identification and capture of single neurons. Furthermore, existing algorithmic frameworks fail in such tasks, partially owing to high variability across neuronal images. However, deep learning tools are well suited for such tasks, which we demonstrate in “SNIM20”, a 20-class CNN classifier for neuronal image annotation. The image dataset used in making of SNIM20 model can be accessed on huggingface.co (see Data Availability). Platforms like RaftSeq and other image-based functional genomics platforms require a rapid categorization of a large volume of images to identify single neurons. Developing classifiers such as SNIM20 and its variants can address these concerns. As an open-source and versatile tool, this framework presents a more accessible and customizable tool for semi-automated image classification [1, 18–19].

We opted to use a CNN framework because it presents a good tradeoff between quality and training size as it is commonly used in image recognition, segmentation, detection, and feature extraction [20]. To enhance the model's performance, we varied the CNN architecture and the number of fully connected dense layers. Fine-tuning the model involved randomizing hyperparameters and selecting the best classifier from a series. These optimizations allowed us to fine-tune the classifier while also mitigating overfitting, leading to favorable results for the identification of single neurons. SNIM20 enables automated annotation of neurons to recognize single neurons from microscopic images, streamlining the analytical process. Despite the training set for these classifiers originating from cells on microrafts, these classifiers can be trained and inferred on more common plate-based formats (Fig. S2). This broadens the utility of this classifier across diverse settings.

We have recently started initiatives for two new applications of the classification system. In one, we hand classify co-cultures of microglia and neurons. In this context, our objective is to categorize microglia based on their ‘neighborhood status’, discerning whether microglia are solitary or in proximity to neurons. In another application, we plan to examine adipocytes on a feeder layer of mesenchymal cells, where we will hand-annotate adipocytes based on the intensity of their respective lipid droplets. In both cases, a single expert annotator can then be virtually ‘copied’, so that the classifier can automatically give similar classifications in a manner only biased by the original training set, but not by different human labelers, as would be otherwise.

Another important future aim is to address neurite disentanglement directly. While the raft-based screening keeps the neurites separated, many contexts include highly overlapping and complex neurite arbors. The development of a disentanglement-focused model will enable users to analyze images featuring multiple neurons, even in absence of ‘rainbow’ stains like BrainBow or TetBow [21]. This approach would effectively address the limitations inherent in many current image segmentation methodologies [22–25]. With the expansive potential of a CNN-based classification system, we can harness it to advance the field of biomedical image analysis and examine individual cells and its organelles in a high throughput manner.

Human iPSCs Maintenance and Neuronal Differentiation

The neuronal cells were derived from the WTC-11 human induced pluripotent stem cell (iPSC), some with the addition of an inactive gRNA library for downstream experiments. The iPSCs were obtained from Genome Engineering and Stem Cell Center at Washington University in St. Louis, propagated in mTeSR plus media (100–0276, StemCell Technologies, Vancouver, Canada), and seeded onto 6-well plates coated with Matrigel (354277, Corning, Corning, USA), diluted 1:6 in DMEM/F12 media (11320033, Thermo Fisher, Waltham, USA). The iPSCs were maintained, expanded, and incubated at 37°C and 5% CO₂. The media was replaced every day and cells were passaged once they reached 80% confluency. Cells were rinsed with Dulbecco’s Phosphate-Buffered Saline (DPBS, 14190144, Thermo Fisher) and treated with 1mL ReLeSR (05872, StemCell Technologies) at room temperature for one minute. ReLeSR was then aspirated, and cells were incubated in a 37°C incubator for 5 minutes before being collected and split at a 1:10 dilution in a new plate.

Upon reaching 80% confluency, iPSCs were differentiated into motor neurons as per the protocol outlined by Du et al [26]. In brief, the natural developmental signaling factors were recapitulated with small molecules including Ascorbic acid (72132, StemCell technologies), CHIR99021 (72054, StemCell Technologies), DMH-1 (73634, StemCell Technologies), SB-431542 (72234, StemCell Technologies), Retinoic acid (R2625, Sigma, St. Louis, USA), Purmorphamine (72204, StemCell Technologies), ROCK inhibitor (Y-27632, 72304, StemCell technologies), and Compound E (73952, StemCell Technologies) using the protocol26. Additionally, the differentiation media also contained N-2 (17502001, Thermo Fisher) and B-27 (A3582801, Thermo Fisher) supplements. iPSCs underwent differentiation into motor neurons across four distinct stages, incorporating varying time and concentrations of the various small molecules (Fig. S1).

The induction of neuroepithelial cells was initiated by introducing stage 1 differentiation media composed of 50% neural basal (21103049, Thermo Fisher), 50% DMEM/F12, 0.5X N2, 0.5XxB2, 0.1mM Ascorbic acid, 1X Glutamax (35050061, Thermo Fisher), 1X 0.1% Penicillin/Streptomycin (15140122, Thermo Fisher), 2 µM DMH-1, 2 µM SB-431542, and 3 µM CHIR99021. On day 7, the cells were then transitioned to stage 2 media, introducing two additional small molecules, namely, 0.5 µM Purmorphamine and 0.1 µM Retinoic acid, while reducing CHIR99021’s concentration from 3 µM to 1 µM. This stage induces OLIG2 + expression, initiating the cells' motor neuron progenitor (MNP) identity. On day 4 of being on stage 2 media, about 1 million cells were frozen down per aliquot for later experiments (see below, raft plating and imaging).

Between each stage, cells were dissociated with TrypLE (354277, Gibco, Gaithersburg, MD, USA) and re-plated. Each time the cells underwent the differentiation process and required dissociation; they were treated with 10mM ROCK inhibitor (72304, StemCell Technologies) for 24 hours following the passage. For dissociation, cells were washed with DPBS, 1X TrypLE was introduced, and the vessel was incubated at 37°C for 3–5 minutes. The cells were then centrifuged at 200g for 5 minutes before resuspending in appropriate media. It is worth mentioning that while resuspending the cell pellet, several triturations were performed to break the cell clusters into single cells.

Raft Plating and Imaging Motor Neurons

MNPs were seeded on two different kinds of raft plates, a single reservoir and a quad reservoir, each containing 100x100µm microrafts (Cell MicroSystems, Inc., NC, USA). To prepare the plates for cell seeding, the plates were washed with 1 mL/well and 5 mL/well (for quad and single raft plates, respectively) of 1xPBS (10010023, Corning) and incubated at 37°C for 5 minutes. These washes were done three times to remove the glucose layer from the raft plates. The raft plates were then coated with 4µg ⁄mL Poly-d-lysine (PDL, P0899, Sigma) and incubated overnight at 37°C. The next day, the raft plates were rinsed three times with molecular grade water (46-000-CV, Corning) and coated with 5µg/mL mouse laminin (23017015, Fisher Scientific) for similar overnight incubation. Both PDL and Laminin solutions were made with molecular grade water.

MNPs frozen on stage 2 day 4 were thawed, centrifuged at 200g for five minutes, resuspended in 1mL stage 2 media, and subsequently plated on a Matrigel-coated 6-well plate. The cells were then grown for an additional 3 days on stage 2 media. On the 14th day since the start of the differentiation process, the cells were nucleofected with Cas9 plasmid and plated on stage 4 media consisting of 50% neural basal media, 50% DMEM/F12, 0.5X N2, 0.5X B27, 0.1mM Ascorbic acid, 1X Glutamax, 1X 0.1% Penicillin/Streptomycin, 0.1 µM Purmorphamine, and 0.5 µM Retinoic acid. Approximately 2 million cells were nucleofected in a large cuvette per reaction using Lonza’s P3 primary cell 4D-nucleofector kit (V4XP-3024, Lonza, Basel, Switzerland). The cells were allowed to recover from the transfection for 3 days while inducing MNX + motor neuronal cell fate. In this manuscript, Cas9 plasmid was not added, but this detail is included for more clarity on the overall timing.

On day 18 of differentiation, the cells were seeded on raft plates. For quad and single raft plates, 80,000 cells/well and 320,000 cells/well were seeded respectively. Cell counts for raft plating was performed using a hemocytometer (with higher accuracy than automated cell counters). The next day, media was changed to stage 5 media, which consisted of same small molecules as of stage 4 media but with the addition of 0.1 µM Compound E. Notably, the protocol mentions stage 3, an optional MNP expansion step, which we skip. Full media changes are conducted every day during stages 1, 2, and 4, and a half media change every other day when on stage 5.

Motor neurons were stained and imaged 48-hours after the introduction of stage 5 media. A master mix of the staining media was prepared, consisting of 4.05 µM Hoechst 33342 (H3570, Thermo Fisher), and 1 µM Tubulin Tracker Deep Red (T34076, Thermo Fisher). For staining, the media was changed by gently removing from the plates and adding 500µL/well and 2mL/well of stage 5 media to the quad and single plates, respectively. Following the application of the stain, the plates were then incubated at 37°C for 30 minutes, after which the stain media was removed and three washes with stage 5 media in the same volume as the stain media were performed. For imaging, the plates were positioned on a Cell Microsystems plate adapter and imaged on Molecular Device’s INCell 6500HS Confocal microscope at 20x with 0.45 NA. The live cell chamber was used, set to 5% CO₂ and 37°C. The imaging protocol included 40,000 rafts / 1,368 fields and 25,600 rafts / 960 fields in the single and quad plates, respectively, with 12% field of view overlap. Exposure times for Hoechst (405 nm), Tubulin Tracker Deep Red (642 nm), and brightfield all averaged less than 100ms.

Immunofluorescence to Confirm Motor Neuron Identity

To determine the iPSC's differentiation proficiency, cells were analyzed with immunofluorescent staining at different stages during the differentiation process. First, the cells were fixed with 4% paraformaldehyde (PFA, 158127, Sigma Aldrich) for 15 minutes and washed three times with 1X PBS. Next, cells were permeabilized using 0.5% Triton-X 100 (93443, Sigma Aldrich) for 15 minutes and blocked with 3% Bovine Serum Albumin (BSA, A9418-10, Sigma- Aldrich,) for 1 hour. Both primary and secondary antibodies were diluted with BSA, placed on the cells and incubated for 1 hour. The primary antibodies that were tested included 1:75 SOX1 (AF3369, R&D Systems), 1:500 Olig2 rIgG (AB9610, Sigma), 1:100 NKX2.2 mIgG (74.5A5, DSHB), 1:75 MNX1 mIgG (81.5C10, DSHB), 1:70 ISL1 mIgG (40.2D6, DSHB), 1:300 CHAT gIgG (AB144P, Sigma-Aldrich), and 1:100 MAP2 rIgG (MAB3418-25UG, Thermo Scientific). The secondary antibodies that were used included 1:1000 donkey anti-mouse (A-31571, Thermo Fisher), 1:1000 donkey anti-rabbit (ab150075, Thermo Fisher), and 1:200 donkey anti-goat (A-21432, Thermo Fisher). Through confocal microscopy (INCell Analyzer 6500), the cells were then imaged, and cell tracing and fluorescence intensity extraction was done using IN Carta Image Analysis Software (Molecular Devices).

Passaging and Maintenance of U20S Cells

In some experiments we utilized a human osteosarcoma cancer cell line, U2OS cells (HTB-96, ATCC). The U2OS cell line engineered to express doxycycline inducible Cas9 was acquired from GESC@MGI WashU. The U2OS cells were cultured in McCoy's 5A Modified Medium (16600082, Gibco) supplemented with 10% tetracycline-free fetal bovine serum (FBS, PS-FB3, Peak Bio, Pleasanton, CA, USA). Cells were grown in either T75 or T150 tissue culture flasks and were passaged at 90% confluency. Trypsinization consisted of placing 0.25% Trypsin-EDTA 1x (25200056, Gibco) on the cells for 5 minutes at 37°C, followed by collection of cell suspensions and centrifugation for 3 minutes at 1200 RPM.

U2OS Cell Imaging

The U2OS cells were plated on raft plates, either a single plate containing 40,000 microrafts or a quad plate containing 25,600 microrafts (Cell Microsystems). 48,000 U2OS cells were seeded in a single raft plate, while 12,000 U2OS cells were seeded per well in the quad plates. Cells were plated 48 hours prior to staining with 4.05 µM Hoechst, and 0.5 µM MitoTracker Green (M7514, Thermo Fisher) for 30 minutes at 37^oC. Like neuronal imaging, U2OS imaging consisted of placing the plates in a Cell MicroSystems plate adapter and imaging on the Molecular Devices INCell 6500HS Confocal microscope at 20x with 0.45 NA. The live cell chamber was used, set to 5% CO₂ and 37°C. In total, 40,000 and 25,600 fields were imaged in the single and quad plates respectively, with 12% field of view overlap. Exposure times for Hoechst (405 nm) and MitoTracker Green (488 nm) averaged < 100ms.

Training Image Annotation and Sorting

Some of the general details pertaining to image annotation via FIVTools are listed in and illustrated in Figure S2. To ensure comprehensive testing, a separate set of 11,308 images including all three wavelengths (brightfield, tubulin, and Hoechst) was selected, distinct from the 12,689 images utilized for training. The 16-bit multichannel images of single 20x fields-of-view were loaded directly into FIVTools. If the cells were plated on microrafts, then raft calibration was performed (which connected the plate coordinates to the raft coordinates). For 96-well plate experiments, nuclei segmentation was performed first, to identify the raft-sized regions around each nucleus. Image-display parameters were then adjusted to optimize the contrast for both nuclear and accessory channels. The neuronal classifiers were trained with nuclei in the blue channel and tubulin in the red channel. U2OS classifiers were trained with nuclei in the blue channel and mitochondria in the green channel. It is worth noting that these channels could be swapped as needed. Initially, half a dozen experienced scientists manually annotated each of the scanned raft images in a well by pressing a number key (which designated the class index, initially arbitrary), then clicking on a raft to assign a class label (see Fig. 2 for the list of labels). These labels could be checked using the “Check Annotation” feature in the program, enabling the user to quickly visualize groups of images from different classes. This functionality provided flexibility, allowing users to modify annotations at any point in time. Next, all the annotated regions were exported with the adjusted parameters, so that the images were placed in a sub-folder with the name of the class. These images were exported as RGB 8-bit per channel bitmaps with a width and height of 128x128 pixels. The number of images per class is listed in Table S1.

Machine Learning Training

We used Keras/Tensorflow 2.10 to train the CNNs. The workflow was executed in a Jupyter notebook inside of Visual Studio Code, using a python kernel in Anaconda. See “TF_NB_MainCells01_1.ipynb”, then navigate to the code section “Raft CNN – Classifiers” > “Directory Defines Classes” > “RGB 8-bit per Channel”. First, images were loaded into a TF dataset, cached, and prefetched (see Code Availability). Each classifier was generated using “neural architecture search”, by randomizing various parameters and then inferring on additional images. The following parameters were randomly assigned: CNN kernel size (2–4), CNN initial number of filters (18–36), CNN depth (max(1, log(image width, kernel size))), and filter increase rate (1.25 to 1.75). The number of dense layers after convolution started with a random size that was no more than half of the flattened size. In addition, the dense layer dropout was randomized, regardless of the presence of batch normalization in the dense layers or residuals in the CNN layers. Additionally, the decision to use single loss or multi loss for each class was also randomly turned on and off.

Images were resized, rescaled between 0 and 1, then fed through the convolutional layers or the transfer learning layers. Early stopping was used if the loss stopped decreasing, usually with a patience set at ten percent of the number of epochs, which was usually set to be around 200. Batch size was 64, and training:test split was 80:20. As soon as each training finished, the script exported several evaluation metrics, including the best loss and accuracy as well as inference on the full dataset. Then the thread was reset, a new random set of parameters was created, and the process was repeated. Over a hundred individual classifiers were automatically and independently trained in each round.

Most of the manuscript did not use transfer learning with pre-trained layers, but this was implemented for one specific experiment (Fig. 6). In that case, a variety of transfer layers used included Keras’s built in convnext.ConvNeXtTiny, efficientnet_v2.EfficientNetV2B0, inception_v3.InceptionV3, MobileNetV3Small, nasnet.NASNetMobile, ResNet50, and xception.Xception instead of the CNNs mentioned above.

Evaluation and Analysis

In addition to assessing the loss and the accuracy of each classifier, we also treated the fully inferred output as two distinct classes to calculate additional metrics. Given that both the classifiers, neuronal and U2OS, were designed to identify one ‘ideal’ class, we minimized the labels to represent the preferred class (1 Neuron or 1 Cell) vs all the other classes. Afterwards, we computed the AUC for sensitivity/specificity and the Max F1 score for precision/recall. Figures and results were visually represented using either the software Spotfire (Tibco, California) or with the Matplotlib library in Python.

Data Availability

All the 11,308 images used in the testing of SNIM20 can be found on huggingface.co https://huggingface.co/datasets/FIVE-MGI/SNIM20. The pretrained models created for this manuscript are also found at https://huggingface.co/FIVE-MGI/Single_Neuron_Identification.

Code Availability

Open access source code for FIVTools is available on GitLab (https://gitlab.com/buchserlab/fivetools). Python script for training and evaluating SNIM20 can be found on GitLab as well (https://gitlab.com/buchserlab/fivetools/-/tree/master/FIVE_Tools_Main/NeuronID).

Competing interests

The authors declare no competing interests.

Author Contribution

P.P., L.K.MA., M.G.W., K.P.G., J.E.W., C.L.K., and W.J.B. wrote the manuscript. W.J.B., P.P., J.E.W., and C.L.K. designed the experiments. Initial planning and ideas brought by W.J.B., P.P., J.E.W., and C.L.K. P.P., C.L.K., J.E.W., L.K.MA., M.G.W., K.P.G., U.K., S.N.E. performed the experiments. U.K., L.K.MA., W.J.B., designed the figures. P.P., L.K.MA., G.W.B., and W.J.B. analyzed the results. W.J.B. wrote some of the key software. All reviewed the paper and gave suggestions.

Acknowledgement

We would like to thank Jack Bramley, who started early versions of this in 2019. We would also like to thank Marianna Vakaki for setting up cell culture protocols involving neuronal differentiation. Thanks to Diana Grigore and Josh Milbrandt for their help with aspects of the project. We would also like to thank Jeffrey Milbrandt and Robi Mitra for their continued support. We would like to thank the McDonnell Genome Institute, specifically GESC@MGI. Finally, we would like to thank the Genetics and MGI administration and our maintenance and cleaning staff. Funding for this study was from the McDonnell Genome Institute.

Shapson-Coe, A. et al. A petavoxel fragment of human cerebral cortex reconstructed at nanoscale resolution. 2024; doi:10.1126/science.eadk4858.
Yenkin, A. L. et al. Pooled image-base screening of mitochondria with microraft isolation distinguishes pathogenic mitofusin 2 mutations. Communications Biology. 2022;doi:10.1038/s42003-022-04089-y.
Lecun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature vol. 2015; https://doi.org/10.1038/nature14539
Sunkin, S. M. et al. Allen Brain Atlas: an integrated spatio-temporal portal for exploring the central nervous system. 2013;doi:10.1093/nar/gks1042.
Gähwiler, B. H., Capogna, M., Debanne, D., McKinney, R. A. & Thompson, S. M. Organotypic slice cultures: a technique has come of age. Trends Neurosci. 1997;doi:10.1016/s0166-2236(97)01122-3.
Zhou, Z., Kuo, H.-C., Peng, H. & Long, F. DeepNeuron: an open deep learning toolbox for neuron tracing. Brain Inform. 2018; doi:10.1186/s40708-018-0081-2.
Quan, T. et al. neuroGPs-tree: automatic reconstruction of large-scale neuronal populations with dense neurites. Nat Methods. 2016;doi:10.1038/nmeth.3662.
Manubens-Gil, L. et al. BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets. Nature Methods. 2023;doi:10.1038/s41592-023-01848-5.
Tong, L., Langton, R., Glykys, J. & Baek, S. ANMAF: an automated neuronal morphology analysis framework using convolutional neural networks. Scientific Reports. 2021;doi:10.1038/s41598-021-87471-w.
Vecchi, J. T. et al. NeuriteNet: A convolutional neural network for assessing morphological parameters of neurite growth. J Neurosci Methods. 2021;doi:10.1016/j.jneumeth.2021.109349.
Fogo, G. M. et al. Machine learning-based classification of mitochondrial morphology in primary neurons and brain. Scientific Reports. 2021; doi:10.1038/s41598-021-84528-8.
Cetina, K., Buenaposada, J. M. & Baumela, L. Multi-class segmentation of neuronal structures in electron microscopy images. BMC Bioinformatics. 2018; doi:10.1186/s12859-018-2305-0.
Pratapa, A., Doron, M. & Caicedo, J. C. Image-based cell phenotyping with deep learning. Current Opinion in Chemical Biology. 2021; https://doi.org/10.1016/j.cbpa.2021.04.001
Moen, E., et al. Deep learning for cellular image analysis. Nat Methods. 2019; doi:10.1038/s41592-019-0403-1.
Verzat, C., Harley, J., Patani, R. & Luisier, R. Image-based deep learning reveals the responses of human motor neurons to stress and VCP-related ALS. Neuropathol Appl Neurobiol. 2022; doi:10.1111/nan.12770.
Han, D., Liu, Q. & Fan, W. A new image classification method using CNN transfer learning and web data augmentation. Expert Syst Appl. 2018; https://doi.org/10.1016/j.eswa.2017.11.028
Xia, H., Zhu, F., Li, H., Song, S. & Mou, X. IET Image Processing Special Section: Multidisciplinary advancement of imaging technologies: from medical diagnostics and genomics to cognitive machine vision, and artificial intelligence Combination of multi-scale and residual learning in deep CNN for image denoising. IET Image Process. 2020; https://doi.org/10.1049/iet-ipr.2020.1020
Gautier, M. K. & Ginsberg, S. D. A method for quantification of vesicular compartments within cells using 3D reconstructed confocal z-stacks: Comparison of ImageJ and Imaris to count early endosomes within basal forebrain cholinergic neurons. J Neurosci Methods. 2021;doi:10.1016/j.jneumeth.2020.109038.
Dickstein, D. L. et al. Automatic dendritic spine quantification from confocal data with Neurolucida 360. Curr. Protoc. Neurosci. 2016;doi:10.1002/cpns.16.
Sharma, N., Jain, V. & Mishra, A. An Analysis Of Convolutional Neural Networks For Image Classification. Procedia Comput Sci. 2018;https://doi.org/10.1016/j.procs.2018.05.198
Sakaguchi, R., Leiwe, M. N. & Imai, T. Bright multicolor labeling of neuronal circuits with fluorescent proteins and chemical doitags. Elife. 2018;doi:10.7554/eLife.40350.
Pani, G. et al. MorphoNeuroNet: An automated method for dense neurite network analysis. Cytometry Part A. 2014; https://doi.org/10.1002/cyto.a.22408
Berg, S. et al. ilastik: interactive machine learning for (bio)image analysis. Nat Methods. 2019;doi:10.1038/s41592-019-0582-9.
Hopkins, A. M., Wheeler, B., Staii, C., Kaplan, D. L. & Atherton, T. J. Semi-automatic quantification of neurite fasciculation in high-density neurite images by the neurite directional distribution analysis (NDDA). J Neurosci Methods. 2014;doi10.1016/j.jneumeth.2014.03.006.
Long, B. L. et al. GAIN: A graphical method to automatically analyze individual neurite outgrowth. J Neurosci Methods. 2017;doi:10.1016/j.jneumeth.2017.03.013.
Du, Z.-W. et al. Generation and expansion of highly pure motor neuron progenitors from human pluripotent stem cells. Nat Communications. 2015; doi:10.1038/ncomms7626.
Addenda

No competing interests reported.

Download PDF

Editorial decision: Revision requested
10 Oct, 2024
Reviews received at journal
29 Sep, 2024
Reviews received at journal
25 Sep, 2024
Reviewers agreed at journal
17 Sep, 2024
Reviewers agreed at journal
14 Sep, 2024
Reviewers invited by journal
04 Sep, 2024
Editor assigned by journal
16 Aug, 2024
Submission checks completed at journal
10 Aug, 2024
First submitted to journal
02 Aug, 2024

You are reading this latest preprint version

Classification of iPSC-Derived Cultures Using Convolutional Neural Networks to Identify Single Differentiated Neurons for Isolation or Measurement

Status:

Version 1

Abstract

Figures

Introduction

Results

Overview of the Imaging and Training Pipeline

Differentiated Neurons and Class Definitions

Neuronal Classifier Performance

Nuclei, Tubulin, and Brightfield Images can each be used to Classify Neurons

Coarse and Fine Class Definitions and Model Performance

Custom CNN versus Existing Pre-Trained Transfer Learning

Classification Task in Non-Neurons

Discussion

Methods

Human iPSCs Maintenance and Neuronal Differentiation

Raft Plating and Imaging Motor Neurons

Immunofluorescence to Confirm Motor Neuron Identity

Passaging and Maintenance of U20S Cells

U2OS Cell Imaging

Training Image Annotation and Sorting

Machine Learning Training

Evaluation and Analysis

Declarations

Data Availability

Code Availability

Competing interests

Author Contribution

Acknowledgement

References

Additional Declarations

Supplementary Files

Status:

Version 1