Employing Deep Networks for Image Processing on Small Research Datasets
Amil Dravid Glenbrook South High School, 4000 W. Lake Ave., Glenview, IL 60026
avdravid@gmail.com
Abstract: Deep neural networks have attracted considerable attention because of their state-of-the-art performance on a variety of image restoration tasks, including image completion, denoising, and segmen- tation. However, their record of performance is built upon extremely large datasets. In many cases (for example, electron microscopy), it is extremely labor intensive, if not impossible, to acquire tens of thou- sands of images for a single project. The present work shows the possi- bility of attaining high-accuracy image segmentation, isolating regions of interest, for small datasets of transmission electron micrographs by employing encoder-decoder neural networks and image augmentation.
Keywords: Transmission electron microscopy (TEM), segmentation, neural networks, data augmentation, accuracy
Introduction Transmission electron microscopy (TEM) is widely used
for resolving the ultrastructures of biological samples, materials, and hybrid soſt/hard systems. Recent advances in sample pres- ervation and microscope hardware have dramatically increased imaging throughput and have enabled access to higher statistical power for biological studies. As more samples can be examined within a fixed time, the challenge associated with data processing also increases. Image processing is becoming a significant com- ponent of modern microscopy projects, particularly as a required step in image analysis. Surprisingly, one of the most labor-inten- sive tasks in electron micrograph processing for biological sam- ples involves the separation of the various cellular organelles, such as the nucleus, mitochondria, Golgi, etc. In order to investi- gate the linkage between ultrastructure and higher-order cellu- lar function, the organelles must first be “selected” or segmented from the cellular background. For example, a common analysis to understand the connection between chromatin topology and gene transcription involves comparing EM images of nucleus texture from normal cells and cancer cells [1]. Usually, manual nucleus segmentation is important for
determining correct statistical relationships (for example, not impacted by imprecision and random fluctuations within the cellular membrane space surrounding the organelle). As modern microscopy offers easy access to large datasets, image acquisition is no longer the bottleneck for biological studies, but labor-intensive manual image segmentation is. Moreover, humans are prone to bias and inconsistencies, which might also render the downstream statistical analysis unreliable. Tere- fore, an accurate, automatic, and consistent computational tool for image segmentation should be useful to accelerate biologi- cal research. Tis article describes such an automation tool.
Automated Segmentation Image segmentation using neural networks. To meet this
end, machine learning can provide a potential solution for “label- ing” EM images by automating the segmentation process aſter observing a set of “ground truth” hand-processed examples.
18 doi:10.1017/S1551929518001311
“Labeling” is classifying the data, or in the case of this study, seg- menting the image. Tis approach has already found use in criti- cal applications as disparate as self-driving cars and segmenting diagnostic features in patients’ medical CT scans [2]. Supervised machine learning methods, such as deep learning, on the other hand, rely on inferring a common pattern from a set of “perfect” or “ideal” ground truth examples [3]. Tis would be “learning by example.” In the present work, a class of deep learning methods,
known as convolutional neural networks (CNNs), is used for image segmentation. A CNN contains a large set of matrices that are iteratively convolved with the input data, typically an image. A convolution is a specific matrix operation that can reduce an image to features of interest. Initially, the convolu- tional matrices in a CNN are populated by random numbers, such that input images are randomly transformed and distorted. By comparing the output of the network against the desired output, values inside the network can be optimized using the chain rule, a calculus technique for optimization. Minimizing the error at the end of the process using a differentiable loss function results in a set of parameters that can perform a com- plex task, such as image segmentation. Optimizing the CNN with training data. A CNN can have
millions or tens of millions of individual parameters that must be optimized, or tuned, using input/output pairs, which are the labeled data pairs that the model learns from. A lack of labeled data can result in overfitting the data, when the algorithm becomes overly focused on details that are specific to the small set of images in the training. In all cases, the generalizability (or the quality of predictions on unseen data) of a CNN is directly dependent on the overall size and quality of the training data [4]. Most computer vision tasks leveraging CNNs require thousands to millions of labeled data pairs, as in the case of PASCAL VOC 2011 [5] and ImageNet [6]. For biological EM, analyzing and processing the raw data oſten involves expert knowledge and a significant amount of time, so hand-labeling tens of thousands of images as training data is likely prohibitive. Tus, although the idea of using CNN for image segmentation has been around for several years, implementation of it remains challenging. Smaller training sets. Many efforts have been made toward
generalizing CNNs with a small training set. Among them, data augmentation has been effective for several types of biological samples [7]. In data augmentation, a small set of input/output pairs are artificially altered through simple geometric transfor- mations to create new entries. Tese transformations include rotations, scaling, translations, and elastic deformations, among others. In addition to data augmentation, smaller mod- els with fewer parameters based on encoder-decoder structures, such as the U-Net CNN architecture, also have been successful in segmenting biological samples from differential interference
www.microscopy-today.com • 2019 January
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52
