A Two-Stage Multi-Objective Enhancement for Fused Magnetic Resonance Image and Computed Tomography Brain Images

Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) are the imaging techniques for detection of Glioblastoma. However, a single imaging modality is never adequate to validate the presence of the tumor. Moreover, each of the imaging techniques represents a different characteristic of the brain. Therefore, experts have to analyze each of the images independently. This requires more expertise by doctors and delays the detection and diagnosis time. Multimodal Image Fusion is a process of generating image of high visual quality, by fusing different images. However, it introduces blocking effect, noise and artifacts in the fused image. Most of the enhancement techniques deal with contrast enhancement, however enhancing the image quality in terms of edges, entropy, peak signal to noise ratio is also significant. Contrast Limited Adaptive Histogram Equalization (CLAHE) is a widely used enhancement technique. The major drawback of the technique is that it only enhances the pixel intensities and also requires selection of operational parameters like clip limit, block size and distribution function. Particle Swarm Optimization (PSO) is an optimization technique used to choose the CLAHE parameters, based on a multi objective fitness function representing entropy and edge information of the image. The proposed technique provides improvement in visual quality of the Laplacian Pyramid fused MRI and CT images.


1-Introduction
Glioblastoma is the fastest growing Grade IV malignant tumors found in the brain with a survival time of less than a year after their detection [1]. The detection of these tumors has always been a challenge to doctors. Some of the reasons for delay in detection are due to failure to understand the early symptoms, lack of awareness, inadequate healthcare facilities like imaging, preliminary screening for the patients and expertise with doctors. An inability to distinguish tumors by doctors and experts is mostly due to incorrect imaging procedures, patient"s condition at the time of image acquisition and noise may lead to delayed prognosis. For this reason, medical imaging is paramount in detection, identification, grading and diagnosis of the Glioblastoma. Doctors recommend many imaging techniques for detection of Glioblastoma like Computed Tomography (CT), Magnetic Resonance Imaging (MRI) and its variants, Fluid Attenuated Inverse Recovery (FLAIR) and Positron Emission Tomography (PET) [2]. These images are acquired sequentially through different scanning machines at different times. Each of the modalities provide different information of the brain. For example, the CT image provides the structural information of the brain like bone structure, tissue symmetries, changes in tissue density and space occupying lesions [3]. It also shows changes made in the nearby skull region due to tumor extension and calcification of tumors. Conversely, CT images fail to indicate tumor borders and infiltration in the nearby regions. These can be easily visualized with MR (Magnetic Resonance) images, which provide structural and functional information of the brain along with high contrast and resolution for soft tissues like tumors or lesions [3].
FLAIR images are a special type of MRI, which is sensitive to changes at the periphery of the cerebral hemispheres. PET images assess the tumor growth and spread [4]. These multimodal images are the noninvasive ways to detect Glioblastoma. Based on the multimodal images, surgical resection or complete removal of tumor is made followed by radio chemotherapy [4]. Thus, a single imaging technique is never sufficient to confirm the presence or extent of the tumor [5]. The latest development in medical imaging is generation of a complete image, obtained by fusing the multimodal images to a single image [6]. This helps in early detection of tumors and requires lesser digital storage [7][8]. Some of the techniques for fusing the multimodal images are Discrete Wavelet Transform (DWT), Laplace Transform (LT), Contourlet Transform (CT) and Non-sub Sampled Contourlet transform (NSCT) [9]. The process of fusion begins with decomposing the multimodal images in to approximation coefficients (low frequency components of the image) and detailed coefficients (high frequency components) using the above techniques. These components are combined using different rules like averaging, summing, weighted summing, max-min or max-max fusion rules [10]. The fused coefficients are recomposed using inverse transforms to generate a fused image. Fig.1 shows the methodology of the fusion process using DWT [10]  The visual quality of the fused image is evaluated based on contrast, edge information and peak signal to noise ratio. Contrast is measured in terms of standard deviation of the image and is significant in fused images, because it helps to distinguish the healthy and tumorous cells. The edge information in the image represents the tumor boundary and also assists in locating the tumor. Noise exists at the time of image acquisition due to various reasons like patient position, scanner machines or in processing the MRI or CT machine. Peak signal to noise ratio indicates the signal to noise ratio in the fused image. The fusion process introduces blocking effect, noise and artifacts that greatly reduce the visual quality of the fused image. Fig.2 shows a fused CT and MR image obtained by DWT indicating blocking effect at the edges and corners [10]. The fused images with poor visual quality make it difficult for experts to interpret the tumor presence or its spread. Hence, there is a need for an enhancement technique to improve the visual quality and reduce blocking effect, noise and artifacts. Most of the enhancement techniques deal with improving image contrast, as it helps in differentiating the Glioblastoma and the normal cells. Nevertheless, the drawback with these techniques is that it only increases the dynamic range of the image, which is a function of pixel intensity alone. Tumors or any abnormality in the brain appears distinct, bright or light intensity in MR images. Thus, by varying the contrast, the normal and the tumorous cells can be differentiated easily. On the contrary, the high contrast images or low contrast MR image makes it challenging to differentiate the Glioblastoma [6]. The contrast enhancement is mostly a 2fold process, consisting of contrast stretch and tonal enhancement. The contrast stretch improves the brightness differences uniformly across the dynamic range of the image and tonal enhancement improve the brightness differences in different areas like dark, gray or bright regions in the image [6]. The paper deals with enhancement techniques for fused imagescontrast enhancement, improvement in entropy and structural information.

2-Enhancement Techniques
Generally, the contrast enhancement is achieved by the following techniques -Non-Linear Transfer function, Histogram based and Frequency Domain [11]. Among them, Histogram Equalization (HE) is the most popular The enhanced image shows improved characteristics like contrast, entropy, peak signal to noise ratio and edge information with respect to the original image. Although HE is a popular enhancement technique, it suffers from visual artifacts like intensity saturation and amplification due to large number of homogenous pixels. Moreover, the equalization is accomplished uniformly for all the pixels of the image, leading to enhanced global contrast. However, the lowest intensity pixels become less significant, thereby reducing the local contrast [12]. In order to overcome the drawbacks of HE, newer enhancement techniques are proposed for improved visual quality with the use of median filters, adaptive gamma correction and homomorphic filtering [13][14].
An Adaptive Histogram Equalization (AHE) technique is a block based adaptive method, which is capable of reducing the drawbacks of HE. It deals with the local contrast rather than global contrast. The local contrast is more significant in the detection of Glioblastoma. In this technique, histogram equalization is performed on sub-images or small and equal sized blocks obtained by splitting the image. The equalization is executed on every block independently and mapped to new intensity levels based on a transformation function. The new pixel values are solely based on the neighboring pixel characteristics. Then, bilinear interpolation is used to combine the blocks after equalization [14]. The major challenge with AHE is the selection of the block size and the transformation function, which are significant and greatly affect the quality of the enhanced image. However, AHE suffers from blocking effect, at the time of combining the blocks. In addition to blocking effect, over-amplification is also seen due to large homogenous regions of the image. Youlian Zhu et al have proposed an adaptive histogram equalization technique for CT images. A user defined parameter β, is suggested based on the gray level of the image. The entropy is used as an objective function to select the β adaptively [15].
A variant of AHE is the Contrast Limited Adaptive Histogram Equalization (CLAHE), proposed by K Zuierveld, is also a block-based contrast enhancement technique with focus on local contrast. Unlike AHE, CLAHE provides uniform equalization with clipping the excess portion of large peaks found after the histogram equalization, thereby avoiding over-amplification. The excess portion removed depends on a parameter called clip limit, which is a function of the dynamic range of the image and block size. CLAHE involves setting of three operational parametersclip limit, block size and distribution function, which must be selected before performing the image enhancement to achieve good contrast images, free from noise and artifacts [16]. Various histogram based enhancement techniques are compared and analyzed, CLAHE is observed to perform better for MRI brain Images [17].
The simplest technique of setting the operational parameters for CLAHE is by trial and error. However, this is time consuming, may deviate from the actual values and changes for every image. Some of the techniques used to set the parameters are based on textureness of the image, maximum curvature of entropy, Least Mean Square (LMS) algorithm, multi-objective optimization technique and fuzzy rules. Despite various techniques for contrast enhancement, CLAHE seems to provide good local contrast, however it largely fails to enhance the pixels with low gray level intensity. Moreover, there is no standard for finding the optimal clip limit for a specific region of interest in medical images. Generally, clip limit is proportional to the multiple of average height of the histogram, where the multiplication factor is user-defined and varies for different images, which is the major drawback of CLAHE. Therefore, there is a need to choose clip limit adaptively for every block of image without any user intervention [23].
The clip limit is also a function of dynamic range of the grayscale image, block size and slope of transformation function. Initially, the clip limit and block size is chosen empirically and then obtain the optimal values based on statistical parameters like entropy, peak signal to noise ratio or edge information. Yet clip limit may change depending on the type of images. This makes the enhancement process very extensive and time consuming [23]. Moreover, inaccurate selection of clip limit can cause over-amplification in CLAHE. Bilateral Filter and Median filters are used to overcome this drawback [24][25].
Optimization techniques provide a convenient way of determining the CLAHE parameters without any heuristics and compute them adaptively for every image block. Particle Swarm Optimization (PSO) -a population-based optimization technique proposed by Eberhart and Kennedy [27]. The motivation for PSO is from the biological social groupings of animals, which interact with each other to find food or save each other from predators. It uses swarm intelligence to solve any optimization problem. A swarm is a group of possible solutions also called particles, which provide a solution to the optimization problem. The performance of the optimization is evaluated based on a fitness function. Each particle of the swarm is identified with its velocity and position, which are updated through iterations. The search for best solution terminates at the end of the iterations or when the solution generates the highest fitness value [27]. Malik Braik and Alaa Sheta have implemented the PSO algorithm for enhancing general images [28].
The objective of this paper is to enhance the contrast of Multimodal and Multiresolution fused images, obtained by a fusing MRI and CT brain images. The fusion process induces blocking effect, noise and artifacts that greatly reduce the visual quality of the fused image. CLAHE is used to improve the contrast of the fused image. The structural information is of great importance in the fused MRI and CT images, as they indicate the periphery of tumor region. The PSO algorithm is used to select of the operational parameters of CLAHE automatically as well as enhance the image quality based on entropy and edge information. The operational parameters for them are set manually or empirically, however they change with images, making it tedious to set them manually every time.

3-Enhancement Technique for Fused MRI and CT Image
The research work includes preprocessing of MRI and CT images, image registration, and image fusion followed by image enhancement. More than 200 MRI and CT images containing Grade IV tumors -Glioblastoma is taken from www.Radiopedia.org for this work. There are quite a few databases available publicly for MRI images, but the challenge in our research is to get multimodal images for the same patient. Since these are acquired at different times and from different machines, they must be registered and preprocessed before fusing them. The Fig.3 shows the block diagram for the enhancement process.
The preprocessing stage deals with resizing the image to size 256x256 and converting them to gray scale. The images contain Gaussian and Rician noises, which need to be eliminated; Non-Local Means Filter is used in filtering the CT and MRI images [29].
The preprocessing is followed by image registrationmandatory step, where both the images are matched for size, orientation and scaling. Subsequently, the images are fused using Laplacian Pyramid [31], since it provides excellent contrast for fused images. The fusion process introduces blocking effect and noise in the fused image, thereby reducing the image quality as discussed in section 1 [30]. Consequently, the enhancement technique is required to enhance the fused image with minimum loss of original information. Since the focus is the tumor region, adaptive block-based enhancement technique like CLAHE is chosen. The operational parameters for CLAHE are block size, clip limit and distribution function. The proposed technique is executed for different block sizes like 2x2, 3x3, 4x4, 5x5, 7x7, 8x8 and 10x10. The 8x8 provided superior results in terms of contrast and structural information, hence chosen to be constant for the enhancement process. There is a trade-off for high contrast large block size is considered and for high edge information low block size is chosen. Similarly, the enhancement process was performed with the various distribution functions like Uniform, Rayleigh and Exponential distribution. The Uniform distribution performed better than the other distribution functions. Hence, uniform distribution is considered in the proposed enhancement process. The clip limit is initialized randomly in the range of 0 to 0.01, the PSO algorithm adaptively chooses the clip limit based on a fitness function. The proposed enhancement process is two-stage; firstly, the contrast of the MRI and CT fused image is enhanced by CLAHE algorithm, which increases the dynamic range of the image. Secondly, the multi-objective fitness function assists in choosing the clip limit that maximizes the entropy and edge information of the image. The Particle Swarm Optimization (PSO) algorithm is used to find the optimal clip limit. It begins with initialization of a group of 50 particles called as a swarm. In this work, the particles in the swarm represent the values for clip limit randomly chosen in the range of 0 to 0.01. The fitness function given by Eq. (1) helps to determine the best clip limit.
The cost function/fitness function is a product of entropy, sum of edge intensities and number of edge pixels. Since, multiple parameters are considered to measure the degree of enhancement; this function is called multi-objective function. Every particle i in the swarm is represented by two parameters velocity and position. For any particle "i", the position and velocity indicate its location in the swarm and fitness value respectively. These two parameters are initialized with some random values and is updated in every iteration using Eq. (2) and Eq. (3).
where v i (t) and x i (t) represent the velocity and position for an particle i and iteration t. Eq.(2) comprises of three componentsfirst component representing the initial velocity of the particle, the second component represents the particle"s decision based on its own experience and the third component indicates the particle"s decision based on swarm"s experience. In every iteration, the image is enhanced using CLAHE with the selected clip limit (each particle). The fitness function is computed using Eq. (1). This process is repeated for all the particles to get the best fit (clip limit). The clip limit that maximizes the fitness function can be accessed from the swarm based on its position and velocity and is represented as 'pbest' or p i (t). This denotes the best local solution for that iteration. The enhancement process is repeated for all the iterations to get "pbest" or p i (t) for each iteration. In case the "pbest" value in the current iteration is greater than the previous one, then the "pbest" is updated with a new "pbest" and "gbest", otherwise the "pbest" from previous iteration is retained as "pbest" and "gbest". The "gbest" or g(t) in Eq. (2) is the global solution for the enhancement process obtained at the end of all the iterations. When "pbest" appears equal "gbest" over a predefined number of iterations the enhancement process terminates.  A balance between "pbest" and "gbest" is achieved by inertia weight represented as w, c1 and c2 -the positive acceleration constants and r1 and r2 are random values in the range of [0,1] . The, an optimal value for the clip limit is obtained by PSO algorithm based on maximum value of the fitness function [32]. Fig. 2 shows the pseudo code for proposed CLAHE-Particle Swarm Optimization (CPSO) algorithm.

4-Experiment and Results
The CT and MRI images prior to enhancement are preprocessed, registered and fused using Laplacian Pyramid [31]. The dataset includes many variants for CT and MRI images -(i) images with contrast agent (ii) images without contrast agent and (iii) delayed images with contrast agent. The combination of CT and MRI T1 image, CT and MRI T2 and CT and MRI FLAIR images are used to generate the fused image. The proposed enhancement technique is tested on more than 200 MRI and CT fused images, containing the different types of Glioblastoma -Multicentric Glioblastoma, Multifocal Glioblastoma, Cystic Glioblastoma and Giant Cell Glioblastoma.     Although, the standard deviation and entropy is high for enhanced image obtained by CLAHE, the PSNR, SSIM, UIQI is lesser than those obtained with the proposed enhancement technique.This is because, the CLAHE deals with enhancing the dynamic range and increasing the high intensity pixels and further decreasing the low intensity pixels of the image, however ignores the entropy or structural information of the image. Moreover, the MSE value is minimum for enhanced image in the proposed technique, indicating more similiarity to the orginal image with minimum loss of information. The clip limit in CLAHE is kept constant at 0.01, but with the proposed technique the clip limit is chosen adaptively in the range of 0 to 0.01 for each block of size 8x8, and it is found to present good results. Table  3 shows the performance paramters for the 10 image Datasets.

5-Conclusion
A novel technique for enhancement of fused CT and MRI T1 weighted images is proposed. CLAHE is used to enhance the contrast of the fused image and the selection of operational parameters like clip limit, block size and distribution function is automated. The block size is chosen for the experiment as 8x8. The uniform distribution provides better results for the fused images. The clip limit is in range of 0-0.01, adaptively chosen by PSO algorithm. The proposed technique improves the visual quality. Further, tumors can be segmented and classified efficiently. Experiments show superior results for PSNR, SSIM, UIQI and MSE. This technique can be applied for enhancement of CT-PET images and MRI-PET images. There is an improvement in contrast and entropy for Laplacian Pyramid fused image and CPSO enhanced image by 2.55 % and 5.18% respectively. CPSO enhanced image indicates an huge increase in PSNR, SSIM and UIQI by 16.32%, 32% and 20.3% respectively. Also, the MSE has greatly reduced. The proposed technique performs better than CLAHE in terms of PSNR, SSIM, UIQI and MSE.