聯合多模型提升影像品質：應用於大腦DTI與海馬迴分割

Joint Models for Improving Image Quality: Application to Brain DTI and Hippocampal Segmentation

許豐育(Feng-Yu Hsu)

指導教授 ： 陳志宏

國立臺灣大學/電機資訊學院/生醫電子與資訊學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202200419

摘要

對於研究分析和臨床診斷的準確性，取得具有高分辨率（High Resolution, HR）的核磁造影（Magnetic Resonance Imaging, MRI）圖像非常重要，其提供了重要的結構紋理訊息，有助於早期診斷和後續分析。但是實際上，由於硬件設備、成像時間、信噪比（Signal-to-Noise Ratio, SNR）和運動假影等因素，要獲得高品質的MR圖像相對困難，因此提升影像品質的需求相應而生。
過去已有許多超解析度（Super-Resolution, SR）與去噪(Denoise)相關的技術用於改善MR圖像品質，然而其中透過插值的SR方法，雖然能快速而簡單的執行超解析度，卻時常過度平滑影像，並且通過濾波方式的去噪方式也常造成影像的重要細節丟失。而近年，基於深度學習（Deep Learning, DL）的方法由於更好的模型表達能力，在SR與Denoise問題都達到更加優越的效能。然而許多網路未充分利用影像的先驗知識，以更複雜的網路改進效能，導致可能訓練困難或硬體需求增加。因此我們將聚焦於提升影像的解析度與SNR上，首先將提出的SR模型方法應用在優化海馬迴子區分割上，再結合去噪模型應用於擴散張量造影(Diffusion Tensor Imaging, DTI)，改善DTI重建分析以驗證去噪與提升解析度的效果。
首先，我們提出通道分裂邊緣導引殘差網絡（NLCSERN）用於提升人腦T1影像解析度，透過引進影像邊緣以及欠採樣影像，讓模型獲取更多種與輸入相關的影像特徵，並使用三種主要架構改進，在參數量下降約四十七萬的情況下，讓模型更好地學習這些特徵，最終在人腦T1影像的峰值信噪比與結構相似度上，從35.834 dB/0.9148提升至36.45 dB/0.9228。在海馬迴分割方面，低解析度分割結果顯示，在26個子區域中，有22個子區與高解析度的分割結果呈現顯著差異。透過SR模型提升解析度後，我們有效地使其中14個子區域與高解析度的分割結果變為非顯著差異。
我們也收取了DTI影像應用於實驗室已開發的去噪模型(MDNet)與NLCSERN。前者提升B-null影像至少兩倍以上SNR，使FA map平均值與標準差更接近高SNR影像(低SNR影像: 0.1074±0.0977, 去噪影像: 0.0846±0.083與高SNR影像0.0832±0.0865)，並且低SNR影像去噪過後，與高SNR影像的神經角度差變得更小，如在external capsule平均角度差從12.57下降至8.23。
NLCSERN則使B-null影像強度分布更接近高解析度影像，其中平均強度的誤差下降63% ，FA map也經由line profile證實許多原本因partial volume未能看出的神經走向，經由提升解析度而重建出來，並且結構如external capsule的神經角度差從16.13度下降至7.81度。
最後我們結合了兩個模型，同時學習去除DTI影像噪音與提升其解析度，重建出SNR兩倍提升且解析度亦兩倍提升的結果，使得在總體造影時間能大大縮小，並且使其在神經追蹤重建後角度差有所下降，如在Capsule從16.99度下降至9.45度。有了我們的聯合去噪與超解析度模型，將能為現代醫療研究提供所需的高品質的人體影像，以利後續相關的精準醫療與研究使用。

關鍵字

去噪 ； 超解析度 ； 影像分割 ； 深度學習 ； 卷積神經網路

並列摘要

To enhance the accuracy of research analysis and clinical diagnosis, High Resolution (HR) Magnetic Resonance Imaging (MRI) images are crucial since they provide sufficient structural texture information for early diagnosis and follow-up analysis. However, due to limitations including hardware equipment, imaging time, Signal-to-Noise Ratio (SNR), and motion artifacts, the acquisition of high-quality MR images is relatively non-efficient. Therefore, the need to improve image quality correspondingly rises.
Throughout the years, numerous super-resolution (SR) and denoise (Denoise) related technologies have been used to improve MR image quality. However, although SR methods through interpolation are fast and simple implementations of super-resolution, they often over-smooth images. On the other hand, denoising methods through filtering often cause the loss of important structural details. Recently, Deep Learning (DL) based methods achieved superior performance than SR and denoise methods due to their better model expression capabilities. Nonetheless, multiple networks also improved their performance with more complex networks instead of making full use of the prior knowledge of images, resulting in possible training difficulties or increased hardware requirements. Therefore, our research focuses on improving the resolution and SNR of the images. First, our proposed SR model is applied to improving the segmentation of hippocampal subfields. Then, the model is combined with the denoising model to enhance Diffusion Tensor Imaging (DTI), and analysis of DTI reconstruction is used to validate the performance of denoising and SR.
First, we proposed the Non-Local Channel Split Edge-guided Residual Network (NLCSERN) to improve the resolution of human brain T1 images. The model obtained more input-related image features by introducing image edges and under-sampled images. An additional architecture improvement further enhanced the model’s learning ability while the number of parameters decreased by about 470,000. Finally, the super-resolved images' peak SNR and structural similarity increased from 35.834 dB/0.9148 to 36.45 dB/0.9228. For segmentation of hippocampal subfields, 22 out of 26 segmentation results of low-resolution images showed a significant difference between results of HR images. With our SR model, we efficiently made 14 subfields that showed significant differences become not significantly different.
We also established a DTI database to apply our denoising model (MDNet) and NLCSERN. The denoising model improved the SNR of B-null images by at least twice, making the FA map average or standard deviation closer to high SNR images (low SNR image: 0.1074±0.0977, denoising image: 0.0846±0.083 and high SNR image 0.0832±0.0865). The model also reduced the angle difference of nerve orientation with high SNR images, e.g., from 12.57 to 8.23 in the external capsule. On the other hand, NLCSERN made the intensity distribution of B-null images closer to high-resolution images, in which the average intensity error was reduced by 63%. Through the line profile results, we also confirmed in the FA map that many neural trends originally not visible due to the partial volume effect can be reconstructed by increasing the resolution. The nerve angle difference of the external capsule dropped from 16.13 to 7.81 for instance.
Finally, we combined the two models to remove DTI image noise for a 2x SNR boost while achieving 2x resolution. The resulting overall imaging time was shortened while the angle difference of nerve tracking significantly reduced, e.g., from 16.99 to 9.45 degrees in the external capsule. With our joint denoising and super-resolution model, we can provide high-quality human images for modern medical research to facilitate follow-up precision medical and research applications.

並列關鍵字

Denoising ； Super-Resolution ； Deep learning ； Convolution neural network ； segmentation

參考文獻

1. Zaretskaya, N., Fischl, B., Reuter, M., Renvall, V., Polimeni, J. R. (2018). Advantages of cortical surface reconstruction using submillimeter 7 T MEMPRAGE. Neuroimage, 165, 11-26.

2. Fan, L. Y., Lai, Y. M., Chen, T. F., Hsu, Y. C., Chen, P. Y., Huang, K. Z., ... Chiu, M. J. (2018). Diminution of context association memory structure in subjects with subjective cognitive decline. Human brain mapping, 39(6), 2549-2562.

3. Jyh-Horng Chen, et al. Nuclear Magnetic Resonance and Magnetic Resonance Imaging. P.3-16

4. Iglesias, J. E., Augustinack, J. C., Nguyen, K., Player, C. M., Player, A., Wright, M., ... Alzheimer's Disease Neuroimaging Initiative. (2015). A computational atlas of the hippocampal formation using ex vivo, ultra-high resolution MRI: application to adaptive segmentation of in vivo MRI. Neuroimage, 115, 117-137.

5. Frank, L. R. (2002). Characterization of anisotropy in high angular resolution diffusion‐weighted MRI. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 47(6), 1083-1099.