Recent Expansion of Deep Learning Reconstruction
Toshinori Hirai, MD, PhD
The societal impact of artificial intelligence (AI) has expanded rapidly in recent years, and AI has become a focus of intense research activity in the field of medicine. With regard to MRI, Deep Learning Reconstruction (DLR) is attracting a great deal of attention as a technology to improve SNR, and an increasing number of studies on the applications of DLR in MRI are being published. Given these trends, Kumamoto University has joined with the University of Bordeaux and Canon Medical Systems Corporation to conduct joint research on Advanced intelligent Clear-IQ Engine (AiCE) since 2018 and on Hybrid DLR since 2020, as well as ongoing research on Precise IQ Engine (PIQE) since 2022. This report presents an overview of these three research projects.
Advanced intelligent Clear-IQ Engine (AiCE)
AiCE is advanced noise reduction technology that allows high-SNR images to be created from low-SNR images. Deep Learning is employed to train the system using only the high-frequency noise components, excluding the low-frequency components that contribute to good contrast. This approach results in a substantial reduction in image noise and a marked improvement in SNR.
It is generally accepted that in order to improve image quality, the number of acquisitions must be increased. However, in a review conducted at our hospital to evaluate T2WI of the head, we found that applying AiCE to noisy images acquired with only 2 acquisitions allowed us to obtain high-quality images comparable to conventional images acquired with 5 acquisitions1. Similar results were observed for FLAIR and MPRAGE images1. For example, in high-resolution heavily T2WI 3D MR cisternography, the nerve tracts could be more clearly visualized than with conventional images2, and the depiction of fine blood vessels such as the ophthalmic artery was also improved in 1.5-T TOF-MRA images3.
Based on the above findings, the advantages of AiCE can be summarized as follows: 1) it can selectively eliminate noise in images, 2) it is applicable to a wide range of scan regions and sequences, 3) the time required for reconstruction is relatively short, and 4) it can be used in combination with SPEEDER, Compressed SPEEDER (CS), Fast 3D mode, etc., which are high-speed scanning methods based on undersampling of the k-space data. The combined methods in item 4) above, which are collectively referred to as "Hybrid DLR", are discussed below.
It is generally accepted that in order to improve image quality, the number of acquisitions must be increased. However, in a review conducted at our hospital to evaluate T2WI of the head, we found that applying AiCE to noisy images acquired with only 2 acquisitions allowed us to obtain high-quality images comparable to conventional images acquired with 5 acquisitions1. Similar results were observed for FLAIR and MPRAGE images1. For example, in high-resolution heavily T2WI 3D MR cisternography, the nerve tracts could be more clearly visualized than with conventional images2, and the depiction of fine blood vessels such as the ophthalmic artery was also improved in 1.5-T TOF-MRA images3.
Based on the above findings, the advantages of AiCE can be summarized as follows: 1) it can selectively eliminate noise in images, 2) it is applicable to a wide range of scan regions and sequences, 3) the time required for reconstruction is relatively short, and 4) it can be used in combination with SPEEDER, Compressed SPEEDER (CS), Fast 3D mode, etc., which are high-speed scanning methods based on undersampling of the k-space data. The combined methods in item 4) above, which are collectively referred to as "Hybrid DLR", are discussed below.
Hybrid DLR
T2WI scanning of the pituitary gland was performed with a CS factor of 2.4 and scan time of 1 minute and 8 seconds. Images to which only CS was applied were then compared against images to which Hybrid DLR (AiCE and CS) was applied, and it was found that the SNR was markedly improved in the Hybrid DLR images4.
Figure 1 shows T2WI in a patient with a hamartoma in the hypothalamus acquired with a scan time of approximately 3 minutes. The conventional image (a) was compared against the Hybrid DLR (AiCE and CS) image acquired with a scan time of approximately 1 minute (b). Structures in the tuber cinereum and pars infundibularis are much more clearly depicted in image (b) than in image (a).
Figure 1 shows T2WI in a patient with a hamartoma in the hypothalamus acquired with a scan time of approximately 3 minutes. The conventional image (a) was compared against the Hybrid DLR (AiCE and CS) image acquired with a scan time of approximately 1 minute (b). Structures in the tuber cinereum and pars infundibularis are much more clearly depicted in image (b) than in image (a).
Figure 1: Comparison between conventional T2WI and Hybrid DLR T2WI in a patient with a hamartoma in the hypothalamus
In addition, a conventional TOF-MRA image acquired with a scan time of 4 minutes and 40 seconds and a TOF-MRA image acquired with a scan time of 1 minute and 26 seconds to which Hybrid DLR (AiCE and Fast 3D mode) was applied were compared in a patient with an aneurysm of the middle cerebral artery. The depiction of fine arteries was found to be comparable in the two images. A conventional MRA image and a high-resolution MRA image to which Hybrid DLR (AiCE and Fast 3D mode) was applied, both acquired with a scan time of 4 minutes and 40 seconds, were compared in a patient with occlusion of the internal carotid artery. The peripheral branches of the ophthalmic artery and anterior cerebral artery were more clearly depicted in the Hybrid DLR image thanks to the reduced noise levels and higher SNR. In MRCP performed in a patient with autoimmune pancreatitis, images acquired using respiratory-gated scanning and breath-hold scanning (with and without AiCE) were compared. Breath-hold scanning with AiCE provided clear images of the bile ducts in the liver comparable to those acquired by respiratory-gated scanning5.
Based on above findings, the advantages of Hybrid DLR can be summarized as follows: 1) it can be used in combination with high-speed scan methods based on undersampling of the k-space (Parallel Imaging (PI), CS, and Fast 3D mode), 2) it reduces noise and improves the SNR when high-speed, high-resolution scanning is combined with AiCE, 3) it is applicable to a wide range of scan regions and sequences, and 4) the time required for reconstruction is relatively short.
Based on above findings, the advantages of Hybrid DLR can be summarized as follows: 1) it can be used in combination with high-speed scan methods based on undersampling of the k-space (Parallel Imaging (PI), CS, and Fast 3D mode), 2) it reduces noise and improves the SNR when high-speed, high-resolution scanning is combined with AiCE, 3) it is applicable to a wide range of scan regions and sequences, and 4) the time required for reconstruction is relatively short.
Precise IQ Engine (PIQE)
1. Overview of PIQE
The high resolution reconstruction technology known as PIQE, which is a method for increasing the number of pixels per unit length, can improve spatial resolution by up to a factor of 9. When PIQE is applied to relatively low-resolution images containing noise, high-resolution images with a high SNR can be obtained.
The high resolution reconstruction technology known as PIQE, which is a method for increasing the number of pixels per unit length, can improve spatial resolution by up to a factor of 9. When PIQE is applied to relatively low-resolution images containing noise, high-resolution images with a high SNR can be obtained.
Figure 2: Overview of PIQE
PIQE is composed of two separate neural networks: a denoising neural network and an upsampling neural network (Fig. 2).
After noise has been minimized using the same technique as employed in AiCE, the resolution of the image is improved by zero-fill interpolation (ZIP). The ZIP method subjectively improves the apparent resolution by filling the high-frequency components in k-space with zeros, but its limitations are that some blurring remains and ringing artifacts are generated. Image resolution can be improved by removing the blurring and artifacts by upsampling using deep learning technology and thus recovering the image sharpness. This means that PIQE is designed to ensure versatility and stability by employing deep learning for denoising and upsampling.
After noise has been minimized using the same technique as employed in AiCE, the resolution of the image is improved by zero-fill interpolation (ZIP). The ZIP method subjectively improves the apparent resolution by filling the high-frequency components in k-space with zeros, but its limitations are that some blurring remains and ringing artifacts are generated. Image resolution can be improved by removing the blurring and artifacts by upsampling using deep learning technology and thus recovering the image sharpness. This means that PIQE is designed to ensure versatility and stability by employing deep learning for denoising and upsampling.
Figure 2: Overview of PIQE
2. Applications of PIQE
In MRI, there are usually trade-offs between spatial resolution, SNR, and contrast. However, PIQE makes it possible to achieve both high contrast and high spatial resolution.
Figure 3 shows high-resolution coronal T2WI with a matrix of 960×960 and a slice thickness of 3 mm. Compared to a conventional image acquired using AiCE (a) with a scan time of 11 minutes and 40 seconds, a PIQE image (b) acquired with a scan time of 3 minutes and 56 seconds shows an extremely high SNR and excellent contrast, with outstanding image quality seemingly comparable to that of a 7-T system.
Figure 4 shows an enlarged view of cross-sectional T2WI acquired with a matrix of 448×448. The branches of the trigeminal nerve (orange circles) in the Meckel's cave, which are poorly visualized in a conventional image (a), are clearly depicted in a PIQE image (b) thanks to the high spatial resolution.
In MRI, there are usually trade-offs between spatial resolution, SNR, and contrast. However, PIQE makes it possible to achieve both high contrast and high spatial resolution.
Figure 3 shows high-resolution coronal T2WI with a matrix of 960×960 and a slice thickness of 3 mm. Compared to a conventional image acquired using AiCE (a) with a scan time of 11 minutes and 40 seconds, a PIQE image (b) acquired with a scan time of 3 minutes and 56 seconds shows an extremely high SNR and excellent contrast, with outstanding image quality seemingly comparable to that of a 7-T system.
Figure 4 shows an enlarged view of cross-sectional T2WI acquired with a matrix of 448×448. The branches of the trigeminal nerve (orange circles) in the Meckel's cave, which are poorly visualized in a conventional image (a), are clearly depicted in a PIQE image (b) thanks to the high spatial resolution.
Figure 3: Effect of PIQE in high-resolution T2WI
Figure 3: Effect of PIQE in high-resolution T2WI
Figure 4: Imaging of the trigeminal nerve using PIQE
Figure 4: Imaging of the trigeminal nerve using PIQE
Figure 5 shows an image of the pituitary gland acquired in a T1-weighted dynamic study with a matrix of 192×192, a slice thickness of 2 mm, and a scan time of 22 seconds. The SNR and resolution are significantly improved by PIQE (b). PIQE is considered to be an extremely promising technique for performing dynamic studies in the future.
Figure 6 shows fat-sat T2WI of the orbits and paranasal sinuses. The olfactory nerve is very clearly depicted when PIQE is applied (b, orange arrow). In addition, structures such as the extraocular muscles, the optic nerve, and the superior ophthalmic vein in the orbit, as well as the mucosa of the paranasal sinuses and nasal mucosa, are also depicted with outstanding clarity. The applications of PIQE in examinations of the head and neck are expected to be of great clinical value in the future.
Figure 6 shows fat-sat T2WI of the orbits and paranasal sinuses. The olfactory nerve is very clearly depicted when PIQE is applied (b, orange arrow). In addition, structures such as the extraocular muscles, the optic nerve, and the superior ophthalmic vein in the orbit, as well as the mucosa of the paranasal sinuses and nasal mucosa, are also depicted with outstanding clarity. The applications of PIQE in examinations of the head and neck are expected to be of great clinical value in the future.
Figure 5: Effect of PIQE in dynamic studies
Figure 5: Effect of PIQE in dynamic studies
Figure 6: Application of PIQE in head and neck scanning
Figure 6: Application of PIQE in head and neck scanning
3. Advantages and limitations of PIQE
The advantages of PIQE can be summarized as follows: 1) it improves spatial resolution while maintaining good tissue contrast, 2) it can be used in combination with PI, and 3) it is applicable to a wide variety of scan regions. On the other hand, a number of challenges remain: the reconstruction matrix is limited and the volume of data is extremely large.
The advantages of PIQE can be summarized as follows: 1) it improves spatial resolution while maintaining good tissue contrast, 2) it can be used in combination with PI, and 3) it is applicable to a wide variety of scan regions. On the other hand, a number of challenges remain: the reconstruction matrix is limited and the volume of data is extremely large.
Conclusion
AiCE reduces image noise and improves the SNR, and hybrid DLR combined with CS and Fast 3D mode enables faster scanning and higher resolution while maintaining a high SNR.
Also PIQE can improve spatial resolution while maintaining good tissue contrast.//
Also PIQE can improve spatial resolution while maintaining good tissue contrast.//
Toshinori Hirai, MD, PhD
Professor, Department of Diagnostic Radiology, Faculty of Life Sciences,
Kumamoto University, Japan
Professor, Department of Diagnostic Radiology, Faculty of Life Sciences,
Kumamoto University, Japan
References
1 Kidoh, M., et al., Magn. Reson. Med. Sci., 19(3): 195-206, 2020.
2 Uetani, H., et al., Neuroradiology, 63(1): 63-71, 2021.
3 Yasaka, K., et al., Jpn. J. Radiol., 40(5): 476-483, 2022.
4 Uetani, H., Hirai, T., et al., Eur. Radiol., 32(7): 4527-4536, 2022.
5 Shiraishi, K., et al., Eur. Radiol., 2023 (in press).
This article is a translation of the INNERVISION magazine, Vol.38, No.6, 2023.
Disclaimer
The contents of this report include the personal opinions of the author based on his clinical experience and knowledge
Deep learning technology is used in the design stage of the image reconstruction processing. AiCE and PIQE do not have self-learning capabilities.
1 Kidoh, M., et al., Magn. Reson. Med. Sci., 19(3): 195-206, 2020.
2 Uetani, H., et al., Neuroradiology, 63(1): 63-71, 2021.
3 Yasaka, K., et al., Jpn. J. Radiol., 40(5): 476-483, 2022.
4 Uetani, H., Hirai, T., et al., Eur. Radiol., 32(7): 4527-4536, 2022.
5 Shiraishi, K., et al., Eur. Radiol., 2023 (in press).
This article is a translation of the INNERVISION magazine, Vol.38, No.6, 2023.
Disclaimer
The contents of this report include the personal opinions of the author based on his clinical experience and knowledge
Deep learning technology is used in the design stage of the image reconstruction processing. AiCE and PIQE do not have self-learning capabilities.
Learn more about Canon Medical's MRI technologies and products
