Feature | June 03, 2014 | Greg Freiherr

Image Fusion, Cubism and the Future of Imaging

 picasso woman with green hat.

An art critic once wrote, “If all women resembled those painted by Picasso, the Earth would be depopulated toward the end of the century.” The critic was speaking about Picasso’s “Woman in a Green Hat,” describing the model in the painting as “corpselike,” “greenish” and “formless.” 

Anyone not sharing Picasso’s cubist view of the world would likely agree. But Picasso painted what he felt more than what he saw. He painted what could not otherwise be seen. 

Commenting on the “Woman in a Green Hat,” Picasso said he painted the crooked nose on purpose “so that people were forced to see a nose. Later … they will recognize that it is not crooked at all.” 

Medical imaging has been moving in a similar direction for years. Engineers have been assembling technologies, sometimes from wholly different modalities, to form novel constructions that generate information that cannot otherwise be obtained. Positron emission tomography/computed tomography (PET/CT) is an example. 

The fusion of two modalities, PET/CT transcends its components, rendering a combination of anatomical and functional data. The concept of modality fusion took root with this hybrid, blossomed several years later into single-photon emission computed tomography (SPECT)/CT and, most recently, bore fruit as PET/magnetic resonance (MR).  

Each hybrid provides information that cannot otherwise be obtained. But, just as being able to paint in cubist style is not reason enough to do so, neither is being able to fuse imaging technologies enough to justify such hybridization. Only if fused technologies provide clinical value are they worth the effort of their creation. And there is the rub.

Clinical value often is recognizable only in hindsight. And even then this value may be realized in ways not initially expected.

In the early 1990s, engineers were mounting image intensifiers the size of curb-side garbage cans on a ring that spun at high speed around patients’ heads. The dynamic images of 3-D brain aneurysms filling with contrast medium were as remarkable as the scans were frightening to watch. 

The advent of flat panel detectors later in that decade improved image quality and increased the speed with which this ring could be spun. So encouraging was this leap forward that some pundits believed these once gangly and oversized systems might one day replace CTs. What happened instead was the evolution of multi-detector CTs (MDCT) and C-arm rotational angiography.

Today the most advanced MDCTs can cover the entire brain or heart in a single rotation, much like the image intensifier (II) prototype was envisioned to be able to do. Similarly, the technique of rotational angiography proven by those spinning IIs was modified to work with a single flat panel mounted on a C-arm, systems that are now commonplace in imaging centers around the world …
as are multi-detector CTs. 

The footprints of other hybridizations are evident elsewhere. One, involving digital breast tomosynthesis (DBT), developed out of necessity. Seen for decades as the holy grail for evaluating breast cancer, DBT was found to work best with the technology it was meant to replace. The signs of cancer were easier to spot, it was learned, when 3-D and 2-D images were interpreted together. The need for this pairing led one manufacturer to develop an algorithm that synthesizes 2-D images from 3-D data, creating a synergy of 2-D and 3-D mammography in a single unit. 

Imaging technologies will continue to be fused in the years ahead, propelling all of medicine forward as it has in the past. Like the early rotational angiography machines of 20 years ago, their unions may look like they’ve been torn from a Rube Goldberg sketchbook. But their clinical impact may be no laughing matter.

It will serve us well to remember that many of today’s most useful imaging systems began in the imagination of someone who saw, as Picasso said, “with different eyes.”

Greg Freiherr has reported on developments in radiology since 1983. He runs the consulting service, The Freiherr Group. Read more of his views on his blog at www.itnonline.com.

Related Content

Lymphocytic Inflammation in a Lung from a Patient Who Died from Covid-19. The gross appearance of a lung from a patient who died from coronavirus disease 2019 (Covid-19) is shown in Panel A (the scale bar corresponds to 1 cm). The histopathological examination, shown in Panel B, revealed interstitial and perivascular predominantly lymphocytic pneumonia with multifocal endothelialitis (hematoxylin–eosin staining; the scale bar corresponds to 200 μm).

Lymphocytic Inflammation in a Lung from a Patient Who Died from Covid-19. The gross appearance of a lung from a patient who died from coronavirus disease 2019 (Covid-19) is shown in Panel A (the scale bar corresponds to 1 cm). The histopathological examination, shown in Panel B, revealed interstitial and perivascular predominantly lymphocytic pneumonia with multifocal endothelialitis (hematoxylin–eosin staining; the scale bar corresponds to 200 μm). Image courtesy of The New England Journal of Medicine

News | Coronavirus (COVID-19) | May 22, 2020
May 22, 2020 — In a new study in the New
Butterfly iQ devices provide revolutionary portable ultrasound capabilities for faster and easier screening and monitoring
News | Coronavirus (COVID-19) | May 20, 2020
May 20, 2020 —Atrium Health is one of the first major health systems in the nation to put into wide practice a new...
Examples of chest CT images of COVID-19 (+) patients and visualization of features correlated to COVID-19 positivity. For each pair of images, the left image is a CT image showing the segmented lung used as input for the CNN (convolutional neural network algorithm) model trained on CT images only, and the right image shows the heatmap of pixels that the CNN model classified as having SARS-CoV-2 infection (red indicates higher probability). (a) A 51-year-old female with fever and history of exposure to SARS-

Figure 1: Examples of chest CT images of COVID-19 (+) patients and visualization of features correlated to COVID-19 positivity. For each pair of images, the left image is a CT image showing the segmented lung used as input for the CNN (convolutional neural network algorithm) model trained on CT images only, and the right image shows the heatmap of pixels that the CNN model classified as having SARS-CoV-2 infection (red indicates higher probability). (a) A 51-year-old female with fever and history of exposure to SARS-CoV-2. The CNN model identified abnormal features in the right lower lobe (white color), whereas the two radiologists labeled this CT as negative. (b) A 52-year-old female who had a history of exposure to SARS-CoV-2 and presented with fever and productive cough. Bilateral peripheral ground-glass opacities (arrows) were labeled by the radiologists, and the CNN model predicted positivity based on features in matching areas. (c) A 72-year-old female with exposure history to the animal market in Wuhan presented with fever and productive cough. The segmented CT image shows ground-glass opacity in the anterior aspect of the right lung (arrow), whereas the CNN model labeled this CT as negative. (d) A 59-year-old female with cough and exposure history. The segmented CT image shows no evidence of pneumonia, and the CNN model also labeled this CT as negative.  

News | Coronavirus (COVID-19) | May 19, 2020
May 19, 2020 — Mount Sinai researchers are the first in the country to use...
Advanced imaging data exchange is now live in Colorado due to the partnership of Health Images and the Colorado Regional Health Information Organization

Getty Images

News | Radiology Business | May 18, 2020
May 18, 2020 — 
Radiologists from Shanghai discuss modifying exam process and disinfecting exam room, while outlining personal protection measures during the coronavirus disease outbreak

(HIS = hospital information system, RIS = radiology information system) Image courtesy of American Journal of Roentgenology (AJR)

News | Coronavirus (COVID-19) | May 18, 2020
May 18, 2020 — In an open-access article published ahead-of-print
Chest radiograph of a 23-year-old male with no past medical history who tested positive for COVID-19 via RT-PCR and was subsequently discharged from the emergency department with home care and isolation precautions. Portable CXR shows right and left peripheral lower lung zone hazy opacities; total score=2

Top image: Chest radiograph of a 23-year-old male with no past medical history who tested positive for COVID-19 via RT-PCR and was subsequently discharged from the emergency department with home care and isolation precautions. Portable CXR shows right and left peripheral lower lung zone hazy opacities; total score=2.

Bottom image: Chest radiograph in a 32-year-old overweight (BMI=30) COVID-19 positive male with a history of childhood asthma who was subsequently admitted and intubated in the ICU for 3 days. Portable CXR shows opacities in all three right lung zones and in the left middle and lower lung zones; total score=5. Image courtesy of Mount Sinai Health System

News | Coronavirus (COVID-19) | May 15, 2020
May 15, 2020 — Chest X-rays performed on young and
Now a research team — led by Tohoku University Professor, Wataru Yashiro — has developed a new method using intense synchrotron radiation that produces higher quality images within milliseconds.

How the bent crystal changes the direction of the X-rays. Image courtesy of Tohoku University

News | Computed Tomography (CT) | May 15, 2020
May 15, 2020 — Many will undergo a computed tomogr...
Colored areas of the brain represent regions where the loss of brain synapses in people with early-stage Alzheimer’s was greater than people with normal cognitive function.

Colored areas of the brain represent regions where the loss of brain synapses in people with early-stage Alzheimer’s was greater than people with normal cognitive function. Image courtesy of YaleNews.

News | PET Imaging | May 14, 2020
May 14, 2020 — New imaging technology allows scientists to see the widespread loss of brain synapses in early stages