Healthcare organizations today are storing petabytes of medical imaging data — lab slides, X-rays, magnetic resonance imaging (MRI) scans, computed tomography (CT) scans and more — a number that is expanding with no end in sight. To make matters worse, due to regulations, healthcare providers typically must retain medical imaging files for several years; they may even have an enterprise-wide policy of not deleting data ever. Aside from compliance requirements, clinical researchers may need access to the data indefinitely.
This presents a conundrum from both an economic and IT management perspective. Internal storage for large image files is expensive — costing millions a year for some organizations on Porsche-grade network-attached storage (NAS) devices. The data must be secured, replicated and backed up. Meanwhile, in most cases, imaging data is rarely accessed after a few days.
To get more flexibility and cost savings from storage, healthcare organizations are increasing their investments in the cloud. Such decisions can be rife with politics and long-standing institutional perspectives. Health systems are generally risk-averse — they are handling sensitive patient information after all — and tolerance for downtime is usually quite low.
Cloud Tiering for Dear Life
Healthcare professionals depend on accurate, timely data to make the best decisions; the loss of important patient data can have dire consequences. Keeping these large files safe and readily available could be a matter of life or death for a patient with a serious illness.
In discussions with storage managers in healthcare, we’ve learned that organizations can save 60 percent or more by moving pathology images that are 90 days or older from on-premises storage (such as hyperconverged infrastructure (HCI) and NAS arrays) to a third tier on Google Cloud Object Storage. That’s compelling evidence to consider a new unstructured data management strategy.
One healthcare system is scanning 1TB of pathology slides per day; they remain on the Tier 1 HCI storage for three days, after which they are moved to a Tier 2 NAS device. Using a data management solution, the post 90-day-old slides are automatically tiered to Google Cloud storage, and once there, move to lower cost storage as they age further. Since older images stored in the cloud are accessed so rarely, cloud egress fees to bring them back to the on-premises digital pathology solution are typically minimal.
Best Practices for Medical Imaging Cloud Tiering
Medical imaging systems use high-performance NAS devices to store medical images. This ensures fast access to files for the medical staff. However, such high-end storage is expensive and the images are generally not used after patient diagnosis. The right data management solution can automatically move older images to the object storage in the cloud based on policy for significantly cheaper storage and without affecting user experience.
Here’s what to consider when tiering images to the cloud for both performance and cost management.
User experience: Clinicians, technicians and other healthcare employees should be able to find old images from the original file location. However, the old images now reside in the cloud as objects. When a user wants to access an archived image, the data management solution should cache it locally on the NAS for fast access and without changing the way users find and open files.
Cloud-native access: If you can store images in the cloud-native form, researchers can access these images using new cloud-native services and tools for AI and ML processing. For example, Amazon HealthLake is a new data lake service incorporating machine learning models for analytics projects. Azure has several machine learning initiatives in healthcare including a partnership focused on decoding the immune system. By enabling direct cloud access to tiered images and data, healthcare organizations gain a larger portfolio of advanced tools and services to further their R&D efforts. Some data management solutions require access only through the proprietary storage technology, limiting the use of cloud technologies and resulting in additional vendor licensing and cloud egress fees.
Global search: Cutting storage costs is just one advantage of cloud tiering. The ability to create a virtual data lake of all your images using cloud technology can reap long-term benefits for research, analysis and compliance needs. Clinicians can use the data lake as a basis to conduct large research projects on clinical outcomes across a broad demographic or more granular projects focused on analyzing specific patient populations for novel treatments. Look for data management solutions with an open, scalable architecture and which can deliver the flexibility to meet the search needs of IT, business and medical teams. This way, you can not only safely and cost-effectively store medical images but glean new life from them, which can benefit both patients and the organization.
Paving the Way for Medical Image Longevity
As high-value unstructured data like medical images exceed the limits of on-premises storage, the options are becoming increasingly limited. For compliance or other reasons, some healthcare organizations retain images indefinitely; the amount of storage required to house petabytes of clinical data has become prohibitively expensive. By creating a cloud tiering strategy based on hot, warm and cold data, with automation to transparently move files as they age, healthcare organizations can stay in compliance and potentially generate new revenue streams by creating virtual data lakes of historical diagnostic information.
Kumar Goswami is co-founder and CEO of Komprise, a company that helps enterprises handle the incoming deluge of data and deliver an easier path to the cloud with analytics-driven data management that helps customers know first, move smart and take control their data to save costs and extract more business value — without disrupting access.
December 06, 2023 
