You are here:

RAID 6 Failure Disables Epicor DMS in Hospital

March 22, 2023
Rescuing Decades of Data - A Multi-Hospital Recovery Case

Table of Contents

Data loss can have devastating consequences for businesses, particularly those in the healthcare industry where sensitive patient data and medical records are involved. Recently, our team at PITS Global Data Recovery Services tackled an intricate data recovery project involving a multi-hospital healthcare provider in Maryland and Delaware. This case study delves into the technical challenges and innovative solutions that led to the successful recovery of over 30 years’ worth of critical data for the client.

Client Background

Our client, a multi-million-dollar healthcare provider, managed 30 hospitals across Maryland and Delaware, employing over 2,000 staff members. They operated a dedicated fiber optic network connecting their offices and maintained a secure data center with biometric authentication for enhanced security.

The company relied on the robust Epicor Document Management System (DMS) to store, manage, and control access to sensitive patient data, including medical bills, records, DICOM images, MRI, and CT scans. The data, spanning over 30 years, was stored on RAID 6 configured storage devices with Carbonite online backup systems in place. Unfortunately, poor IT management, disaster recovery practices, and a lack of regular server assessments led to catastrophic data loss.

The Importance of Data in the Healthcare Industry

In the healthcare sector, data is a vital resource. Medical professionals rely on accurate, up-to-date patient information to make informed decisions about patient care and treatment plans. 

Additionally, billing departments need access to complete records to ensure timely and accurate invoicing. Losing access to this data can lead to significant operational disruptions, financial losses, and potential compliance issues.

Evaluation of Damaged RAID Hard Drives

Data Loss Problem After ECM Crash

The healthcare provider’s primary storage system, which housed all documents and database files, experienced multiple sequential hard drive failures. 

These failures occurred over several months, unbeknownst to the IT department, due to the absence of notification systems and regular HP ILO Management checks. Eventually, the entire data storage system collapsed, putting decades of vital patient data at risk.

Recovery Process of Failed HPE Server

Upon evaluating the situation, our team of data recovery experts at PITS Global Data Recovery Services devised a customized plan that included the following steps:

Secure Environment

To protect sensitive data, we established a secure environment for data recovery, complete with strict access controls, encryption measures, and an isolated cleanroom workspace.

Hardware Assessment

We performed a comprehensive evaluation of the damaged RAID 6 array, identifying the extent of damage and determining the best course of action for recovery. Our assessment found that several hard drives had failed sequentially over time, complicating the recovery process.

Repairing Failed Hard Drives

The hard drives had experienced mechanical failure, necessitating repair before data recovery could proceed. We employed a mobile, portable ISO Certified Class 10 cleanroom laminar flow hood to open the hard drives in a cleanroom environment and replace failed parts without causing further damage. Our logistics team delivered all necessary tools, mobile laminar flow hoods, and reagents from our New York lab to Maryland. Using these tools, we repaired the HPE 12GB SAS 2.5″ hard drives.

Customized Recovery Solution

We designed a tailored data recovery solution that combined cutting-edge tools and techniques to extract lost data. This involved creating bitwise disk images from the repaired, working hard drives, and rebuilding the RAID 6 array using specialized software.

Data Extraction and Reconstruction

Our team meticulously extracted and reconstructed the RAID array using disk images. We then employed advanced data recovery algorithms to recover the lost files, restoring the Epicor DMS and the file system to their original states.

Handling the Copying Challenge

The primary challenge was to save 40TB of very small files. Copying 40TB of video files versus copying 40TB of files under 100KBs presents a significant difference in timing due to processing time. To overcome this challenge, we installed 10 HP Z G8 Workstations with 512GB RAM and Dual XEON Processors, which enabled us to handle the extensive copying process efficiently. It took us 5 business days to complete the copying.

Quality Assurance

We conducted rigorous quality checks to verify the accuracy and completeness of the recovered data, cross-referencing the data with the client’s records and performing validation tests, such as checking file integrity and opening sample files.

Lessons Learned and Best Practices

This case study highlights the importance of robust IT management, disaster recovery practices, and regular server assessments in preventing catastrophic data loss. To safeguard valuable data, organizations should consider implementing the following best practices:

Regular monitoring and maintenance. Set up automated monitoring systems and perform regular health checks on storage systems to detect potential hardware failures and address them promptly.

Implementing a comprehensive backup strategy. Ensure that multiple backup copies are stored in different locations, including offsite and/or cloud backups, to minimize the risk of data loss due to hardware failures, natural disasters, or cyberattacks.

Employee training and awareness. Educate IT staff and other employees on the importance of data security, early warning signs of potential issues, and the proper procedures for addressing them.

Dell Drives Server Recovery

Establishing a disaster recovery plan. Develop a detailed disaster recovery plan that outlines the steps to be taken in the event of data loss, including the process for engaging a professional data recovery service.

Benefits of Choosing Our Recovery Service

Through our systematic and innovative approach, we successfully recovered 99.8% of the lost data, far exceeding the client’s expectations. The healthcare provider was able to resume normal operations with minimal downtime, ensuring uninterrupted patient care and billing processes across their network of hospitals.

Why Choose Us

PITS Global Data Recovery Services delivers premium data recovery solutions with a focus on quality and individual care for every case. As a BBB-accredited and DUNS-verified business, we are committed to excellence, ensuring 100% customer satisfaction. Our tailored service options suit a range of recovery needs and budgets. For critical situations, we offer emergency data recovery, restoring data quickly and reliably without compromising quality.

If you’re facing a data loss situation, don’t hesitate to contact us. Our 24/7 data recovery services are available to you, 365 days a year. Let us help you recover your precious data today.

We start the recovery process with a risk-free evaluation. Our technicians estimate reasons for data loss and the level of damage. Based on it, we select the most suitable recovery strategy.

With years in the data recovery industry, our company supports the highest customer satisfaction rate. We do everything to provide a positive experience for our clients.

During our remote customer file verification session, you will thoroughly review all necessary documents and records to ensure accuracy and compliance.

We offer data recovery services from over 50 locations across the US. This means that no matter where you are located, you can access our services to recover the data.

With our certified data recovery services and 99% success rate, we are confident that we can recover your precious data and get you back up and running in no time.

The success of this multi-hospital data recovery case in Maryland and Delaware demonstrates our team’s expertise, dedication, and ability to handle complex and sensitive projects with highly technical challenges. At PITS Global Data Recovery Services, we are committed to helping our clients overcome data loss challenges and safeguard their valuable information. 

If you or your organization need data recovery services, do not hesitate to contact us to discuss your requirements and discover how we can help you. With the right preventive measures and a reliable data recovery partner, organizations can minimize the risk of data loss and maintain the integrity of their critical information.

Frequently Asked Questions

A RAID 6 array is a type of data storage system that uses two sets of parity data to protect against data loss due to drive failure. It requires at least four drives to operate, and can sustain up to two drive failures without losing data.

When a drive fails in a RAID 6 array, the system will continue to function and the data on the remaining drives can still be accessed. However, the array will be operating in a degraded state and is vulnerable to further drive failures. It’s important to replace the failed drive as soon as possible to restore the array’s fault tolerance.

The RAID controller should provide an alert or notification when a drive has failed. You may also notice slower performance or data errors when accessing the array. It’s important to monitor the status of your RAID array regularly to detect and address any issues.

A RAID 6 array can sustain up to two drive failures without losing data. However, if two drives fail at the same time or during the rebuild process, the array will fail and data may be lost. It’s important to have a backup of your data to protect against this scenario.

In some cases, data can be recovered from a failed RAID 6 array using specialized data recovery tools and services. However, the success of data recovery depends on various factors, such as the extent of the damage and the quality of the backup. It’s recommended to consult a professional data recovery service for assistance with a failed RAID 6 array.

Related Case Studies

HP-Proliant-ML380-G6-recovery

RAID 5 Data Recovery from HP ProLiant ML380 G6 Server

In this blog, we write about another successful data recovery case from a damaged RAID array. Click here to learn more about the recovery process.

How Data Avoid Loss

Data Recovery from RAID 10 / Dell PowerEdge R430

Our data recovery engineers have recently recovered Dell PoweEdge RAID 10 array and restored lost data. Click to learn more about the case.

Request Help

Our main priority is providing excellent customer service and a top-notch customer experience for all clients. To get started, simply fill out the help form below. A dedicated customer service representative will then reach out to you to assess your needs and provide comprehensive information about our data recovery services.

"*" indicates required fields

Hidden
Hidden
Hidden
Hidden
Hidden
Hidden
Facebook
LinkedIn
Reddit
X