ironstream_how_can_syncsort_help_me

Data Integrity and Security

When dealing with millions of GBs of data (petabytes), the utmost care must be taken to ensure that customer data is protected and preserved by adhering to the highest standards of data security and by engineering systems that ensure the integrity of the data. Although many cloud service providers talk a lot about data encryption, most don’t discuss or provide mechanisms to protect the integrity of the data, which is equally as important. After all, what good is your data if it becomes corrupted or damaged?

The Dangers of Silent Data Corruption

Silent data corruption is a real threat to your data. In a study by CERN, silent data corruption was observed in 1 out of every 1500 files. Redundant storage, RAID, mirroring, checksums on individual components, replication, and even backup are not sufficient to protect against silent data corruption. If data is silently corrupted without detection, this corrupted data is then replicated or backed up, thereby corrupting any redundant or backup copies. The causes of silent data corruption are many and include hardware, firmware, and software faults.

infrastructure-data-integrity

BluPointe’s Unique Data Integrity Assurance Systems

Our data integrity protection systems are second to none. Our end-to-end 256-bit checksums “tag” your data with cryptographically verifiable identifiers, and these checksums are then used to verify the integrity of the data throughout the entire process. If corruption within a particular block is detected, a redundant copy of that block is used to automatically repair the corruption.

The redundant copies of the data are split across physically separate raw storage devices each located in physically distinct storage subsystems. Storing the redundant copies physically far away from each other ensures that clusters of disk failures due to localized vibration, enclosure failures, or IO controller failures do not affect more than one redundant copy of the data, providing an unparalleled level of protection.

In addition to automatic detection and repair upon data access, all data is periodically verified against their end-to-end checksums to proactively identify and automatically repair silent data corruption.

As an additional measure of data integrity assurance, every data block backed up to our cloud is digitally signed by your encryption key and then verified during restore. This provides cryptographic assurance that your data has not been tampered with by anyone and is 100% identical to what was originally backed up.