Encryption on Record Level. Protect the people behind the data – as well as your company – by preventing data theft.
Every Optimaize SearchCluster stores the data for each record not in plain text, but in encrypted form.
Compare a traditional database on the left, such as a relational database table, with an SearchCluster on the right side:
Why encryption is mandatory
Protect the people’s data, protect your company’s reputation and budget, comply with regulations.
Data leaks have become a regular topic in the news, and only the largest of leaks make it to the headlines.
IT security is hard, as systems are inherently unsafe. While companies are levelling up their IT security, so are the black hat hackers improving their capabilities. One weak link can be enough to let unpermitted access to systems and data through. Attack vectors are unpatched systems, human error, insufficient security measures, zero-day exploits (which have become more common), malicious insiders, and social engineering such as phishing attacks on DevOps team members.
Unauthorized access to system happens all the time, and will happen in the future. Once data is copied, it cannot be undone, it cannot be deleted, it is only a matter of time, sometimes years, until it appears publicly on the internet.
Encrypted data is safe, it cannot be read, it cannot be leaked. By using strong encryption and storing the keys off-site it becomes much harder to get to the plain text data.
Supposedly private personal identifiable information to be available publicly is a nuisance for most, but for a few it puts them in real danger. In recent years, regulators especially in Europe, Northern America and Australia attempt to protect their citizens, by means of rules and fines. Rules what data may be stored (only ask for and store PII that you need for your operations), for how long it may be kept (data retention policy, delete old customer records), and how to handle IT infrastructure safety.
Now, having the private data of your customers exposed on the internet not only harms your company’s reputation, but comes with hefty fines, easily a few dollars or euros per person, quickly resulting in millions, in some cases hundreds of millions.
Recent Leaks and Fines
en.wikipedia.org/wiki/List_of_data_breaches
en.wikipedia.org/wiki/GDPR_fines_and_notices
| Year | Company | Fine | Affected people |
|---|---|---|---|
| 2024 | British Airways | £20 million | 400 thousand customers |
| 2024 | Game Freak (Pokémon) | ¥100 million | 3k employees |
| 2023 | Latitude Financial | AUD 76 million | 14 million customers |
| 2023 | T-Mobile | $350 million | 77 million customers |
| 2023 | Medibank | AUD 50 million | 14 million customers |
| 2022 | Zoetop | $1.9 million | tens of millions of customers |
| 2022 | CafePress | $500 thousand | |
| 2021 | Capital One | $190 million | 100 million customers |
| 2021 | Psykoterapiakeskus Vastaamo | €608 thousand | |
| 2020 | H&M | €35.25 million | several hundred employees |
| 2019 | Equifax | $700 million | 150 million customers |
| 2019 | Yahoo! | $35 million | hundreds of millions of users |
| 2019 | Marriott | £18.4 million | 339 million guests |
| 2019 | Cathay Pacific | £500 thousand | |
| 2018 | Anthem | $115 million | 79 million customers |
| 2017 | Target | $18.5 million | |
| 2016 | Uber | $148 million | 57 million customers |
| 2015 | AT&T | $25 million | 280 thousand customers |
| 2014 | Home Depot | $200 million | 53 million customers |
Overview
The records in your application’s index are encrypted with a strong AES 256 bit cipher. The records are only decrypted on demand, one at a time (in parallel), for data processing. This happens when a record is found in a search operation.
About AES 256
Safety
AES-256 is considered one of the most secure encryption standards there is as of today, and uncrackable by brute force.
It would take a single PC 27 trillion trillion trillion trillion trillion years to brute-force one key, to decrypt one record, and every record has a different key.
While quantum computers will be able to break asymmetric encryption, it will only weaken but not break the symmetric encryption algorithms such as AES.
This has led to AES-256 to being the most used industry standard encryption mechanism. It is also commonly used to encrypt the internet traffic going through HTTPs.
Performance
The additional work for the hardware is negligible, not noticeable neither when indexing nor when searching. A single record encryption or decryption on a single core takes a few microseconds, making it possible to process a few thousand records per second per CPU core.
Further reading
Key storage in a Vault
Encryption keys are stored within the IT infrastructure in a Vault cluster.
Each record uses a different, composed encryption key, on the base of the index-wide key from the Vault, to encrypt and decrypt its data.
Key rotation, and change of algorithm
A background process or manual trigger may be used to iteratively replace encryption keys on a timed interval, for example every 3 months. Without any application downtime or noticeable performance hit, the store is converted. During the transformation phase, both keys (old and new) work, each record with what it requires.
With the same process it is possible to select another encryption algorithm, for example moving to AES-512 or an eventual successor of AES.
While AES-256 is considered the best choice as encryption there is as of today, history taught us that over time newer encryption algorithms replace older ones as attack vectors are discovered. Your index is prepared for what the future brings.
Encrypted backups by design
Backups are yet another place where IT security may fail and private data may leak.
SearchCluster maintains a continuous hot backup according to configuration. The data in this backup is the same as in production, and since that data is encrypted, so is the backup. The hot backup is automatically kept up to date with key rotation, as each change to a record triggers the fresh backup event.
Also the off-site cold backup storage is safe. Same data, same encryption built-in. This data will fall behind with rotating keys, however Vaults can retain many previous keys, and backups of historical keys need to be kept in sync with historical data backups.
Encrypted data transfer
Data transfer is going through secure connections with HTTPs and TLS (successor of SSL), which mostly uses AES 256 as well behind the scenes.
Conclusion
Full data encryption keeps your people’s data safe, keeps the people’s privacy safe, protects your company’s reputation and prevents fines.
Always encrypted, always safe.
- When transferring data: encrypted
transport level encryption over HTTPs with AES-256 - When records sleep: encrypted
record level encryption with AES-256 - Hot backup: encrypted
as it is the same format as the live data - Data in cold backup: encrypted
as it is the same format as the live data - HDD and SSD Disk encryption
On Optimaize cloud: disk encryption with LUKS
<- Back to SearchCluster page.
Get in touch
I’m happy to support you in getting ready.
Sasha ARN
Solution Consulting
