The outbreak of COVID-19 has forced companies to focus on their digital strategy and, in many cases, accelerate it. As firms across many industries rapidly switch to remote working, the question about how to protect their data has become more prevalent. Encryption is an integral step to protecting data from unauthorised users who may try to read or corrupt confidential data.
Users will, therefore, welcome the new encryption features available with the recent kdb+ v4.0 release. Combined with OpenSSL v1.1.1, the latest version of kdb+ allows for the encryption of individual files on disk, giving kdb+ architects the ability to satisfy data protection regulations that were previously difficult to achieve, and it’s relatively straightforward to use.
Encryption in kdb+ uses one algorithm (AES256CBC), which is identified as algo 16. It’s applied using a similar code format to compressing data. Once the master key is loaded into a q process (via -36!), data can be easily encrypted and/or compressed by using set:
q)-36!(`:/path/to/your/mk.key;"**********"); // load masterkey+password
q)(`:encrTestFile;17;16;0) set asc 10000?`3; // encrypt file
q)(`:encrCompFile;17;1+16;0) set asc 10000?`3; // compress + encrypt file
Architects can choose to apply encryption across entire databases by default or specify certain tables and columns that require encryption. It carries a small overheard regarding on-disk storage (<2%). However, there will be an impact on query performance during decryption, so the kdb+ architect should explore and analyse the flexibility offered by partial encryption.
It’s also possible to combine encryption with compression, and this can somewhat negate the performance impact, and it’s easily done within the same line of code. Read our white paper to see how to apply encryption and/or compression in various ways, and how each of those affects write-to-disk and query performance.
New: Data-at-Rest Encryption: Explore the options available with the kdb+ v4.0 release
Encryption is integral to protecting data from unauthorized users that try to read or corrupt confidential data.Our Data-at-Rest whitepaper explores encryption with the recent kdb+ v4.0 release; how it works, how to use it, and compares the performance of different techniques available, including encrypting and compressing simultaneously.