The goal of the data masking is to erase those characteristics that allow identifying sensitive data, making them anonymous but still usable. Thus, the risk of theft of sensitive information for the organization is eliminated.
The idea is that with the data masking We give developers access to quality data. They need them in production processes for testing purposes, but as we have seen in other posts, we cannot work there with the actual data. In this way we make sure that our sensitive data is safe.
Even though this seems simple, we may run into some problems. Let's look at a simple topic and four problems that we can find when we use techniques of data masking.
A simple theme: data obfuscation
Developers love working with production data. But today, due to privacy issues, companies are not very keen on giving developers full access to their data. Even for those taken from a production backup. They are very concerned about developers working with customer lists, credit card numbers, Birthday dates, addresses, Names, etc. and that they can subsequently market that data to the competition.
Companies want to use data masking techniques to quickly obfuscate all production data. They want to restore the production databases in development, run some procedure and let developers test their applications without seeing actual production data.
They don't want to obfuscate all the fields. As an example, financial transactions should retain monetary data similar to actual to aid verification of reports. We can't have random tax percentages like VAT, as an example, because we need them to behave predictably when looking at an invoice.
But as we say, this is the easy part of data masking. Obfuscating data is something that can be done with ease. But it has more complicated implications. Let's see them.
Trouble 1: storage profile maintenance with encrypted data
The easiest way to hide your data is to encrypt it. Despite this, data encryption tends to produce a totally different data size. If you intend to see an example in action, you can visit this page encryption demo. Click randomly to generate a key, type plain text and click Encrypt.
As you can see, the data is suddenly much larger. This may not be an obstacle for some types of data, but it is a big problem for fields with integers, dates and credit cards, as an example. Encrypting customer names suddenly makes each record much larger and changes the way queries are performed. Ideally, obfuscated data must be the same size as the original data.
Trouble 2: distribute statistics safely
In a typical phone book there are many people with the surname Garcia. If you look at a typical histogram of surname data, some surnames will have many records and others not many.
A database management system such as SQL Server generates statistics that have histograms showing the distribution of data in each column and then uses those statistics to create execution plans.. When we test SQL queries in development, we want to get similar variations.
It's easy to obfuscate data through data masking, just doing it randomly. If we have a date field, we just use a random number generator, but it will not have the same distribution as our original data.
Ideally, the obfuscated data should have a distribution similar to that of the original data. If we have people in a table 1000 records, all those who have the surname García must be obfuscated in the same way so that they have the same surname. If my people table has 500 people named Garcia and others 500 people with unique surnames, my obfuscated data must also have 501 unique different surnames, one of which will have 500 records in table.
Trouble 3: maintain referential integrity throughout data masking
Sometimes, private data is part of the primary key of the table. Database design ideas aside, the reality is that some people have things like their social security number as part of a primary key. Or even worse, some people don't put foreign key relationships in databases, so they end up with two social security number fields in two different tables that need to be joined.
We cannot trust foreign keys because many people do not use referential integrity in databases.
In addition, several databases may be involved. Sometimes, customers have customer data in a database, sales information in another and configuration data in another, and they all must be linked.
Ideally, the answer would be to determine joins where you could keep the same data in both tables, while allowing users to specify fields that bind even if the foreign keys are not specified in the database. This configuration must be created once and saved, so users don't have to repeat it every time they upgrade production.
Trouble 4: speed of data masking tools
These situations often involve large amounts of data and people want to upgrade development with an obfuscated production copy overnight.. This means that, as usual, it is not practical to export all your data to some kind of application server and then try again. It is also not practical to update the same table repeatedly, once for each column to be obfuscated.
Users want status updates to know approximately how much of the database has been obfuscated, how much is left to do and if it fails halfway to fix things and get back from where the scan tool left off. data masking.
