What is Dark Data?

While it sounds a bit like a reference to a scene from a Star Wars movie, Dark Data actually refers to the storage of business data. Should you know about it? The answer is that you probably should if you use computer technology and digital data storage in your business. The term is also called “dusty data,” and understanding what it is and how to manage it may affect the profit and security of your company.

Defining the Term

The concept is not uniformly defined, but this, from Techopedia, comes close to giving a picture of what is meant by dusty data. It is “unstructured, untagged data that is found in data repositories and has not been analyzed or processed.” In other words, it is data that companies keep “just in case.” Examples are customer records and information, outdated patient records, old financial statements, former employee information, email communications and so on.

Issues in the Data

Imagine that you completed a construction project and have leftover materials. Throwing them away doesn’t make sense; you might be able to use them in a future project. If you throw them away, you will just have to buy them “again.” So you decide to keep the materials, but now you must decide where to store them. You can pile them in an attic but, after amassing materials from several projects, you begin to run out of space. That is the first issue with this “dusty data.” It takes up valuable storage space that your pertinent data needs. Discarding the information, however, means you will not have access to it if you need it in the future. Storage, even of digital data, has costs that affect your profits. Another issue is the security of the data. When so much information is amassed and left unformatted and unprotected, hackers can access data easily, sometimes without immediate detection. Embarrassing information could be discovered and “leaked,” product development information could be recovered that could surface with a competitor and the discovery of old patient records could even lead to legal liabilities for your business. Still, that information might be important in the future and should not be destroyed. if your company comes out with a new product that could be attractive to some of your former customers, for instance, having their information at hand is less costly than researching them again.

Addressing the Problems

One of the first things you can, and should, do to manage your dusty data is to prune it. You should ascertain which data should be kept and which is useless. Going through the retained information, getting it into files that you can recognize and eliminating duplicate data should be done one time. After that, you can perform regular “maintenance” on the data. You can also store the information in formats that take less room in your storage. The final recommendation of virtually all authorities is encryption. All stored data should be encrypted. That includes information on company machines, in personal computers and even in cloud storage.

Companies today collect and store immense amounts of data from customer profiles to patent information and even old resource records. Some of this information could become important in future projects, but much of it is left unanalyzed and unorganized to “gather dust.” Being able to access and use old data is like finding money in a suit coat pocket. That is why the management of Dark Data is vital to business and government today.