Big Data, Big Changes. Few changes in the history of data management have been as sweeping or as disruptive as the phenomenon known as big data. The term generally refers to the avalanche of data now available to help organizations shape strategy, optimize operations, and identify customer tendencies and proclivities. Big data is the fuel for the new digital organizations that power decisions and optimize outcomes. Big data comes in all shapes and sizes. There is the traditional transaction data-or structured data that fits neatly into rows and columns-that has been the currency of business operations for the past 50 years. Then there is the more unruly semi-structured data, emanating from application servers, sensor fields, email systems, and document systems, that mixes structured fields and tags with long strings of text or code. And finally, there is completely unstructured data, such as audio, video, and image files that consists mostly of undifferentiated bit streams.

Big data really is a big deal The opportunities of insight, innovation, and impact are virtually unlimited. Realizing the opportunities, however, depends largely on managing the data well through the entire lifecycle from ingestion to application. That’s a tall order with many moving parts-data, metadata, storage, processing, quality, security, search, preparation, analysis, and more. Big data management involves both management practices for big data and big processes for data management. Big data is more than just NoSQL and unstructured data. It encompasses the entire data hub-data lake, data warehouse, data marts, master data, and ODS-as well as distribution of that dat to sandboxes and local data extracts. All data of enterprise interest, internal or external, that needs to be shared is big data. And big data must be managed.