There is a wide gap between storing data and making it useful, and it is getting worse with the growing volume of unstructured data. In this episode of On-Premise IT Podcast presented to you by Hammerspace, delegates Justin Warren and Chris Evans get together with Hammerspace’s Head of Global Marketing, Molly Presley, to drill into the pains of managing unstructured data, and learn how Hammperspace addresses them with data orchestration. Data orchestration takes the one-dimensional approach of storing data, to the next level that is cleaning, organizing, enriching and making data accessible across systems. It makes it possible to move large volumes of data across distances. Not bound by any one data or infrastructure type, data orchestration helps businesses handle new kinds of complex data and keep up with their changing uses.
Podcast: Play in new window | Download (Duration: 31:44 — 36.3MB)
Subscribe: Apple Podcasts | Spotify | Amazon Music | RSS | More
On-Premises for Today’s Spotlight Podcast:
Panelists
Justin Warren
Chris Evans
Hammerspace Panelist:
Molly Presley, Head of Global Marketing at Hammerspace. Connect with Molly on LinkedIn or learn more on the Hammerspace website.
Moderator
Stephen Foskett
Follow us on Twitter! AND SUBSCRIBE to our newsletter for more great coverage right in your inbox.
Key Points
Data orchestration involves the movement and utilization of data regardless of its location or format. The challenges posed by unstructured data and the rise of edge computing have emphasized the need for efficient data orchestration solutions. Over the past two decades, the transition has shifted from structured to unstructured data, requiring complex workflows and interconnections between different data types. Unstructured data, such as genomics data, microscopy data, and multimedia data, necessitates effective data orchestration for proper management and utilization.
Technology advancements have enabled the decoupling of metadata from individual storage systems, allowing for distributed orchestration and flexibility in leveraging unstructured data. The focus is now on leveraging technology to enable desired actions with data, rather than being constrained by traditional tools. The shift towards NoSQL databases and data lakes reflects the need to make both structured and unstructured data useful and break free from tool limitations. Efficient data orchestration enhances workflows by facilitating data transformations, metadata application, collaboration, archival, retrieval, and interaction, while allowing for flexibility in storage systems and applications.
Data orchestration removes friction and simplifies access to data by eliminating the need to remember specific storage systems or locations. It empowers users to repurpose or modify data efficiently. By separating the storage system from data management, a more flexible approach is achieved, where storage focuses on security, performance, and accessibility, while data policies and actions can be layered on top for effective organization and utilization. Skilled professionals, often referred to as data architects or librarians, play a vital role in managing and organizing information across different storage systems.
The longevity of data is an important consideration, as it often outlasts storage systems and applications. Automation plays a crucial role in ensuring data remains accessible even when the original creators are no longer present. Curation, similar to the role of librarians curating books, is essential for data management. Data curators ensure data quality, facilitate migration between mediums, and appropriately dispose of unnecessary or sensitive information. Frictionless access to data is critical, as its value diminishes when it cannot be easily accessed and utilized. Data management and data orchestration are interconnected, with orchestration facilitating the movement and presentation of data to applications and users while adhering to front-end and back-end policies. The complexity and volume of data necessitate a robust orchestration model to unlock its true value, and advancements in AI engines and commercial products are emerging to meet the evolving needs of data businesses.
