is data back -as-a-thing?

Recently had the opportunity to think about data management for the first time, long time.  Since Infrastructure as Code instigated the demise of what was a professional focus for half a career or so, I figured Data as Code was certainly coming right behind it.  In looking at where things are now, and working on Edge as Code as part of the Composable Edge, maybe I should have looked a bit deeper into just what Data as Code has become.  Since the Wikipedia entry for ‘Data as Code’ returns ‘Code as Data’, maybe I should have looked in on this sooner….

This is the first part of a three-part series on Data as Code.  This first part consists of reflections and some insights on how and where Data as Code might become a more rigorous discipline.  The second part addresses the types of requirements faced by those who need data and its value delivered more regularly.  The third part describes how the Data as Code market is a digital twin, first of digital transformation, then of he resulting modernized organization.  And back again.

In doing research, I found that Data as Code is coming, but probably slower along than Infrastructure as Code (which took ten years to become a commercial market).  Being nonplussed that since the time physical data storage solutions were focused on cloud, trust, and big data, that no significant Data as Code programs have been instituted, maybe there’s a here and now opportunity to help others, help themselves.

A few things to note, as today’s practitioners are confronted with many requirement characteristics which simply did not, and could not, exist previously.  Data management was never easy, and often never much done past the level of semi-structured data, file system data and database solutions.  The notion of a cross-data type, cross-application, and cross-organizational data management scheme has been attempted many times and never quite been able to catch on.  The data types themselves either changed or grew so fast that money allocated to organizing a data-centric inventory went to supporting the operation of the increased data volume.

That there has never been as much data as today and there will never be as little data as there is today again has been well remarked.  The challenges facing today’s developers, architects, and business owners is that as the pile of data increases, so does the technical debt for managing it and the opportunity cost loss of not monetarizing it as rapidly as possible. Or even not monetarizing it at all, comes to mind.

Part of this is that teams are so busy adjusting to macroeconomic events, business transformations, and staff limitations of various kinds that there is still no time to spend on managing the data for purposes of exploiting its intrinsic, and still mostly unrealized, value.  We know something about how to organize a data management practice (or, at least we did once upon a time), and we, too, never had the time or the sponsorship to much follow up on it.  For which, one would be far amiss in not recognizing one’s ownership of some of this, considering the degree to which data related products and services made for something of a career.

That, not aside, but in mind, may have led to this particular junction in the context of a composable edge.  It was mentioned that teams which rely on data sets today have many different considerations than were once the province of information infrastructure specialists.  Cloud was four types of presentation to end users, with five characteristic components.  Trust become security from end to end of a user’s experience, including sustaining the integrity of personal information in a compliance context. Big data’s five V-requirements became important in the context of omni-cloud solutions and cloud-core-edge business models. 

Since one cannot expect that all organizations can approach all these requirements at once, one is best to recommend as many paths to the increasingly useful management of data for the form, fit, and function of an organization’s transformation requirements.  There are three main trends which can be considered in order to create a data management practice for what should already have been a 21st century success story.

The trends are cloud-native computing, the convergence of 5G and edge computing to make most businesses part of the Internet of Things (or Everything) market, and the lack of data conversant specialists.  What we can do, is make sure that we are moving forward with providing reference architectures, best practices, and then combining those artifacts into use cases for end user adoption and blueprints for architecture practices.  There is evidence we can create a data management practice from correlating existing standards into a form that can inform the development of a Data as Code Center of Experience.

Part 2 continues this explication with an analysis of the requirements information infrastructure teams now face.  Both the requirements to be managed, and the data supporting those requirements are growing without measure, and more rapidly that ever.  It is to help orchestrate the automation of a response to these trends that any of this begins to make sense.