If your DevOps team is the rocket engine for your agile delivery processes, then effective collaborative data management should be your rocket fuel.  -Brent Reed

As IT departments continue to embrace DevOps, the lifeblood of our products and services is data.  Thus, we need to include in our “way of working” good agile data practices. By the way, Scott Ambler Co-Founder of DA wrote the Data Modeling and Agile Data book on this and blogged this eons ago in much more detail.

For this blog, I am going to suggest three areas to help improve the ability to bring the concept of agile data into your delivery and operations.  

Stage 1: Agile Data Modeling

When we start the process of a shared vision (what PMI DA calls the goal “Common Vision), of what we want to achieve, it is essential to ensure we include goals, decisions and the options we desire or may want to explore. The team should be open to doing some early light data modelling with data experts and support staff.  

• During your initial discussion, such as requirements, why not model the system using a current and future state (i.e. high-level UML diagram). 
• Next, explore data entities and structures. Start from the domain level and iterate through the logical and physical models over time, the lower level semantics required to develop the next level of detail. This kind of data modelling is what I refer to as “Evolutionary Data Modeling.”
• Collaborate and model collectively, with not just the architects, include your administrators, product owner, help desk and others as you develop the models

There are many reasons why the above points are what I believe are essential data modelling strategies. For example, the first point when discussed, even in a short modelling conversation, can uncover the need to consider, various types of data, locations, legacy or new data sets, need to refactor or replace, the need for flat or cube data, data warehouse and business intelligence concerns. Thus a connection and feedback loop are essential to unlocking the right conversations when doing agile projects and especially critical to DevOps becoming a maturing and the rocket engine to your speed to market for business value, and hypothesis-driven design. Don’t get disrupted, & replaced, you need agile data and business agility to keep ahead of the competition (or a lurking competitor).

Figure 1 Data Management & collaboration equals success

Stage 2: Agile Data Management

When considering your data objectives using a set of hypothesized or validated goals for your product, a Data Management Goal in the image below, is a great start:

Figure 2 Data Management Goals 

The above figure includes a fair amount to ingest & understand.   However, there is little doubt that effective, collaborative data management opens discussions and evolves many of the agile delivery and DevOps practices not yet realized.   Thus, a focus on data management goals begins the conversation to a strategy that is enterprise aware and includes patterns for reuse of information services. this approach helps us to shift left focusing on value, can reduce costs, technical debt, and increase quality and value in a very short time.  Planning data management is a must, and its part of the rocket fuel required, for high-performance teams in DevOps.

Stage 3: Connections, Feedback and the big picture

When we apply disciplined agile to our way of working, the feedback and continuous improvement goal is included and thus already baked into the project cycle. This great, and using this process by understanding the feedback loops, especially around our subject of discussion (Data & Information), is connected to the “enterprise awareness” DA principle that is key to unlocking agile data. Thus we need to reflect on our understanding of our internal and external lines of communication. I cannot emphasize how vital knowing whom our stakeholders are and identifying how and when we engage with them because the nature of agile data is “highly collaborative.” This means that the agile data strategy approach is very different than the traditional data strategy approach because of the nature of agile being flexible and the connection to DevOps, which looks at continuous “flow,” “feedback,” and “learning.” Considering these points also mean that we should model early on in our agile way of working to identify and develop streamlined ways of getting information to and from our stakeholders. This “streamlining” of data conversations works in harmony with the natural approach to speed and flexibility with the agile delivery methods and support the maturity and value of DevOps. Thus with the agile data strategy approach of documenting incremental change though what Scott Ambler calls “JBGE” just barely good enough lean documentation and governance, we can keep the velocity needed while retaining the right amount of information and context in our delivery process. Once again looking at the diagram below we still document change lightly and ensure we are transparently sharing and collaborating with our stakeholders using “lo-fi” a’ la whiteboard and photo, modelling techniques initially and then decide if a further value gained by using “high-fi” or digital modelling tools and techniques.  

There is a ton of concepts here; however, nowhere near as much as I could have covered, that is why we provide training and coaching at Tactec. While this blog is way beyond my intention to go into more levels of detail such as adoption, the transformation from traditional to agile and the big agile data picture, I hope I have been successful in introducing you to the subject of agile data, along with important data modelling concepts. I do plan on putting up a blog extending on the adoption of agile data within your organization; so, please reach out to us at [email protected] for a free hour of discussion and deeper dive. Additionally, please let me know your suggestions or anything you would like to share during your path to agile data and agile modelling.

You can, however, find out all this goodness here from the following references:

• Agile Data (big picture) from Scott Ambler who called this out over 15+ years ago!
• Disciplined Agile (the complete body of knowledge on Agile)
• OpenSource (Brents articles)