This post is for anyone who designs, models, produces, or consumes services (APIs, events, messages, etc.) at a large enterprise.
A Canonical Data Model and canonical schema is designed to define business entities in a standard manner, including the structure, attributes and data format. As a result, the models tend to be very large and often complex” -Canonical Data Models & Microservices, Deloitte Digital
Today we’re going to talk about the oasis that occurs somewhere between two extremes; the first being Canonical Canyon. This is a place where developers and engineers get so bogged down by rigid canonical models that there’s hardly room for iteration (much less, innovation); the second — the Wild, Wild West; where there are no rules, code runs wild, and duplication is out of control.
If you’ll humor us as we run with the extended metaphor, we’ll explore the two polaric states of information models before touching on that most-wanted middleground — where domain-driven design can be applied to canonical for the global enterprise, for any user, from software engineer to citizen developer.
The Wild, Wild, West: No Canonical, Ignored Canonical, or Too Many Canonicals
Let’s start with “The Wild, Wild, West”. This is the state you could declare in an IT department with little or no standard naming conventions, and if there are guidelines, they’re often ignored or locked away in silos far from their users.
Even organizations that keep a tight ship can lack development standards across business domains, but the smallest changes made can turn into an enormous bramble of unchecked code as the years go by.
This operational state is increasingly less favorable for security reasons, yet it is often simply the product of mergers and acquisitions, changes in technology/methodology, and too many departmental walls blocking the view. In the Wild West state, everyone gets tied up trying to meet deadlines, and those who do get assigned to monitor don’t even have the data lineage to even begin tackling the issue.
The side effects of this behavior, which can go on for decades, range in structure. They can look like one large mass of code so huge the application is slow and data is hard for users to access, or on the other hand, it could take the shape of 12 models, all doing the exact same thing — sometimes you end up with duplicate APIs and services that don’t even look like duplicates, since they’re not using consistent models.
It’s easy to see how the state becomes a big mess, with people working in silos and using whatever they have access to, without any governance in place. At a smaller company, this wouldn’t be such an issue. But for the global companies who have different teams building in different languages, it’s a nightmare.
So, how can you ensure the right models are being used for API development or consumption if everyone has access to all of these different flavors or versions of the models? Keep reading; we’ll get there.
Canonical Canyon: One Canonical, Low Flexibility
In this state, there is a source of truth. But it’s more brick than elastic, if you know what we mean. This is the state where a company’s canonical model is so rigid and lacking of flexibility that internal developer consumers can barely use it — not to mention the difficulties of onboarding an external consumer.
The problem with a rigid canonical model is it has no granularity, no user-friendliness, and gets as much use as a pretty postcard. It’s nice… to look at.
Thanks to the SOA days, many companies have beautiful canonical models, built upon company-wide efforts to document a service-oriented architecture that failed them in scale. Those models are often collecting dust in storage. But those are some of our most organized customers.
Let’s face it, developing a service takes enough understanding in its own right. When your guide to building that service, your canonical model, is so big that you need a data modeler to explain it to you — end even then, if you’re needing to know the language of a data modeler in order to even read the model, something isn’t working.
If there’s no way to discover and find the models you’re looking for yourself, you’re going to hit a bottleneck where you’ll have to go back to a data modeler to find everything. And guess what? They’re booked up, because everyone in your organization is in the same canonical canyon. Hope you brought picnic supplies.
It’s time for organizations to think, if we are going to adopt a canonical model for scale (and we should), how do we make it easy for everyone to understand?
The Oasis: Adaptable Canonical Model
The goal is to have flexibility — you want to make sure the model isn’t constraining you. But not too much — there has to be a middlegroud! It needs to be able to reflect what you’re doing in the real world.
What we’ve built into our platform keeps the largest organizations in that middleground. ignite helps you put the governance around and make decisions around how model(s) should be created, or decoupled, depending on what shape they’re in and what the future use is, whether you’re trying to build events, messages, services, etc.
But more importantly, we work with you to make use of what you have (SOA, etc.). We bring everything in, data, services, mappings, to analyze and make those decisions based on your set up. From there, we decouple the monolithic — so your services are getting federated and your model is getting simpler and simpler; and easier to use; self-serviceable. The better for when you’re rolling those models out to the enterprise.
In this state, governance doesn’t cause bottlenecks, it removes them. When developers and API producers take OpenAPI and hack it, they’re the only people who know about their changes. Yet when you have visibility on what’s happening within the underlying model; you know how you got from the model to where you are, you have lineage. You understand the impact analysis of every iteration. That’s what ignite offers, a way to first learn what you have so you always know exactly where to start and how to build.
Read Part 2: Applying DDD to Canonical Models for a deeper dive on how customers use ignite to achieve a Canonical oasis between the extreme states we covered today.