Defining “Design Management”
Often I hear comments like “we don’t need to maintain a design document as the code has all we need” or “well we have this spreadsheet which has where the data comes from – but it’s going to be out of date”. Or perhaps even worse is when someone has documents, diagrams and spreadsheets which they think are accurate, and then you look and you find they are not.
Let me be clear – thinking that the code is sufficient to document the design is a mistake!
Code is focused on getting something done – the actual way it is executed. Sure you can have nicely defined interfaces, you can have high quality documentation added by developers and you can have enforced coding patterns to aid understanding of the code. But reality is that if it’s not essential – then it will not be maintained. It doesn’t matter if you are writing complex algorithms and business logic, or services and APIs. Code is still code. Developers are still developers. Project deadlines are still project deadlines.
What so often happens when an API or service is being created without design management is that things start out with business people identifying a need or opportunity. They write that up – perhaps in a word document. They pass this to some business analyst who does some sanity checking and then works out where the data they need is going to come from. They write that up – usually in a spreadsheet. A number of other people may get involved checking things such as security. They may add to the spreadsheet. They then pass this onto a development team who then takes the word document(s) and spreadsheet(s) and start defining the API shape – such as a swagger document and generating code, or writing some code and generating swagger from that. They find that reality is a little (or lot) different from what’s described in the documents that have been given – so they discuss it, make changes to the source, and continue. Rarely will the documents they received be updated. They then test and finally deploy to the runtime and to developer portal(s). But the important design information is either not present or out of date – so the only way you can really know what’s happening is by looking at the code… and who other than developers can do that? What if those developers are gone? How does the business know what it really has – how does it reuse APIs with confidence? Does it even look to reuse APIs or just build new ones every time, adding to more and more integration spaghetti?
API Design Management is about ensuring that all the information required to actually build services and APIs is shared, up-to date, discoverable and is “executable”.
- It aims to ensure that business and technical staff can work together using concepts and language they each understand (not forcing them to learn a new common language).
- Rather than having information stored all over the place, in emails, spreadsheets, word documents, this “design” is maintained in a shared platform, accessible to all. An API portfolio includes all your requirements, traceability, governance. It contains the end to end process with canonical model management, design and discoverability.
- It allows business people to describe services and APIs in terms they understand, without forcing technical people to duplicate that information – the technical people can focus instead on making the APIs work, and work well. Having things described in business terms means they are inherently discoverable – so when the business is initially identifying a need or opportunity – they can see if the information is already exposed either fully or partially… reducing duplication and IT costs.
- And having the information in a known format means that it can be used to generate much of the build and runtime artefacts needed for services and APIs… such as interface stubs, control logic, mapping code, test cases, flow and sequence diagrams, etc… For example if it is easier to add something to the design and generate an all new test suite, than to add to the code and manually write the new test cases – development teams will want to update the design rather than regard it as a necessary evil.
Why is the big picture important?
Often development of an API is done at a project level – and is focused on the needs of the project. That is fine for the project, but the business as a whole will likely suffer. It is often better to reuse a similar API than to build from scratch. Even if that API needs to be extended or the output transformed into a slightly different format, it is going to be less effort than developing AND MAINTAINING a mostly duplicate API. One approach is to provide “enterprise services” which would be designed by central teams of experts which all projects can use. This often fails as the project needs are rarely fully met. Another approach focuses on projects developing their own services. This has been much more successful in delivering the projects but builds up much technical debt. With the right platform to provide the enterprise view of the project APIs (including things such as data lineage so you know the actual information is what you need) project teams can benefit from other projects, and central teams can review and refine APIs built by the projects – reducing any technical debt and helping projects be quicker through reuse.
Isn’t the runtime enough?
To be used an API or service needs to be deployed and running somewhere – so doesn’t it make sense that that’s the source of truth? The runtime is rightly focused on where and how the API is running… all that is important really is the interface and possibly the source code if it’s hosting the service rather than proxying it. Subscriptions, security, throttling, monitoring – these are all things the runtime needs to manage. All good stuff from the perspective of keeping your APIs running. If the runtime is paired with a developer portal – then the interface information as to how to use the APIs will be available.
But that’s where it stops.
You are unlikely to have any data lineage information. You are unlikely to know how APIs connect to each other. You won’t know the orchestration logic unless someone has written a comment in the description. You won’t see what’s currently planned or know the reason why an API is the way it is. You won’t be able to work out how much effort it would be to change an API. You won’t be able to work out the impact of upgrading a backend system or even changing an existing API beyond its immediate subscribers. You don’t know the business language associated with the API unless someone has bothered to enter it in a description. You don’t know how an API supports the business initiatives and capabilities just by looking at the runtime. So in short the runtime has plenty of interesting and useful information, but it all is focused on the runtime and you’ll miss important non-runtime information.