API Abstraction: Building Processes in the SaaS Landscape to Scale
General knowledge of key concepts recommended prior to reading:
APIs, System Integration, SaaS Sytems / Cloud Architecture, Abstraction Layer
Introduction
Software as a Service (SaaS) systems are ubiquitous among the hottest start-ups of today. They enable agility, scale-ability, and offer a low start up cost to small and growing businesses. With so many applications comes the task of managing all the integrations and their different APIs. When managing a few systems this task isn’t an issue, but when more and more systems are added the complexity increases. This is where utilizing a micro-services and an API Abstraction Layer based architecture helps.
API Management and Abstracted APIs
API Management creates an API layer to run all of your internal processes (or integrations) and potentially external processes. API management allows for you to develop, version, secure,and publish API endpoints.
Abstracted APIs paired with API Management allows for your company to scale and improve existing Application APIs. In Figure 1, Processes, Users, and Applications all authenticate into the Abstracted Get Account API. The API then maps from the Abstracted GET Account API to the SaaS GET Account API, authenticates, and sends the request to the SaaS API. We then take the response and we create our own response profile from that and return it to the consumer of the API. This allows for control of what data is visible and how it is formatted to be sent back to the Processes, Users, and Applications.
Abstracted APIs create a “wrapper” around the SaaS or on-premise application API. This allows for you to control who is accessing what, where, when, and how. There are many API Management vendors who can do this for you or you can create and host the APIs yourself. Figure 2 shows an example of how the architectural concept works by using Abstracted APIs (blue) on top of Application APIs (orange). The blue components are what a user, process, or custom application will see or use, while the orange is the backend and what the SaaS application needs in order to perform the task. This creates a layer of abstraction to control what information is being consumed and how it is requested.
Benefits
Data Quality
One of the biggest benefits is the ability to control the data quality of the APIs that you are using. In Figure 2 we utilize a dateConvert Function to ensure that all dates are in a consistent format. While this only shows the mapping from the Application to the Abstracted API, you can pull in data from multiple sources and push certain pieces to each element. This is also a place where you can put an Master Data Management (MDM) solution and standardize objects.
When running processes on Abstracted APIs, it allows for the business users to trust that the data will be in a consistent format and contains the information they need to build business logic on-top of the data.
Dealing with Changes
Change can be one of the biggest pain to deal with. In the SaaS world things constantly change, with Abstracted APIs you can limit the effort required because of those changes. Some examples of changes and effect on Abstracted APIs:
- New Fields Added/Deleted to SaaS application — Users don’t notice this since they are using the Company API. When you want to add or remove new fields, you simply add/remove them to the JSON profile that is already being used by all of the processes.
- New SaaS System — Having to change systems can be a huge risk and cost to organizations. When you change applications, the application API changes along with it; however processes and objects generally don’t change. So when changing SaaS applications the only mapping that needs to be changed is the mapping from the Application API to your Abstracted API. All the downstream processes and applications are already using your Abstracted API so there will be little to no change needed. This is HUGE for any API that is consumed heavily by internal and external processes because it allows for consistency and agility.
- Mergers/Acquisitions — You can easily process data from other systems in a merger or acquisition. All that needs to be done is the mapping between other companies system APIs and the Abstracted APIs you've created. Then data can start flowing into your processes and systems.
Reusability
This is the one of the biggest benefits of this approach. It creates reusable internal assets or micro-services that can be used by different processes, applications, or users. Once you create an Abstracted API, you can expose it as an External API to customers, vendors, partners, and internal users without providing direct access to your internal systems. As shown below, the GET Expense Reports Abstracted API is used in both the Pay Expense Reports and Expense Reports Status process. This is because it returns a consistent message that can be utilized by both processes. If the source application changes to a database instead of a SaaS Application, the processes still run as if nothing has changed. All that needs to be configured is how the Abstracted API Layer maps to the new database.
One API
APIs are either REST or SOAP, XML or JSON, OAuth or HTTP Basic Auth, and each has different API methods that you use. In other words, each API is different in how you authenticate, interact, and consume them.
When you create Abstracted APIs, this allows for you to control what data is returned. With SaaS applications you have to abide by their rules and methods. Abstracted APIs create a similar experience for how you consume and use them. This makes it so that process developers only need to know “one api”, so that more time can be focused on creating and delivering solutions and less on “How do I authenticate/use this application’s APIs?”.
Security
By taking this approach you have greater control over your APIs and what users can/cant do. For each SaaS application that you have, you need to configure API users and their level of access (some systems are all or none). This needs to be done for multiple users in multiple places and can be hard to manage with everything being distributed to the applications. Instead using API Management, you can control all users and what they should have access to in one location. It essentially creates an API Gateway for you to manage your SaaS application APIs from one place.
Drawbacks
Initial Setup Cost
The biggest drawback is the initial setup cost. It is definitely slower to develop than doing a straight mapping from application to application. However, once these Abstracted APIs are developed, they can be reused, extended to external users, and are flexible to change. So instead of wasting more time later on when changing applications or building another integration to the same object, you invest that time now into creating Abstracted APIs and can reap the benefits of them.
Increased Latency
Another criticism of this approach is the increased latency for processes to run because they have to hit 2 APIs (API Abstraction Layer API and then the Application API). This depends on what is done between the Abstracted API and the Application API, the server or API Management vendor which the Abstracted API is running on, and the application API. There are a few ways to mitigate this which all depending on your circumstances. In my experience, I have not noticed a negative impact to performance with this approach.
Conclusion
APIs enable businesses to automate processes and build custom applications. Abstracted APIs provide a way for your organization to create reusable assets, be resilient to change, and improve data quality for internal and external processes. This approach provides many time saving opportunities and the ability to easily scale processes. While this might not be the best approach for every application or company, it should definitely be considered when using APIs that are heavily consumed.
Please comment below any questions, comments, or concerns that you may have with this approach and I will try my best to answer them (either in comment or in another article if there is enough interest). Thank you for reading!
