The Shim Solution: How I Integrated An External Digital Asset Management System
One of the most intriguing aspects of software development is the different types of problems developers meet each and every day. When faced with engineering challenges, different people will come up with different ways to solve them. My favorite part of being an engineer on the Pica9 development team is collaborating with team members to discuss issues, come up with solutions and evaluate the pros and cons of each method.
In a recent project, I implemented a Digital Asset Management (DAM) Shim that enabled seamless integration between CampaignDrive and an External DAM (SalDAM). There were numerous challenges to be solved.
- Problem
Due to the disparate business needs of the two systems, not all assets from SalDAM are useable in CampaignDrive. (An asset can be an image, a video, a Word document, an executable, a zip file, etc.) For example, when 6 assets on the first page of SalDAM are retrieved, it is possible that CampaignDrive will discard the first and the last assets because of invalid MIME types, unsupported image dimensions, or other reasons. As a result, though CampaignDrive requests for 6 assets, only 4 assets will be useable. Not having the correct number of assets caused problems with user interface components and pagination issues.
- Solution
To improve the user experience so that the user is not shown pages with empty spaces (or even empty pages), I implemented the DAM Shim. The Shim automatically queried SalDAM to find assets to fill the “empty” spaces on the pages within CampaignDrive. This search for assets continued until enough valid assets were found. I also had to make sure that when the user requested the “Next Page” of assets, the search started only from where it was left off so the same assets don’t show up on the following pages again. In other words, the search for each page is highly dependent on the result of the previous search.
- Evaluation
Although the DAM Shim solution solves the integration friction problem, we have to recognize the cost of it and the risks it introduces. First, because the search for almost all pages of assets (except for the first page) is based on the previous page, it is technically very expensive to query pages discontinuously. Hence we disabled the feature to skip to an arbitrary page. Second, performance can become an issue since what was previously costing one request to the external DAM now could take multiple requests (the number of which is unpredictable). In the worst scenario, the search requests would take so long that the browser timeout limit is reached. An infinite spinning wheel on the UI is clearly not a good user experience!
- Summary
After discussions with the internal teams and evaluating the different options with the client, we decided the Shim solution was the most viable and cost-efficient. To alleviate the user experience drawbacks caused by the DAM Shim, I also introduced enhancements such as front-end LRU Cache and Eager Loading. Other potential enhancements included implementing server-side cache and congruously increasing the query page size, but, in the end, these were not implemented as testing showed that the front-end enhancements were sufficient.
