The Challenge

Federal and state regulations require that Greenhouse Gas (GHG) reports be submitted annually by businesses and other entities that fall under the U.S. EPA’s reporting requirements. Many states require that these reports comply with the format defined in the EPA’s GHG Reporting Program (GHGRP). 

However, this report can require up to forty-four subparts, all of which are in XML format, the schema for which can be pretty complex, including multiple-level nesting. Regardless of the complexity, states must make these reports easy to ingest and analyze, especially with increasing requirements and public interest in controlling and reducing greenhouse emissions.

Big Compass’s state government client needed to take in these reports, validate the main and subparts submitted by the reporting entities, add the data to the appropriate database, and facilitate public access to all non-confidential business information (CBI) contained in the reports. The solution needed to handle today’s potentially complex report formats while also being flexible enough to handle future schema changes and additions. The state agency also required a solution that could be quickly reproduced as new subparts were defined.

The Solution

The good news was that the solution pattern for this challenge is well known - take the XML document input, transform it, and load it into the back-end database for access by analytics tools such as Tableau. The bad news was that, due to the complexity of the GHG schemas, the solution is still time-consuming to implement and requires a thorough testing cycle to ensure data validity.

To implement this architectural paradigm, the state agency needed a platform that would support it and an API design that would meet the multiple requirements of the project. Big Compass worked alongside various state employees to architect, implement, and deploy the solution. Big Compass developed a solution based on API-led connectivity. This solution consists of an Experience API to interact with the Content Management solution and a System API to write the transformed data to the back-end database. The System API implements a MuleSoft component called “Dynamic Evaluate” that facilitates supporting and maintaining subparts without changing the System API design.

The Content Management solution, containing the XML reports, invokes the Experience API. The Experience API validates the XML schemas, queues XML documents to the System API, and sends an asynchronous status response to the content management system.

The System API then processes the mandatory GHG report subpart and determines what other subparts exist as part of the report and are supported by the reporting solution. The subpart(s) are processed, the data is upserted into the database, and updated processing status is queued for the Experience API.

MuleSoft’s API Manager is used to secure access to the Experience API.  Additionally, an out-of-the-box XML Threat Protection policy was applied at the API Manager layer to ensure malicious code isn’t embedded in the source XML, which could be inserted into the persistent store or pushed out to those analytics tools that rely on that database.

After architecting the complete API solution, Big Compass developed the initial GHG report subpart. With the first subpart implemented, others within the state agency used it as a pattern to continue the rapid development of the additional subparts. This also gave the state employees hands-on experience with the API development so that new subparts or future modifications could be implemented without additional third-party support.

The Results

The MuleSoft based, API-led solution enabled the state to work efficiently with the EPA and their schema requirements and improve the ability to adopt any future subpart requirement changes.