What We Do
Who We Are
Industries
Technologies
AWS
Azure
Boomi
Confluent
MuleSoft
Snowflake
Software AG
Events
Insights
INSIGHTS
Blog Posts
Industry News and Updates
Case Studies
Recent Projects
Offerings
Project Accelerators
White Papers & eBooks
Insights and Education
Contact
Contact Us

Logistics Leader Turns System Logging Data into Revenue Stream with Snowflake and AWS

AWS
MuleSoft
Snowflake
Design & Development
Customer Scenario

Digital transformation provides innovation and creates new opportunities. A logistics industry client decided to leverage their ecosystem to monetize their data and to fine-tune their operations.

The client was already using AWS, DynamoDB, and MuleSoft. What they needed was a way to ingest and store data around business information like shipments, loads, invoices, acknowledgments, tenders, and so on and share that information internally and with clients. Doing so would promote better, data-backed business decisions for the company and provide a new product that could be sold to customers.

Solution

The first order of business was to create a place to store the data to make it accessible without requiring significant transformation and could be used with the existing systems. The decision was made to create a Data Lake to accommodate the structured and unstructured data.

For this to be a product, the data needed to be shareable with clients. The decision was made to implement Snowflake for the project. Snowflake was already known to work well with AWS and would slot into the existing architecture nicely.

Creating a logging framework for the data handling and processing drew on Big Compass's previous experience with other clients where we'd developed similar frameworks and allowed us to optimize the solution from the start. The architecture was designed to load the logs into AWS S3 from different systems using MuleSoft to AWS SQS. Once in S3, the files were aggregated using an AWS Lambda function to temporarily load the logs into DynamoDB. The extracted logs could then be broken into 10-megabyte files for streamlined ingestion through Snowflake's Snowpipe API. Lastly, Looker sat on top of the Snowflake data to create consumable dashboards for customers and the internal team. 

Results

Once the solution was in place, customers were able to add the data service to their accounts to get a clearer picture of their usage of the company's services. Customers could perform data analysis on a broad range of data points across the logistics services they used and gain greater insight into the files shared with our client.

Additionally, the new solution allowed internal teams to gain a better understanding of their own business. The log processing provided executives, support teams, and other internal users with the opportunity to create dashboards for analysis and surface errors and process faults faster. In the end, the new logging framework enabled the company to drive decisions based on the wealth of data they were already gathering, furthering the value of their system investments.

Big Compass
subscribe

Sign up to stay on top of the latest news.

We've added you to the list!
Something went wrong. Please try again.
recent Case Studies
From Mule 3 to Mule 4: A Government Agency's Journey Towards Secure Integrations to Ensure Safe Water and Flood Control
Leader in Children’s Play Kits Integrates NetSuite, Shopify, and Boomi to Modernize and Support International Operations
MuleSoft Migration Streamlines and Simplifies ERP Business Processes for Manufacturing Client
View All
No items found.

BUILDING CONNECTIONS IN A DISCONNECTED WORLD

Company
What We DoWho We AreEventsCareersContact us
technologies
AWS
Azure
Boomi
Confluent
MuleSoft
Snowflake
Software AG
insights
Blog Posts
Case Studies
Offerings
White Papers & eBooks
subscribe
Welcome to the Big Compass family!
Oops! Something went wrong while submitting the form. Please try again.
© 2020 Big Compass
  |  
Privacy Policy
  |  
Terms of Service