Supply chain management (SCM) has evolved over the years, and the globalization of trade and commerce has challenged organizations to develop more optimized, reliable and scalable supply chain disciplines, which include supply planning, product planning, demand planning, sales and operations planning and supply management. 

Supply chains continue to evolve

All of these disciplines are interconnected. Gone are the days of examining, every month, what products to build, what store to stock the most for specific items, which products to mark down and projected demand. Consumers and businesses are now driving the market to deliver high-quality products on shorter and shorter notice and at the lowest cost. Same-day delivery of goods and services used to be far-fetched but it’s here today with services like Amazon Prime. Amazon.com generates over $1B in revenue per day. Other vendors like SAP, Oracle and many others are also improving their solutions to meet the new business demand. 

Social media has also influenced companies to improve their ability to provide the best experience possible. With all of these developments, organizations must find ways to make better and faster decisions about everything from raw materials to product delivery. 

Everyone wants to break down siloed systems within their SCM and achieve the following goals.

• Enable real-time supply chain planning
• Unify data coming from the supply chain, sales and operations, and financial planning
• Take advantage of AI/ML to predict customer demand
• Leverage real-time data across all points of the supply chain
• Adapt to changes quickly

Legacy technology remains a challenge

Legacy systems that businesses still use today are typically siloed. For the business stakeholders to fully understand the health of their supply chain, a diverse group of data and technical disciplines within the company must extract the data from different sources, build complicated, brittle ETL pipelines, aggregate data into models and publish reports to BI and analytics tools. The entire process is extremely time-consuming and expensive to maintain, taking weeks to months to go live. Changes to any part of that process can take weeks.

By contrast, the need to quickly identify and act on insights only continues to grow. A modern data infrastructure must leverage the power of automation, performance, reliability and scalability, as well as modularity and flexiblity. This means ingesting, storing and analyzing data at high volumes with minimal interruption.

Consider how IoT is heavily used today in manufacturing, the delivery of goods and retail stores, producing huge volumes of sensor data in real time. It’s a key component of the overall supply chain management system, allowing analysts to detect early misconfiguration of equipment in production lines, which affects yield. Downstream of factories, consider how refrigeration systems in truck fleets can be monitored to prevent goods from spoiling before delivery. An entire fleet can be easily maintained using visibility into real-time truck health.

With rapid progress in AI and data infrastructures, organizations can gain real-time observability into their systems and detect unusual patterns that affect their daily operations. 

Accelerating innovation and modernizing SCM with Fivetran and Dremio

Fivetran and Dremio are driving innovation in supply chain management analytics. Fivetran is an automated data movement platform providing a range of deployment options so that an entire data pipeline can be created and deployed within minutes. With over 500 native connectors and many data sources used in the supply chain such as databases, ERPs, CRMs, event streams and file stores, Fivetran enables organizations to self-serve data pipelines quickly, reliably and securely. 

Dremio is the Unified Analytics Platform for Self-Service Analytics and AI. It provides a low-cost, high-performance SQL Query Engine and scalable Apache Iceberg Lakehouse Management service. Organizations can use their existing BI tools to analyze data and take advantage of Dremio's scale and performance. 

The combination of Fivetran and Dremio provides real-time supply chain management analytics and enables organizations to shift from reactive to proactive. With Iceberg, data teams no longer have to worry about changing data partitions and evolving schema, which is very costly to implement and impacts their ability to release data products on time. Fivetran captures all datasets from supply chain planning, sales and operations, financial planning, product planning, demand planning and supply management and loads all data to Iceberg tables. From there, Dremio users can understand their overall operations and find patterns that enable them to reduce cost, improve revenue and deliver quality products on time. 

An architectural viewpoint

Organizations can operate without interruption by extracting data from different systems from the various SCM data sources and landing them in an Iceberg storage format that promotes schema evolution and dynamic partitioning. Business stakeholders can get a 360° view of their entire supply chain management operations in real-time. 

Organizations are guaranteed a reliable solution that can scale with ever-growing demand and ever-changing business needs.

Fivetran + Dremio = supply chain insights

The value organizations realize from modern solutions like Dremio and Fivetran is substantial. 

• Enable revenue-generating decisions to happen on a real-time basis rather than weekly or monthly
• Unify all data sources including supply chain, sales and operations and financial planning
• Allow for adoption of AI/ML applications to predict customer demand
• Adapt to changes quickly

There are also several technical value drivers to consider with Fivetran and Dremio:

• Accommodate changing data dimensions quickly with minimal to no business interruptions
• Fresh data is always immediately available for analysis
• One place to query all data using SQL or Python
• Lower cost to extract, load and analyze data at scale
• 500+ connectors (covering just about any supply chain source including databases, applications, file systems and event-driven systems)
• No-code, self-serve and connect in minutes
• Simplified and standardized architecture
• Fully automated and fully managed 
• Multiple deployment options from cloud to private clouds and on-premises
• Log-based, log-free and agent-based CDC data replication for RDBMS
• Centralized global security and governance
• Reduced complexity = reduced costs, duplication and errors

Get started now with Dremio and Fivetran!

You can try out Dremio by going to the Dremio Getting Started page. You can also catch the annual Subsurface event on-demand here. It's the only event where data architects and engineers collaborate to explore open data lakehouses. You can register here to attend.

If you want to give Fivetran a spin for any data workload use case, it’s easy with a 14-day free trial. Also, you can catch Fivetran and Dremio together on-demand talking data lakes, self-service analytics, automated data movement and demoing the end-to-end solution.

About the authors

Kelly Kohlleffel leads the Fivetran Global Partner Sales Engineering organization, working with a broad ecosystem of technology partners and consulting services partners on modern data product and solution approaches. He also hosts the Fivetran Data Podcast, where some of the brightest minds across the data community talk about their data journey, vision and challenges. Before Fivetran, he spent time with Hashmap and NTT DATA (data solution and service consulting), Hortonworks (in Hadoop-land) and Oracle. You can connect with Kelly on LinkedIn or follow him on Twitter.

‍Rommel Garcia is the Principal Partner Solutions Architect at Dremio with extensive experience in big data and cloud computing for diverse verticals, including manufacturing and consumer goods. Prior to joining Dremio, he led Solutions Engineering at Imply (creator of Apache Druid), Kinetica DB (GPU database) and Global Security SME Lead at Hortonworks.