BACK

Migrating Teradata, Hadoop to Databricks with Azure

Our Client is a rapidly growing enterprise in the technology sector, faced multiple challenges with their existing on premise Teradata and Hadoop systems.

Challenges

  • Very high operational costs on legacy Teradata and Hadoop systems

Data was not consistently available and accessible

  • Data latency in the legacy environment was multiple hours
  • Inconsistencies and errors in data sources

Implementation

  • Orchestrated a seamless migration of on-premise Teradata and Hadoop environments to Databricks on Azure
  • Converted the DB objects, including 18000 tables, 15000 views, 4,000 SPs, 3,000 Hive Jobs, and 5,000 other scripts to Databricks and Azure Synapse
  • Setup, orchestration, and scheduling for workloads using Autosys and Azure Data Factory
  • Migrated the PowerBI data from legacy to the Azure environment

Results

  • Resolved data latency, data quality, and data availability issues by moving to the modern Databricks on Azure cloud environment.
  • Significant improvement in controlling the operational costs and optimized
  • 100% efficiency in data migration and a 40% faster ramp to the cloud
  • Increased confidence in the accuracy and integrity of the migrated data

Tech Stack