Databricks Engineer Job at Openkyber, Florida

NlNjak5Cd3psTCt5S2ZWOUJSQk5tYyttY3c9PQ==
  • Openkyber
  • Florida

Job Description

Role: Databricks Data Engineer Location: Remote Duration: 12+ Months Must be Databricks certified.

The Databricks Data Engineer owns the end-to-end migration strategy, target architecture design, and technical execution of moving legacy ETL workloads to the Databricks Lakehouse. He will establish migration standards, optimize PySpark pipelines, orchestrate complex data workflows, and deploy proprietary automation tools to ensure a seamless, high-performing transition from legacy systems.

Key Responsibilities Architectural Strategy & Governance:
  • Design the target Databricks Lakehouse architecture utilizing Delta Lake, Photon, and Unity Catalog.
  • Establish global code refactoring standards, optimization benchmarks, and PySpark best practices.
  • Resolve highly complex dependency mappings and architect seamless, zero-downtime dual-run strategies.
  • Lead the technical deployment and integration of specialized migration accelerators
Hands-on Engineering & Optimization:
  • Review automated output from migration tools and manually refactor complex legacy logic into high-performing PySpark notebooks.
  • Eliminate legacy anti-patterns such as massive row-by-row processing and inefficient lookups.
  • Optimize PySpark code performance using advanced Spark features including Z-Ordering, partitioning, and caching.
  • Build robust Databricks Workflows and orchestrate complex DAGs based on comprehensive source lineage.
Technical Skills & Competencies:
  • Core Platforms: Databricks, Delta Lake, Unity Catalog, Photon, DataStage.
  • Languages & Frameworks: PySpark, Python, SQL, Shell Scripting.
  • Cloud & DevOps: AWS alongside CI/CD deployment pipelines.
  • Orchestration: Apache Airflow, Databricks Workflows.

For applications and inquiries, contact:hirings@openkyber.com

Job Tags

Remote work

Similar Jobs

SOFTGAMES

Game Content Designer - Fully Remote Job at SOFTGAMES

We are looking for a Game Content Designer to join the team building a new, unannounced free-to-play casual game. In this role you will shape what players actually experience day-to-day: the story that pulls them forward, the world they progress through, and the items and... 

SGS Consulting

Research Associate - III Job at SGS Consulting

 ...including database analyses using administrative claims or electronic medical records, as well as observational studies such as prospective...  ...Education/Experience: Advanced degree in health services research, health policy, pharmacy, public health, epidemiology, health... 

United Food and Commercial Workers

U.S. Payroll Staff Accountant Job at United Food and Commercial Workers

 ...United Food and Commercial Workers U.S. Payroll Staff Accountant Based in Washington DC near Metro (on-site) Organization: United Food and Commercial Workers International Union (UFCW) is North Americas neighborhood union; standing together to improve... 

Epic Games

Senior Game Designer Job at Epic Games

 ..., and creative environment. Whether its building award-winning games or crafting engine technology that enables others to make visually...  ...innovating to raise the bar of engine and game development.DESIGNWhat We DoPut your design skills to work at a cutting-edge company... 

MatrixSpace

Senior Federal Account Manager - Federal Agencies Job at MatrixSpace

 ...Senior Federal Account Manager Federal Agencies Location: Remote, United States Type: Full-Time, Remote Expand MatrixSpaces...  ...demonstrations. Build relationships with relevant federal integrators, contractors, and industry partners to strengthen MatrixSpaces position...