Organization: aKube Inc
Location: United States
Category: IT Services Location: Onsite/ Hybrid/ Remote: Remote Duration:1 Month Rate Range: Upto $89/hr on C2C or $82/hr on W2 Work Authorization: GC, USC, All valid EADs except H1B, OPT, CPT Must Have: Advanced Databricks engineering experience Delta Lake Data ingestion from new and diverse sources Medallion Architecture: Bronze, Silver, Gold Databricks Genie Spaces Genie Space debugging, query tuning, and performance optimization Advanced SQL and Spark/PySpark Responsibilities: Build and support Databricks pipelines for new data source ingestion. Design and develop Bronze, Silver, and Gold data layers. Optimize Delta Lake tables, queries, and platform performance. Debug Genie Spaces and improve natural language query response quality. Tune underlying SQL and data models used by Genie Spaces. Support short-term delivery needs across the Databricks platform. Troubleshoot and resolve complex issues related to Genie Spaces and Medallion Architecture. Collaborate with cross-functional teams to ensure seamless integration of Databricks pipelines. Develop and maintain documentation for Databricks pipelines and Medallion Architecture. Qualifications: Strong hands-on Databricks and Delta Lake engineering background. Proven experience building Medallion Architecture in production. Experience integrating multiple data sources into centralized data platforms. Direct experience with Databricks Genie Spaces is required. Strong SQL, Spark, and data performance tuning experience. Nice to Have: Unity Catalog Databricks workflows Data governance and observability Lakehouse migration experience Details Originally posted on Himalayas
Get Latest Updates: Join our WhatsApp Group
Job Requirements:
- Design and develop Databricks pipelines for data ingestion from new and diverse sources.
- Build and support Databricks pipelines for data ingestion from new and diverse sources.
- Design and develop Bronze, Silver, and Gold data layers for Medallion Architecture.
- Optimize Delta Lake tables, queries, and platform performance.
- Debug Genie Spaces and improve natural language query response quality.
- Tune underlying SQL and data models used by Genie Spaces.
- SUPPORT short-term delivery needs across the Databricks platform.
- Collaborate with data engineers to ensure seamless integration of data pipelines.
- Develop and maintain documentation for Databricks pipelines and Medallion Architecture.
- Participate in code reviews and provide feedback on Databricks pipeline development.
Additional Requirements:
- Experience with Databricks Genie Spaces and Medallion Architecture is required.
- Strong understanding of data engineering principles and practices.
- Ability to design and develop efficient data pipelines and data architectures.
- Excellent problem-solving skills and ability to troubleshoot complex issues.
- Strong communication and collaboration skills, with experience working with cross-functional teams.