DataBricks Architect Job at Abidi Group LLC, Plainsboro, NJ

L1IwTXBrbE4rOGlFQXlJZjZ6NkJVTTJ1RkE9PQ==
  • Abidi Group LLC
  • Plainsboro, NJ

Job Description

Job Description

Job Description

Technical Expertise:

  • Data bricks Mastery: Deep, expert-level knowledge of the Data bricks Platform, including:
    • Unity Catalog: Designing and implementing data governance and security.
    • Delta Lake & Delta Live Tables: Architecture and building reliable, scalable data pipelines.
    • Performance & Cost Optimization: Expertise in tuning Spark jobs, optimizing cluster usage, and managing platform costs.
    • MLOps: Strong, practical understanding of the machine learning lifecycle on Data bricks using tools like MLflow.
    • Data bricks SQL: Knowledge of designing and optimizing analytical workloads.
    • Mosaic AI: Knowledge of designing and optimizing AI Agents.
  • Cloud & Infrastructure: Deep knowledge of cloud architecture and services on AWS. Strong command of Infrastructure as Code (Terraform, YAML).
  • Data Engineering & Programming: Strong background in data modeling, ETL/ELT development, and advanced, hands-on programming skills in Python and SQL.
  • CI/CD & Automation: Experience with designing and implementing CI/CD pipelines (preferably with GitHub Actions) for data and ML workloads.
  • Severability: Familiarity with implementing monitoring, logging, and alerting for data platforms.
  • Automation: The platform is ephemeral, and all changes are implemented using Terraform and Python. Expertise in Terraform and Python is a must.

Job Tags

Similar Jobs

Weatherby Healthcare

A Neurologist Is Needed for Temp to Perm Help in SC Job at Weatherby Healthcare

 ...A Neurologist Is Needed for Temp to Perm Help in SC at Weatherby Healthcare summary: A neurologist position is available for temp-to-perm placement in South Carolina, focusing on outpatient neurology with a Monday to Friday schedule and managing 15+ patients daily.... 

Tanner Goldsmith - State Farm Agent

Receptionist - State Farm Agent Team Member Job at Tanner Goldsmith - State Farm Agent

 ...Paid time off Signing bonus Training & development Company Overview: Tanner Goldsmith - State Farm Agent, a dynamic and forward-thinking player in the insurance industry, is actively seeking energetic and customer-focused individuals to join our team as Receptionist... 

Argon Agency

Social Media Community Manager Job at Argon Agency

 ...Job Description Job Description The Social Media Community Manager will be responsible for building and maintaining our brands online presence across all social media platforms. This role involves creating content, managing community interactions, and developing strategies... 

Stay at Home Pet Services, llc

Pet Sitter Job at Stay at Home Pet Services, llc

 ...weight while making money? Passionate about animals? You may be just who were looking for! Stay At Home Pet Services, LLC is an in-home pet sitting service. Were looking for great people who share our obsession for amazing customer service! If youre ready... 

Confidential

Civil Project Engineer Job at Confidential

 ...design computations done by others and general coordination of design aspects. Qualifications: Bachelor's degree in civil engineering or related discipline. Registered Professional in field of expertise strongly preferred. Eight or more (8+) years of experience...