Data Engineer (Databricks)
Location: Hybrid (Washington D.C) 2-3 times a week
Clearance: Active Secret
Employment Type: Full Time
Company Description
Big Impact Tech (BIT) is a Small Business providing IT and business management consulting to federal and commercial clients. We deliver mission-focused solutions in data, cloud, cybersecurity, and program management.
Position Overview
We are seeking a skilled Data Engineer with strong Databricks expertise to support Coast Guard data modernization, analytics, and cloud transformation initiatives. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines, data lakehouse solutions, and cloud-based data platforms that enable efficient data integration, storage, processing, and analytics.
The candidate will work closely with data architects, analysts, developers, and business stakeholders to deliver reliable, secure, and high-performing data solutions.
Key Responsibilities
Design, develop, and maintain scalable data pipelines using Databricks and cloud-native technologies.
Build and optimize ETL/ELT processes for structured, semi-structured, and unstructured data.
Develop and manage data ingestion frameworks from various source systems, APIs, databases, and files.
Implement and maintain Delta Lake architecture and Lakehouse solutions.
Develop PySpark, Spark SQL, and SQL-based transformations for large-scale data processing.
Support data warehousing, data lakes, and enterprise analytics initiatives.
Monitor and optimize data pipeline performance, reliability, and scalability.
Ensure data quality, governance, lineage, and security across data platforms.
Collaborate with business stakeholders and technical teams to understand requirements and deliver data solutions.
Implement CI/CD practices and automated deployment processes for data engineering solutions.
Troubleshoot and resolve production issues related to data ingestion, transformation, and reporting.
Create and maintain technical documentation, data dictionaries, and architecture diagrams.
Required Qualifications
Bachelor’s degree in Computer Science, Information Systems, Engineering, Data Science, or a related field.
5+ years of experience in Data Engineering.
3+ years of hands-on experience with Databricks.
Strong experience with PySpark, Spark SQL, and SQL.
Experience building large-scale data pipelines and ETL/ELT solutions.
Experience with cloud platforms such as Azure, AWS, or Google Cloud.
Strong understanding of data modeling, data warehousing, and Lakehouse architectures.
Experience working with relational and NoSQL databases.
Preferred Qualifications
Experience supporting Coast Guard or other government-related programs.
Experience with Azure Databricks, Azure Data Factory, Microsoft Fabric, Synapse Analytics, or Snowflake.
Knowledge of Delta Lake, Unity Catalog, and Databricks Workflows.
Experience with real-time and streaming data solutions.
Familiarity with DevOps practices, Git, CI/CD pipelines, and Infrastructure as Code.
Experience with Power BI, Qlik, or Tableau reporting environments.
Relevant Databricks or cloud certifications preferred.
Technical Skills
Databricks
PySpark
Apache Spark
Spark SQL
SQL
Delta Lake
Data Lakehouse Architecture
Azure Databricks
Azure Data Factory
Microsoft Fabric
Snowflake
Data Warehousing
ETL/ELT Development
Data Modeling
Git
CI/CD
Cloud Platforms (Azure, AWS, GCP)
Lifeteam is seeking a dedicated Rotor Wing Pilot to join our emergency medical service team. In this role, you will be responsible for piloting and navigating rotor wing aircrafts used in medical emergencies. Our pilots play a critical role in our mission to provide rapid...
...requiere tareas manuales como empacar comidas fras, mantener el rea limpia y apoyar en actividades generales dentro de la planta. El trabajo se realiza en un ambiente de ritmo acelerado, colaborativo y con temperatura controlada de aproximadamente 30F (como un...
...partnering with a fast-paced digital media and real estate technology organization seeking a creative and trend-savvy Social Media Content Creator to support brand growth and engagement across major social platforms. This is an exciting opportunity for someone who lives and...
Job Description Job Description Armored Car Armed Guard/Driver Company Overview At Montgomery Armored Car Service, Inc., we specialize in secure transportation and cash management services. We are committed to safety, reliability, and professionalism, ensuring...
...Please use Google Chrome or Mozilla Firefox when accessing Candidate Home. By joining the American Red Cross you will touch millions of lives every year and experience the greatness of the human spirit at its best. Are you ready to be part of the world's largest humanitarian...