26-159.B - Data Engineer

Essential Job Responsibilities:

• Serve as an administrator of the Databricks platform, managing workspaces, clusters, jobs, permissions, and configurations across environments.
• Monitor and optimize Databricks cluster performance, compute usage, and storage to ensure cost-effective and efficient operations.
• Track and report on platform costs, identifying opportunities for cost reduction and resource optimization across Databricks and associated cloud services.
• Proactively monitor the health, availability, and performance of data pipelines and platform components, implementing alerting and observability best practices.
• Develop, manage, and maintain scalable and reliable data pipelines for collecting, processing, and storing large volumes of structured and unstructured data.
• Create and optimize ETL processes to ensure high-quality data is available for analysis and reporting, leveraging tools such as dbt, Spark, and Databricks workflows.
• Integrate data from multiple sources, ensuring data consistency, quality, and accessibility across the organization.
• Implement and monitor data quality checks and validation processes to ensure accuracy and reliability of data throughout the pipeline lifecycle.
• Architect and maintain robust, scalable, and efficient data infrastructure to support data-driven applications and analytics initiatives.
• Build and deploy data applications using a combination of personal technical expertise and AI-assisted development tools and methodologies.
• Support the deployment and maintenance of data-driven applications, ensuring they are optimized for performance and scalability.
• Monitor and improve the performance of data systems and queries, ensuring efficient data processing and minimal downtime.

Other Job Responsibilities:

• Maintains employee confidence and protects company assets, including intellectual property, by keeping information confidential.
• Maintains technical knowledge by attending educational workshops, reviewing professional publications, establishing personal networks, and participating in professional associations.
• Other duties as assigned by management.

Qualifications:

• Three (3) years minimum experience in a data engineering role, prefer oil and gas experience.
• Strong working knowledge of Databricks, including workspace administration, cluster management, Delta Lake, Unity Catalog, and Databricks SQL.
• Experience with cloud application infrastructure, particularly AWS (S3, IAM, EC2, Lambda, etc.).
• Proficiency with data engineering tools such as dbt, Apache Spark, SQL, and Python.
• Experience tracking and optimizing cloud platform costs and resource utilization.
• Demonstrated ability to build data applications using modern development practices, including leveraging AI-assisted development tools.
• Exceptional ability in diagnosing and solving technical issues related to data platforms and pipelines.
• Ability to establish and maintain effective working relationships with employees, supervisors, other departments, officials, and the public.
• Ability to complete multiple, diverse tasks of differing priorities with minimum supervision.
• Excellent written and verbal communication skills, with specific ability to translate complex data systems information into an organized and presentable manner.
• Ability to travel to remote locations via large and small planes, helicopter, or boat. Expectation is quarterly trips to our asset locations.
• Required: Working knowledge of oil and gas industry applications and processes.

Education Requirements:

• Bachelor's Degree from an accredited four-year university or college with a degree in Computer Science, Data Engineering, Information Systems, or a closely related field. Additional experience may substitute for education on a year-for-year basis.