Senior Data Engineer

Job Details: 

• Title: Sr. Data Engineer
• Salary: $180-$230K 
• Requirements: 8+ years of software dev exp (Large Data Sets or Big Data)
• Location: 100% onsite (San Francisco Bay Area) 

Our client, a gaming startup, is seeking a Sr. Data Engineer to join their passionate team. The ideal candidate is an experienced engineer who will play a vital role in reconstructing and refining data processes, collaborating closely with the engineering team to guarantee smooth integration of data solutions.

Responsibilities

• Comprehend and optimize existing data processes within Databricks.
• Collaborate with the engineering team to seamlessly integrate data solutions.
• Translate basic product requirements into robust back-end systems.
• Demonstrate proactive implementation of solutions to meet project deadlines.
• Work on systems involving various stakeholders, requiring effective communication skills.
• Provide technical leadership and guide the team on coding best practices, data management, and system architecture.

Requirements

• Bachelor's or Master's degree in Computer Science, Software Engineering, or related field.
• Senior Back-End Engineer experience with focus on data-related projects.
• Proficiency in various data storage solutions including basic RDBMS, key-value stores, data warehouses, and data lakes.
• Familiarity with caching systems.
• Extensive experience in at least one queue system such as Kafka, RabbitMQ, or similar (understanding of enterprise solutions like Confluent or MSK is a plus).
• Basic understanding of open-source technologies in data pipelines, such as Airbyte or Meltano.
• Exposure to enterprise solutions like Fivetran or Databricks is advantageous.
• Real-time data processing and streaming techniques using Spark structured streaming and Kafka.
• Ability to work independently, architect solutions, and perform data ingestion development and support.
• Segregation capability between BI analytics and data engineering workflows.