Apache Flink Expert

Responsibilities

1. Design and implement innovative solutions for streaming data processing, addressing challenges like data skew and out-of-order events, using Apache Flink.
2. Develop and maintain near real-time ingestion pipelines for data lakes, ensuring efficient and reliable data flow.
3. Optimize and manage the data infrastructure supporting streaming systems to ensure high performance and scalability.
4. Collaborate closely with cross-functional team members to integrate data processing solutions seamlessly into the overall system architecture.

Requirements

Requirements

1. Enthusiasm for working in a dynamic startup environment with a small, dedicated team, and a passion for tackling complex technical challenges.
2. Proficiency in at least one programming language, such as Java, Scala, Kotlin, or Python, with a strong understanding of software engineering principles.
3. Hands-on experience with big data systems like Apache Spark, Apache Flink, and Apache Kafka, including building and optimizing data processing pipelines.
4. Knowledge of common data storage formats such as Avro, Proto, and Parquet, along with experience working with large-scale data lakes.
5. Experience with at least one cloud platform, such as AWS or GCP, and familiarity with deploying and managing applications in a cloud environment.
6. Familiarity with containerization technologies like Kubernetes and experience with related tools such as Helm, Prometheus, and Grafana.

Preferred Qualifications

1. Experience with data analytics tools like Apache Hive, ClickHouse, Presto, Trino, or AWS Redshift, for querying and analyzing large datasets.
2. Familiarity with modern data lake formats such as Apache Iceberg, Hudi, and Paimon, and experience in optimizing data storage and retrieval in these formats.
3. Proficiency in DevOps practices, Infrastructure as Code (IoC), and experience with tools like Terraform for automating infrastructure management.
4. Background in Machine Learning or Recommendation Systems, with a strong understanding of relevant algorithms and techniques.

Benefits

• Competitive annual salary ranging from $120,000 to $180,000 USD, depending on experience and qualifications.
• Stock options to provide ownership and alignment with the company's success.
• Comprehensive health, dental, and vision insurance coverage for you and your family.
• Flexible work arrangements, including remote work options and flexible hours.
• Opportunities for career growth and professional development, with access to training and learning resources.
• Collaborative and inclusive work culture, with regular team-building activities and events.
• Paid time off and holidays to promote work-life balance and well-being.