Software Engineer_Data Infrastructure

日本語の求人票はこちら

About Us

Macromill is a leading company in the fields of online research and marketing research. We leverage internet-based survey data and consumer purchase behavior insights to help our clients better understand consumer needs and behavior, enabling them to craft more effective marketing strategies.

To achieve further growth and transformation, Macromill is revitalizing its systems, building a robust data infrastructure powered by cutting-edge technology to deliver faster and more accurate marketing research solutions, ultimately driving growth for our clients.

To realize this vision, we are actively strengthening our engineering team and are looking for individuals passionate about developing advanced systems and data-driven platforms. We embrace agile development, machine learning, IaC, and other modern technologies, fostering a flexible, dynamic environment for fast and efficient service delivery.

If you’re excited about connecting businesses and consumers through data and driving transformative impact in business, or if you’re eager to take on new technologies and lead system advancements, we invite you to join us in building the future of Macromill.

About the Job

As a Data Infrastructure Engineer, you’ll take ownership of designing, building, and optimizing high-performance data infrastructure and pipelines. You’ll work on transforming raw, unstructured data into reliable, scalable, and accessible systems that support key business initiatives and data products.

• Architect and implement large-scale data systems to ensure high performance, reliability, and scalability.
• Enhance ETL processes, ensuring efficient data movement, quality, and availability.
• Establish and maintain data quality standards, automated validation, and monitoring for data integrity.
• Identify and experiment with new technologies, driving innovation and tool selection for scalable solutions.
• Lead by example, mentoring junior engineers, and sharing best practices to elevate the team’s technical capabilities.
• Work closely with data scientists, analysts, and product teams to define infrastructure needs and support data-driven decision-making.

Why Join Us

• Lead critical infrastructure projects, shaping the future of data at Macromill.
• Handle vast amounts of data in both streaming and batch environments, playing a key role in turning raw, unstructured data into valuable business insights. You’ll have the chance to create and optimize systems that process high-volume data efficiently and reliably.
• Experiment with and implement cutting-edge technologies, driving rapid iteration and innovation in data systems.
• Work with state-of-the-art tools, cloud platforms, and big data frameworks to build scalable, high-performance infrastructure.
• Opportunities for leadership, mentorship, and professional development in a fast-evolving environment.
• Represent Macromill at technical events, building both your personal brand and the company’s reputation within the tech community.

 Application Qualifications

Minimum Qualifications

• Hands-on experience with at least one major cloud platform (GCP preferred, AWS, or Azure), including infrastructure components like storage (e.g., BigQuery, S3), compute resources (e.g., Kubernetes, Dataproc), and networking.
• Strong proficiency in one or more programming languages (Python, Java, or Scala) with experience building complex data processing solutions.
• Advanced SQL skills, including query optimization and data warehousing techniques for high-volume data environments.
• Familiarity with big data frameworks and tools, such as Apache Spark, Hadoop, or Kafka, and experience implementing both batch and real-time processing solutions.
• Proven track record of designing and building ETL/ELT processes and data workflows for high-volume data in production environments.
• Understanding of data governance practices, data quality management, and setting up automated validation and monitoring for data integrity.

Preferred Qualifications

• Deep knowledge of distributed systems, including understanding of scalability, fault tolerance, and reliability in data systems, with hands-on experience in technologies like Spark, Flink, or Kafka Streams.
• Experience with workflow orchestration tools like Apache Airflow or Google Cloud Composer for managing complex data pipelines and automated workflows.
• Proficiency in containerization and orchestration (e.g., Docker, Kubernetes) and familiarity with DevOps practices for CI/CD, automation, and deployment in cloud environments.
• Familiarity with data security best practices, including data encryption, role-based access control, and industry compliance standards (e.g., GDPR).
• Experience with data monitoring, logging, and alerting tools (e.g., Prometheus, Grafana, DataDog) to ensure data infrastructure reliability and performance.
• Experience leading technical projects or initiatives, with a demonstrated ability to make architectural decisions, mentor junior engineers, and drive best practices across teams.

Who are we looking for

• A collaborative and proactive team player who can thrive in a fast-paced environment.
• A detail-oriented professional with a strong commitment to data accuracy and quality.
• An analytical thinker who can troubleshoot complex data pipeline issues with creative problem-solving.
• A lifelong learner who is enthusiastic about exploring new tools, techniques, and trends in data engineering.
• Someone passionate about transforming raw data into actionable insights that drive real business impact.