Overview
As a Senior Data Engineer in the MIDAS (Management Integration & Data Analytics System) Data Platform Team, you will build from scratch and maintain the central data hub connecting most systems found inside one of Japan's more innovative digital banks.
You will work with modern cloud-based data technologies to ingest data from various banking systems, apply complex business logic on it, then serve it to downstream systems for enterprise management, regulatory reporting, risk management and many other applications.
Thanks to the high expectations towards the banking domain, you will have the opportunity to work on complex data engineering challenges including data quality, reconciliation across multiple systems, time-critical data processing, and complete traceability.
This is a senior individual contributor role where you will design and implement complex data pipelines, mentor mid-level engineers, and participate in architectural decisions for the platform.
※ This position involves employment with Money Forward, Inc., and a secondment to the new company (SMBC Money Forward Bank Preparatory Corporation). The evaluation system and employee benefits will follow the policies of Money Forward, Inc.
Who we are?
We are a startup team partnering with Sumitomo Mitsui Financial Group and Sumitomo Mitsui Banking Corporation to establish a new digital bank. Our mission is to build embedded financial products from the ground up, with a strong focus on supporting small and medium-sized businesses (SMBs).
Responsibilities and Duties
- Design and implement data pipelines to ingest data from multiple source systems (CRM, CBS, CLM, LOS) using REST APIs or database connections
- Build and maintain Bronze/Silver/Gold layer transformations ensuring data quality, consistency, and performance
- Implement data quality checks and cross-system reconciliation logic (e.g., validating CBS transaction records against ledger balances)
- Develop and optimize SQL queries and transformations using Databricks SQL/notebooks or Delta Live Tables
- Design and implement data models for analytics and reporting use cases, with a primary focus on CRM data integration, alongside regulatory reporting and the outbound Conseek risk-export mart (ALM/ERM risk computation itself is handled externally by Conseek)
- Build REST APIs or data serving layers for downstream consumers
- Participate in architecture decisions for data platform components
- Write unit tests, integration tests, and data quality tests for pipelines
- Monitor data pipeline performance, troubleshoot failures, and implement improvements
- Optimize query performance through partitioning strategies, Z-ordering, and query tuning
- Implement infrastructure as code for data platform components using Terraform
- Set up CI/CD pipelines for automated testing and deployment of data pipelines
- Mentor mid-level engineers and conduct code reviews
- Contribute to documentation and best practices for the team
- Collaborate with backend engineers to define API contracts and data schemas
- Work with Technical Lead on platform design and technology selection decisions
- Lead features and initiatives within the data platform
- Support EOD (End-of-Day) data collection processes that align with Zengin settlement timing
Required Skills and Experience
- 5+ years of experience in data engineering or analytics engineering
- Strong proficiency in SQL and Python
- Hands-on experience building data pipelines using modern tools (Databricks, Spark, or similar)
- Experience with cloud data platforms (AWS, Azure, GCP) and storage systems (S3, ADLS, GCS)
- Strong understanding of data modeling techniques including dimensional modeling, data vault, or event-driven architectures
- Proven ability to debug and optimize slow queries and data processing jobs
- Experience with version control (Git) and CI/CD pipelines
- Understanding of data governance concepts: access control, audit logging, data lineage
- Strong problem-solving skills and ability to work independently
- Experience mentoring junior or mid-level engineers
- Excellent communication skills for collaborating with cross-functional teams
- Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience
- Language ability: Japanese at Business level required
Preferred Skills and Experience
- English proficiency (Business level or above, e.g. TOEIC 700+) is a plus for cross-team documentation and collaboration
- Experience with data quality validation and testing frameworks
- Experience in financial services, fintech, or other regulated industries
- Knowledge of banking domain concepts: core banking systems, payment processing, regulatory reporting, AML/transaction monitoring
- Experience implementing data platforms that comply with regulatory requirements (FI
- SC Security Guidelines, FSA/BOJ reporting, GDPR, APPI)
- Hands-on experience with Databricks platform (AutoLoader, Unity Catalog, Delta Live Tables, Databricks SQL, Databricks Workflows)
- Experience implementing cross-system reconciliation for financial data
- Experience with performance tuning on Databricks: partitioning strategies, Z-ordering, query optimization, cost management
- Experience building REST APIs with Python (FastAPI, Flask, or similar) for data serving
- Knowledge of streaming data ingestion patterns and external tools (Kafka, Kinesis)
- Experience with Terraform
- Contributions to open-source data engineering projects
- Experience with Databricks SQL Dashboards or BI tools (Tableau, Looker, PowerBI)
- Experience leading technical initiatives from design through implementation
- Track record of improving data platform performance or reducing costs (provide specific metrics)
- Experience in AI development and/or experience in using AI tools to improve development processes.
- Money Forward is at a major turning point, shifting "from Cloud to AI." We are currently driving "AX (AI Transformation)"—the next step beyond DX—with the goal of providing "Digital Workers," where AI agents autonomously execute tasks. As we enter a phase of evolving into Japan's No. 1 back-office AI company by integrating AI agents into all of our products in the future, we are looking for individuals who can contribute to AI-driven development and value creation.(More information here)
Technology Stack and Tools Used
Cloud Infrastructure
- AWS (primary cloud platform in Tokyo region)
- S3 for data lake storage with VPC networking for secure connectivity
- AWS IAM for security and access management
Data Lakehouse Architecture (Databricks)
- Modern lakehouse architecture using Delta Lake for ACID transactions, time-travel, and schema evolution
- Columnar storage formats (Parquet) optimized for analytics
- Bronze/Silver/Gold medallion architecture for progressive data refinement
- Partition strategies and Z-ordering for query performance
- Unity Catalog for centralized governance and metadata management
Orchestration & Processing (Databricks)
- Databricks Workflows for managed workflow orchestration
- Distributed data processing with Apache Spark on Databricks clusters
- Serverless compute and auto-scaling clusters for cost optimization
- Streaming and batch ingestion patterns with Databricks AutoLoader
Data Transformation (Databricks)
- Delta Live Tables for declarative ETL pipelines with built-in data quality
- SQL and Python for data transformations in Databricks notebooks
- Incremental materialization strategies for efficiency
Query & Analytics (Databricks)
- Databricks SQL for high-performance analytics queries
- Serverless and auto-scaling SQL warehouses for variable workloads
- Query result caching and optimization
- REST APIs for data serving to downstream consumers
- Direct Delta Lake access for advanced consumers (Athena, Redshift Spectrum, etc.)
Data Quality & Governance (Databricks)
- Automated data quality with Delta Live Tables expectations
- Cross-system reconciliation and validation logic
- Fine-grained access control with column/row-level security using Unity Catalog
- Automated data lineage tracking for regulatory compliance
- Audit logging and 10-year data retention policies
Business Intelligence (Databricks)
- Databricks SQL Dashboards for internal analytics and monitoring
- Integration with enterprise BI tools (Tableau, PowerBI, Looker) via Databricks SQL endpoints
Development & DevOps
- Languages: SQL (primary), Python
- Platform: Databricks (notebooks, workflows, SQL)
- Version Control: GitHub
- CI/CD: GitHub Actions
- Infrastructure as Code: Terraform
- Monitoring: Databricks monitoring, AWS CloudWatch integration
- AI-Assisted Development: Claude Code, GitHub Copilot, ChatGPT
Work Environment
At Money Forward, we provide an environment where we can create world-class services together, and we are looking forward to welcoming you.
- Provided PC Specs: We provide PCs equipped with the latest CPUs (MacOS or Windows). Custom-made PCs tailored to business requirements and replacements with the latest OS are also possible.
- Money Forward Library: We have a library system where you can freely borrow books, ranging from technical books to management books. Desired books can be purchased at the company's expense.
- Referral Driven: We cover the cost of recruitment meals. There is a referral reward system.
- Conference Participation Support: The company partially covers participation in domestic and international conferences, such as RubyKaigi and Google I/O.
| 職種 / 募集ポジション | Senior Data Engineer - MIDAS Data Platform, Digital Bank, Tokyo |
|---|---|
| 雇用形態 | 正社員 |
| 給与 |
|
| 勤務地 | |
| Salary System | <Salary Range> Min 542,000 JPY / month（6,504,000 JPY / year）〜917,000 JPY / month（11,004,000 JPY / year） Each including fixed allowances of 157,808 JPY〜266,988 JPY / month. |
| Bonus | A「High Performance Bonus」may be paid to employees who receive high evaluations based on semi-annual evaluations in addition to their salary. ※Please note that the remuneration of the High Performance Bonus is subject to change according to the company's performance. |
| Probation Period | 3 months from join date |
| Working Hour System | Discretionary Labor System for Professional Work ※Conditions apply; subject to change to Flextime System. |
| Working Hours | 9:30 - 18:30 (60 min break) are the basic working hours. However, employees are able to choose their working hours at their own discretion. ※There is potential for overtime work outside the determined hours. |
| Work Style Policy | Hybrid work style ・As a standard practice, employees are required to work at the office a minimum of 2 days per week. Employees are encouraged to spend 3 or more days in the office. (This policy may be subject to change based on the company and job circumstances) ・The specific "team office days" may vary depending on the assigned team. |
| Holidays/Vacations | ■ Saturdays / Sundays / Japanese national holidays ■ Paid holidays ■ Summer holidays (3 days) ■ Winter holidays (2 days) ■ Year-end and New Year’s holidays (Dec 31st～Jan 3rd) |
| Benefits | ■ Various social insurances (employee pension, health insurance, employment insurance, industrial accident compensation insurance) ■ Neighborhood housing allowance and neighborhood moving allowance ■ Salary-based rent deduction benefit ■ Health check and gynecological checkup ■ Influenza vaccine ■ Book purchases support ■ Defined-contribution corporate pension ■ Employee stock ownership plan ■ Preferential treatment when using the following services（limited to businesses under contract with Money Forward） - Rental agency - Housekeeping services - Babysitting - Online English conversation school |
| Selection Process１ | Casual interview/Document Screening ↓ First interview (Depending on the position, there may be a technical assignment before the interview.) ↓ Several interviews (The number of interviews depends on the position. We will request you to take an aptitude assessment.) ↓ Final interview (We may ask for a reference check before or after the interview.) ↓ Job offer/Offer meeting ※The process may be subject to change depending on the case. |
| Selection Process２ | ■ About the Aptitude Test It is a simple web-based survey taking around 20 minutes to complete. ■ What are reference checks? Money Forward may ask for your cooperation with reference checks using a reference check service tool. We believe that mutual understanding is limited to the selection process alone. Therefore, we would like to gather information about you from your supervisor and colleagues at your current or former company to ensure a more reliable match and facilitate your early success after joining our company. ※We do not make employment decisions based solely on the contents of reference checks. ※The fact that you are in the selection process with us will not be disclosed to referees. |
| Notes | ・Range of change in job description: Work as determined by the company ・Range of change in work location: Work location as determined by the company |
| Reference Information | https://recruit.moneyforward.com/#introduction |
| 会社名 | 株式会社マネーフォワード |
|---|---|
| 代表者 | 代表取締役社長グループCEO 辻 庸介 |
| 創業 | 2012年5月 |
| 取締役 | 金坂 直哉 中出 匠哉 竹田 正信 石原 千亜希 |
| 社外取締役 | 安武 弘晃 宮澤 弦 Ryu Kawano Suliawan 菊間 千乃 芦田 健 上田 梨々子 |
| 監査役 | 畠山 優実 田中 克幸 西山 茂 |
| グループCxO・VPox | 上利 陽太郎 瀧 俊雄 山田 一也 坂 裕和 松岡 俊 伊藤 セルジオ 大輔 関田 雅和 松久 正幸 野村 一仁 長尾 祐美子 金井 恵子 梅田 康吉 丸山 嘉伸 |
| 執行役員 | 田平 公伸 本川 大輔 冨山 直道 永井 博 駒口 哲也 廣原 亜樹 島村 誠一郎 永井 七奈 木村 慎治 吉本 憲文 工藤 裕之 島内 広史 小山 幸宏 渡辺 恵伍 松村 道夫 岩崎 大 渋谷 亮 |
| オフィス | 本社オフィス 〒108-0023 東京都港区芝浦3-1-21 msb Tamachi 田町ステーションタワーS 21F 北海道支社 〒060-0061 北海道札幌市中央区南一条西4-5-1 札幌大手町ビル3階 東北支社 〒980-0021 宮城県仙台市青葉区中央2-2-10 仙都会館 5F 東海支社、名古屋開発拠点 〒450-6213 愛知県名古屋市中村区名駅4-7-1 ミッドランドスクエア 13F 京都支社、京都開発拠点 〒604-8004 京都府京都市中京区三条通河原町東入中島町78番地 明治屋京都ビル 4階 関西支社、大阪開発拠点 〒541-0042 大阪府大阪市中央区今橋 2-5-8 トレードピア淀屋橋 9階 広島支社 〒730-0015 広島市中区橋本町9-7 ビル博丈5F 九州・沖縄支社、福岡開発拠点 〒810-0041 福岡県福岡市中央区大名2丁目6-50 福岡大名ガーデンシティ 16F |
| 社内コミュニケーション活性化の取り組み | ■全社週次/月次朝会/半期総会 ■代表との意見交換会（CEOセッション） ■全社懇親会（MF Happy Hour） ■他部門社員との交流会（シャッフルランチ・ディナー） ■上長との定期1on1（ツキイチ面談） ■社内公募制度（MFチャレンジシステム） ■社員満足度調査（MFグループサーベイ） ※一部正社員のみ |
| 労働条件 | 屋内原則禁煙（喫煙室あり）等 |
| 中途採用比率 | 2021年11月末 93.8％ 2022年11月末 90.0% 2023年11月末 76.6％ 2024年11月末 88.7% |