Confidential Company
|Data & Software Engineer
Lagos, Lagos State, Nigeria
Summary
Led end-to-end design, development, and operational management of scalable ETL pipelines and data infrastructure, leveraging Talend, BigQuery, and Linux-based automation to enhance data freshness, reduce costs, and ensure high system uptime.
Highlights
Engineered enterprise ETL workflows in Talend Open Studio, integrating 10+ source systems into Google BigQuery data warehouse, improving data freshness from daily to hourly.
Automated and monitored 25+ production Talend jobs using Webmin + cron scheduling on Linux, achieving 99.7% pipeline uptime with zero manual intervention.
Optimized BigQuery SQL models and partitioned tables, reducing query costs by 35% while supporting 500K+ rows ingested daily via Talend.
Designed Talend joblets and reusable components for data validation, error handling, and logging, cutting development time for new pipelines by 50%.
Managed Linux servers via Webmin to configure cron jobs, permissions, and service monitoring for mission-critical ETL infrastructure.
Built wrapper scripts in Bash/Python to trigger Talend jobs, add retry logic, and send Slack/email alerts on failure, improving Mean Time To Recovery (MTTR) by 70%.
Developed data APIs to expose curated BigQuery datasets to downstream applications, handling authentication, rate limiting, and caching.