Lead Data Engineer
Remote - USA
Full Time Senior-level / Expert USD 210K - 235K
Wizard
Meet the AI-powered, two-way SMS commerce platform helping brands provide engaging, delightful, and personalized mobile shopping experiences—all via text.About Us
Wizard is revolutionizing the shopping experience using the power of generative AI and rich messaging technologies to build a personalized shopping assistant for every consumer. We scour the entire internet of products and ratings across brands and retailers to find the best products for every consumer’s personalized needs. Using an effortless text-based interface, Wizard AI is always just a text away. The future of shopping is here. Shop smarter with Wizard.
The Role
We seek a Lead Data Engineer to take charge of our data engineering initiatives, focusing on enhancing data collection, storage, and analysis across all of Wizard's dynamic services. This senior position is pivotal to our data infrastructure, enabling data-driven decision-making and supporting our ambitious growth objectives.
Key Responsibilities:
- Architect and scale a state-of-the-art data infrastructure capable of handling batch and real-time data processing needs with unparalleled performance.
- Collaborate closely with the data science team to oversee data systems, ensuring accurate monitoring and insightful analysis of business processes.
- Design and implement robust ETL (Extract, Transform, Load) data pipelines, optimizing data flow and accessibility.
- Develop comprehensive backend data solutions to bolster microservices architecture, ensuring seamless data integration and management.
- Engineer and manage integrations with third-party e-commerce platforms, expanding Wizard's data ecosystem and capabilities.
You
- Bachelor's degree in Computer Science or a related field, with a solid foundational knowledge of data engineering principles.
- 7-10 years of software development experience, significantly focusing on data engineering.
- Proficiency in Python or Java, with a deep understanding of software engineering best practices.
- Expertise in distributed computing and data modeling, capable of designing scalable data systems.
- Demonstrated experience in building ETL pipelines using tools such as Apache Spark, Databricks, or Hadoop.
- Extensive experience with NoSQL databases, including MongoDB, Cassandra, DynamoDB, and CosmosDB.
- Proficiency in real-time stream processing systems such as Kafka, AWS Kinesis, or GCP Data Flow.
- Skilled in utilizing caching and search technologies like Redis, Elasticsearch, or Solr.
- Experience with message queuing systems, including RabbitMQ, AWS SQS, or GCP Cloud Tasks.
- Familiarity with Delta Lake, Parquet files, AWS, GCP, or Azure cloud services.
- A strong advocate for Test Driven Development (TDD) and experienced in version control using Git platforms like GitHub or Bitbucket.
Additional Preferred Qualifications
- Exceptional written and verbal communication skills, capable of articulating complex technical concepts clearly and concisely.
- A collaborative team player, eager to share knowledge and learn from peers, passionate about mentoring junior team members, and leading by example.
Please note you will only be considered for the position if you meet the minimum technical requirements. We offer a remote-friendly environment; however, employees must reside within the United States and be eligible to obtain or hold the legal right to work in this country.
The expected salary for this role is $210,000 - $235,000 depending on skills and experiences.
Benefits
- Early-stage startup with massive growth potential and ability to grow as Wizard grows
- Competitive compensation packages, including equity
- Health
- Comprehensive, high-quality medical coverage
- Dental & vision insurance
- OneMedical memberships for you and dependents
- Spring health platform for mental healthcare personalized to your needs
- XP Health eyewear benefits ($180, 3x per year)
- Rightway Health Guide
- Wealth
- 401(k) Plan
- Life & Disability insurance covered by Wizard
- Work/Life
- Flexible PTO and sick time to take care of yourself and your family
- 12 paid holidays
- 16 weeks parental leave for primary and secondary caregivers
Tags: Architecture AWS Azure Bitbucket Cassandra Computer Science Databricks Data pipelines DynamoDB E-commerce Elasticsearch Engineering ETL GCP Generative AI Git GitHub Hadoop Java Kafka Kinesis Microservices MongoDB NoSQL Parquet Pipelines Python RabbitMQ Spark TDD
Perks/benefits: Competitive pay Equity Flex vacation Health care Insurance Medical leave Parental leave
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Manager jobs
- Open Principal Data Engineer jobs
- Open Lead Data Analyst jobs
- Open Marketing Data Analyst jobs
- Open Power BI Developer jobs
- Open Data Science Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open MLOps Engineer jobs
- Open Data Scientist II jobs
- Open Business Data Analyst jobs
- Open Product Data Analyst jobs
- Open Junior Data Scientist jobs
- Open Business Intelligence Developer jobs
- Open Data Analytics Engineer jobs
- Open Sr Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Senior Data Architect jobs
- Open Principal Data Scientist jobs
- Open Sr. Data Scientist jobs
- Open Research Scientist jobs
- Open Big Data Engineer jobs
- Open Manager, Data Engineering jobs
- Open Azure Data Engineer jobs
- Open Data Quality Analyst jobs
- Open Junior Data Engineer jobs
- Open GCP-related jobs
- Open Data quality-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Business Intelligence-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open PhD-related jobs
- Open Deep Learning-related jobs
- Open Data visualization-related jobs
- Open NLP-related jobs
- Open PyTorch-related jobs
- Open Finance-related jobs
- Open TensorFlow-related jobs
- Open APIs-related jobs
- Open LLMs-related jobs
- Open Generative AI-related jobs
- Open Consulting-related jobs
- Open CI/CD-related jobs
- Open Snowflake-related jobs
- Open Hadoop-related jobs
- Open Kubernetes-related jobs
- Open Data governance-related jobs
- Open Databricks-related jobs
- Open Data warehouse-related jobs