Description here

Responsibilities

Duties/Responsibilities:

· The Data Engineer will be responsible for writing Data ingestion pipelines using python and the necessary libraries to obtain Transactional and historical data from our commercial tools.

· These ingestion pipelines will either use RESTful or Graph APIs to connect to different sources of our transactional data.

· The Data engineer will also be responsible for designing the schema and setting up a postgres DB to store the transactional data from various sources.

· All IHM deployments and hosting are in AWS and scripts will be executed as Lambda functions

· You will be assisting with the development of data visualizations so you will be working in an environment using Tableau, Qliksense, Jupyter notebooks with beakerX.

· You will help in evaluating different options to choose Relational DBs or Document DBs based on business need and then implement the approved designs.

· You will be responsible for setting up DB backups and Geo redundancy within AWS.

Skills and Experience

Required Skills/Abilities:

· Must have worked on AWS, EC2, Lambda function, SNS, S3 buckets and other services.

· Strong experience in Python libraries – Numpy, Pandas, Matplolib

· 5+ years of building data structures that supply data visualization

· Environment like Jupyter Notebooks with BeakerX extensions is a strong plus

· Well versed in application frameworks, such as Micronaut, Grails, Spring, Django

· Must have experience in Kubernetes and containerization.

· Must have worked with Github and Git actions for deployments.

· Must have worked with different AWS services - RDS, document DB, Aurora

· Must have worked on security standards like OAuth2, Client credentials for API calls

· A strong data science background such as running regressions and being conversant

with linear algebra.

· Experience in Golang, JAVA or other OO Programming languages besides python is a must.