- Analyze Business Requirement Documents and Implement Technical Solutions for privacy related applications.
- Develop ETL process for supporting Data Extraction, transformations and loading.
- Perform data conversions and aggregations using different transformations such as Merge, Merge join, Union condition split, sort, order by. Derived columns convert and cast transformations and row count and lookup and fuzzy lookup transformations.
- Develop UNIX scripts to load the data from Source server to Teradata and validate the files between different servers.
- Develop new process to implement state level privacy regulations based on each state law in Big Data Platform.
- Create Temperory/Fact tables, loading with data and writing Teradata and Spark SQL queries.
- Optimize/tune ETL objects, indexing and partitioning for better performance and efficiency.
- Validate the performance metrics and work on performance tuning for SQL, HQL and Spark SQL queries.
- Perform testing and Provide test support for various level of testing phases like Unit, User Acceptance, Regression, Parallel and System testing.
- Promote the components to production environment through CI/CD process by using Git hub .
- Script task and execute SQL tasks to execute SQL code. Work on containers for loop and for each loop container to run a group of tasks into a single container and repeating tasks.
- Create the data flow to extract data from sources to OLEDB Source, Excel, XML, flat files sources and destination is SQL data warehouse.
