United States

QA Engineer

Bachelor's degree

Naresh Dulam

Vice President Sr Lead Software  Engineer

Communication

Problem-solving

Time management

Leadership

AI/ML

Python

Spark Hire

Databricks

Java

HTML

JavaScript

Matplotlib

Empathy

Public Speaking

Conversational AI

Machine learning

Making remote and globally distributed teams work for you and your needs.

Working remotely

Navigating micro-aggression, getting paid what you are worth and finding a community.

Being a woman in tech

Tips for transitioning into a new role or company.

Changing roles

How to successfully prepare and position yourself for a job promotion?

How to get promoted

How to ask for a promotion and negotiate for salary raise?

Negotiating a job offer

Get actionable feedback on your resume or portfolio.

Resume and portfolio review

Practice HR, technical and behavioural interviews with mentors.

Practice mock interviews

How to create a positive workplace culture that drives collaboration and productivity?

Fostering a good team culture

Steps to building and maintaining a strong, cohesive and effective team.

Building an effective team

Learn the tips to successfully manage a team in a remote environment.

Managing a remote team

Tips on how to work and collaborate with designers.

Working with designers

Managing being the only designer at your company and working with non-designers.

Being the only designer

Tips on how to work and collaborate with engineers and developers.

Working with engineers

What is a successful user onboarding flow and how to achieve it?

Onboarding new users

Best practices for documenting projects, epics, and stories

Writing product requirement documents

Built WM Risk & Control Analytics is next gen data platform, Controls Data as Product from ground up to replace legacy data warehouse system, and Hadoop as part of modernization with seamless user experience for 1500+ business users. This helps the organization to reduce vendor license and consulting costs. With Data mesh architecture of New Data Analytics platform built on Data mesh principles reduce the storage cost, no duplication of data and with open-source cloud native technologies.
o
Envisioned, Architected, and operationalized the WM Data Analytics Hybrid Platform to run the MIS Reporting, data science and analytical usage for Private Bank.
o
Data lake with capacity of 1.5 PB supporting more than 2000 users on-prem (PB/IPB Data) and Cloud Data Lake for Market/Index data serving 200+ data scientists for AI/ML Use cases.
o
Designed and built a generic data ingestion pipeline framework, which helped in quicker data onboarding on to the platform.
o
Architected the design pattern for self-serve for both ingestion and consumption. This design pattern was made a standard for other technology teams, thus reducing build time for multiple teams across lines of businesses.
JPMC Advanced Data Ecosystem is a firm wide public cloud -based data platform comprised of reusable components required by LOBs to establish and machine learnable datasets to the public cloud – making it available for AI/ML, analytics, BI Use cases.
As part of cloud migrations all the app teams are required to build the cloud pipelines which is duplication of work. A framework required which adheres to JPMC security standards.
o
Architect/Develop the cloud pipelines for LOB to use the same pipelines in their cloud migration journey.
o
Responsible for building containerized applications for both data ingestion to cloud and transformation.

Migrating the existing containerized application running in private cloud to public cloud AWS (Use case Airflow with OIDC Authentication)

Provider Analytical platform is used to help 3rd party vendor, claim processing. This initiative will help the users to give a 360 degrees overview about claims and vendor consumption.
o
Responsible for the implementation of the Data Ingestion framework using spark to consume the xml and mainframe files.
o
Developed custom input formats to read ORC data from Acid tables in Hive for Spark jobs.
o
Designed and developed physical and logical models for data used in different extracts.

During the Covid pandemic, due to sudden change in the markets, the front office team wanted to be able to do lot more analysis, bring in their own data and identify patterns using ML and AI algorithms. Though we had data in our platform, the way they can leverage it was limited since they lacked the right tools and set up. We designed and built sandboxes in a matter of weeks to help our advisors deliver to the clients.
o
Monetized data and made it available for business quants and analysts via a self-serve platform.
o
A huge value-add for the analysts which helped them generate more revenue by providing targeted WM Solutions.
o
Completed design, build, and roll out in less than 3 months, done over multiple iterations.

As a Senior Big Data Consultant build the data marts for different applications teams across ATT.
o
Migrate existing MR jobs built on vendor framework to Open-source spark jobs.
o
Providing data sets available from different sources on to Hadoop as per Adhoc customer requirements.
o
Developed monitoring and alerting framework using Kafka connect that is being used by operations teams to monito application in production.
o
Build the Sqoop to bulk load data HBase for complex data types.

Hadoop as a Service(HAAS)- Hadoop as a service is a new initiative in BoA to build one centralized cluster across all the applications. As part of this initiative all the Hadoop clusters (53) across bank will be merged into HAAS cluster. This enormous cluster configuration, resources, tools, standards are defined by HAAS Architecture Team.

As a key member of Architecture Team I did get an opportunity to work on below tasks. Biggest challege on the platform is to support multi tenants on the clusters

•	Worked on evaluating frameworks like Spark, Spark Streaming, Kafka, Flume, HBase and other tools to empower tenants of HAAS Clusters.
•	Actively worked with Cloudera support to debug multiple P1, P2 issues on the clusters.
•	Analyzed dynamic resource pools (fair scheduler) in depth and proposed resource pools to handle various workloads from varied tenants. 
•	Reviewed and recommended Hadoop cluster configurations for optimal utilization.
•	Worked on defining standards and guidelines for Hadoop and Hive
•	As part of security tool review, evaluated flume and its defined work around for the security loop holes. 
•	Configured flume nodes with multiple flows, used morphline to convert Json data to avro data.
•	Worked on Design to provide High availability for critical flows that cannot have any outage.
•	Worked on Design to provide Active –Active cluster where both production and DR clusters are made available all the time.
•	Worked on HDFS Encryption POC, to support PCI guidelines on the cluster.
•	Evaluated multiple projects and provided consulting work for various internal teams on implementation of a Bigdata projects.
•	Implemented Spark POC using Scala as part of tool acceptance test.
•	Worked on HDFS Encryption POC to support PCI guidelines on the cluster.
•	Evaluated multiple projects and provided consulting work for various internal application teams on implementation of Big data projects.

Hadoop developer

Exported data using Sqoop from HDFS to Teradata on regular basis.

Converting map reduce/pig/jobs into Spark jobs; also transform the Hive jobs into Spark SQL

Implemented POC using Spark and Cassandra

Write Hive queries for data analysis to meet the business requirements. (also create tables using Hive QL)

Create the workflow to run multiple Hive and Pig jobs, which run independently with time and data availability.

Developed PIG Latin scripts for the analysis of semi structured data; involved in debugging PIG scripts

Worked closely with data warehouse architect and business intelligence analyst to develop solutions.

Design, develop Hive Data model, loading with data and writing Java UDF for Hive

Developed Pig Latin scripts and used Pig as ETL tool for transformations, event joins, and filter.

Designed and Developed Sqoop scripts to extract data from a relational database into Hadoop.

•	Developed multiple Map Reduce jobs in Java for data cleaning and preprocessing
•	Importing and exporting data into HDFS and Hive using Sqoop Jobs
•	Extracted files from Oracle DB through Sqoop and placed in HDFS and processed Developed Hive queries, Hive UDF’s according to business requirement
•	Developed PIG Latin scripts for the analysis of semi structured data. Involved in debugging PIG scripts
•	Involved in designing data modeling for hive/cassandra tables
•	Troubleshooting the  existing jobs while migrating them to higher version
•	Developed pig scripts to preprocess data before moving the data into final tables
•	Load and transform large sets of structured, semi structured and unstructured data
•	Supported Map Reduce Programs those are running on the cluster
•	Wrote shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions
•	Involved in loading data from UNIX file system to HDFS, configuring Hive and writing Hive UDFs
•	Utilized Java and MySQL from day to day to debug and fix issues with client processes
•	Managed and reviewed log files
•	Implemented partitioning, dynamic partitions and buckets in HIVE

o	Oracle Certified Java Professional with over all 8 years of experience in Information Technology and a wide range of progressive experience in providing product specifications, design, analysis, development, testing, documentation, coding and implementation of the business technology solutions.
o	I have well experienced in training, mentoring and motivating other members of the team and other teams to achieve the goals of the company.
o	Worked with development and maintenance teams for developing and maintaining in Oracle Fusion Product development.
o	Performed admin tasks like installing and configuring the Weblogic server and Fusion Tech stack installation for Fusion Applications.
o	We-versed in SQL/PLSQL.
o	Good Exposure to Big Data technologies like hadoop.

Naresh Dulam🇺🇸

About Naresh Dulam

Expertise

Sessions with Naresh Dulam