United States

Data engineering

Data Product Management

Back-end

Master's degree

Bachelor of Engineering (B.E.)

Vipul Bharat Marlecha

Senior Big Data Engineer

Communication

Problem-solving

Organisation skills

Leadership

Advanced data analytics

AI/ML

Database management

Airflow

Apache Spark

Axure

Databricks

Java

GitLab

PostgreSQL

Understanding factors that lead to a motivated team - recognition, communication and more.

Keeping team members motivated

Strategies for maximizing your design internship or co-op program.

Getting the most out of my internship

Tips for transitioning into a new role or company.

Changing roles

How to successfully prepare and position yourself for a job promotion?

How to get promoted

How to ask for a promotion and negotiate for salary raise?

Negotiating a job offer

How to think about design career paths and what progression looks like for designers?

Design career path

Learn the career path and progression for product managers.

Product management career path

Building massive scale real-time and batch pipelines for one of the most important teams (Open Connect) at Netflix. This data is critical to driving optimization across Netflix Video Streaming, Streaming Algorithms, CDN-level TCP Connection, Experimentation, Content Steering, and Content Placement.
• Building low-latency, multi-region, and highly resilient Flink pipelines that process billions of events per days (CDN network) for offline and production use cases.

Technologies Used: Flink, Spark, Elastic-Search, Apache Iceberg, Data-Lakes, Kafka, Apache-Druid

Built petabyte scale, distributed and highly resilient data solutions to power Merchant team’s data needs.
• Initiated and lead the cross-teams project (15+ teams from engineering to support) to make menu data accessible for analytical and near real-time use-cases by leveraging lambda data processing architecture.
• Architected and built real-time services using flink, that consumes data from multiple Kafka topics by doing a streaming join, hydrates, processes and sinks data back to Kafka for ingestion to Apache Pinot and Snowflake. I have optimized this service to process ~5 billion events per day, achieved end to end latency of less than ~2 minutes and have scaled our service to 100 + distributed Kubernetes pods.
• Wrote spark job to join and ingest menu data for all our merchants globally, this job made up the batch layer of menu lambda architecture and ingested 40 billion rows per day. I optimized this job extensively which end up saving us around 5x cost.
Technologies Used: Flink, Cassandra, AWS, Databricks, Kubernetes (EKS), Spark, Jenkins, Terraform, Java, Snowflake, Elastic Search

Built real-time and offline solutions to make twitter's revenue data accessible and reliable while leveraging the largest-scalable distributed data processing technologies in the world.
•	Built a scalable data ingestion platform for all revenue teams which runs managed batch jobs to move a large number of datasets of size ~500 TB/hr from on-premises to the cloud. This platform uses dataflow with Apache beam for data transformation (Thrift->Avro), Big Query for data warehousing, Kafka for messaging when the partitions are available on-premises, and Apache Airflow for orchestration 
•	Worked on building a petabyte-scale distributed data warehouse for revenue org by leveraging Big Query.
•	Worked on large scale batch processing pipelines to flatten and move around ~150TB data per hour from on-premises to cloud.
•	Build various large scale tools and libraries that involve Big Query Onboarding tool, HDFS to Big Query data migration tool, Big Query to Druid data migration tools, and various serialization frameworks (e.g. thrift, Avro).
•	Maintained  one of the largest druid (Scalable Distributed Analytics Platform)  cluster in the world  ~6000 nodes.  Maintenance involves handling scalability issues, performance issues, and ingesting various aggregated datasets at scale.
•	Leveraged our internal batch computation frameworks (scalding) and our workflow management platform to assist other teams in building out their data pipelines.
Technologies: BigQuery, Beam, Dataflow, Airflow, GCP, Scalding, Map Reduce, Scala, Batch Processing, Real-Time Streaming, Data Engineering, Hadoop, Druid(Distributed analytics platform), Kafka.

•	Architected and created the database which is used by the global audit team, analyzed and interpreted data feed sources to migrate data into MS SQL Server database using daily automated import processes.
•	Developed automated ETL (Extract-Transform-Load) processes to move data between GRC system and Oracle database thus reducing the load time by 60%.
•	Wrote data load and transformation scripts in SQL Server Integration services (SSIS) to cleanse and standardize the data in accordance with the business rules thus reduced the manual data clean up time by 80%.
•	Created clustered and non-clustered indexed on tables for faster searching and retrieval of data.
 Develops queries to research, analyze, and troubleshoot data and to create business reports; and working with the development team to investigate, correct bugs and deficiencies.
• Generated ad hoc reports using complex queries to support auditors globally for budget tracking, issue tracking and getting useful analytical insights leading cutting budget by 10%.
• Developed executive reports and dashboards in Qlikview to visualize data and measure the performance of business in real time thus reducing the manual report creation time by 50%.
• Wrote and analyzed trending reports, drew extrapolations from findings with the project manager and the team (SSRS, Excel, T-SQL, Queries).
Environment: T-SQL, ETL Development, SQL Server, SSIS, MS Access, MS Excel, Qlikview

•	Customized “EXO”, an open source enterprise platform used by around 18000 internal users, covering development of new features, improving the backend logic, and performance tuning to handle multiple requests across the platform which improved the performance by 40%.
•	Developed “TWIKI converter”, a tool that that parses, converts and migrates more than 10,000,000 files from TWIKI (Structured Wiki Application) to the enterprise platform using REST Web Services.
•	Implemented and integrated multiple Web applications on different Tomcat servers to a single Apache Web server (v1.3) and wrote rules for redirection of HTTP requests to the appropriate Web application.
•	Created responsive front end design and features with emphasis on cross-browser compatibility.
•	Worked with QA to design, plan, and automate functional and performance testing.
Environment: CSS, HTML, Java, JavaScript, JQuery, JSP, MVC, XML

Vipul Bharat Marlecha🇺🇸

About Vipul Bharat Marlecha

Expertise

What mentees say