What is Data Pipeline | How to design Data Pipeline ? - ETL vs Data pipeline (2024)

Apache Airflow, Kafka and Data Pipelines

What is Data Pipeline | How to design Data Pipeline? - ETL vs Data pipeline #datapipeline 📢📢 Subscribe to my FREE newsletter "Normal I.T. Guy" to know more about what I am thinking, learning and doing in my career & life !! CLICK THE LINK BELOW 👇👇👇 https://anshultiwari.substack.com ***Do check out our popular playlists*** 1) Latest technology tutorial (2020) - https://www.youtube.com/watch?v=l5UcUEt1IzM&list=PLcnJIHtHiTA2Hdgon51_-G3Kt43hcU6kI 2) Google Cloud Platform Beginner Series (2020) - https://www.youtube.com/watch?v=OzwSBbuHY-0&list=PLcnJIHtHiTA2Jp3klIkeHiwqGrxjXBzh6 3) Crunching Data Series (2020) - https://www.youtube.com/playlist?list=PLcnJIHtHiTA2HmIleev4scARSFwrQ0bIy Hi Friends, I am Anshul Tiwari, and welcome to our youtube channel I.T. k Funde. More about this video - In this video, we will understand what is a data pipeline with the help of a real-life example. Data pipelines are designed to move data from one point to another. In this video, we will cover below topics - 1 - What is a Data Pipeline? 2 - Why you need a data pipeline? 3 - Basic design of a data pipeline 4 - Types of Data Pipeline - Batch, Streaming, Lambda architecture 5 - Advanced data pipeline design We will also learn about various products that can be used in a data pipeline - SAP BODS, Mongo DB, Apache Kafka, Big Query, MDM, Teradata, SAP Business Objects, Tableau. Credits & Resources - Wikipedia https://www.alooma.com/blog/what-is-a-data-pipeline **********************FOLLOW US ON************************** Facebook - https://www.facebook.com/ITkFUNDE/ Linkedin - https://www.linkedin.com/in/ansh9685 Twitter - https://twitter.com/ansh9685 Blog - https://blogs.itkfunde.com/ Instagram -https://www.instagram.com/itkfunde/ **************************************************************** ******************About This Channel************************* Friends ITkFUNDE channel wants to bring I.T related knowledge, information, career advice, and much more to every individual regardless of whether he or she belongs to I.T or not. This channel is for everyone interested in learning something new!

About the course