Twitter Data Pipeline using Airflow for Beginners | Data Engineering Project

Apache Airflow, Kafka and Data Pipelines

Join Course Python for Data Engineering - https://bit.ly/4140KI9 This is End-To-End Data Engineering Project using Airflow and Python. In this project, we will extract data using Twitter API, use python to transform data, deploy the code on Airflow/EC2 and save the final result on Amazon S3 Code and commands used in the project - https://github.com/darshilparmar/twitter-airflow-data-engineering-project Other resources: Twitter API - https://developer.twitter.com/en/docs/twitter-api Python (Installation & Basics) - https://www.youtube.com/watch?v=eInHnOxTA-0 Setup AWS Account - https://www.youtube.com/watch?v=kbtDJHgjrPc Airflow Doc - https://airflow.apache.org/docs/ Join Data With Darshil Discord Server: https://discord.gg/dHjvD8JCG3 Timestamps 0:00 Project introduction 0:56 Section 1 (Prerequisite) 2:21 Section 2 (Understand basic concept) 7:55 Section 3 (Project execution) 👦🏻 My Linkedin - https://www.linkedin.com/in/darshil-parmar/ 📷 Instagram - https://www.instagram.com/darshilparmarr 🎯Twitter - https://twitter.com/parmardarshil07 🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟 3 Books You Should Read 📈Principles: Life and Work: https://amzn.to/3HQJDyP 👀Deep Work: https://amzn.to/3IParkk 💼Rework: https://amzn.to/3HW981O Tech I use every day 💻MacBook Pro M1: https://amzn.to/3CiFVwC 📺LG 22 Inch Monitor: https://amzn.to/3zk0Dts 🎥Sony ZV1: https://amzn.to/3hRpSMJ 🎙Maono AU-A04: https://amzn.to/3Bnu53n ⽴Tripod Stand: https://amzn.to/3tA7hu7 🔅Osaka Ring Light and Stand: https://amzn.to/3MtLAEG 🎧Sony WH-1000XM4 Headphone: https://amzn.to/3sM4sXS 🖱Zebronics Zeb-War Keyboard and Mouse: https://amzn.to/3zeF1yq 💺CELLBELL C104 Office Chair: https://amzn.to/3IRpiL2 👉Data Engineering Complete Roadmap: https://www.youtube.com/playlist?list=PLBJe2dFI4sgtlK_zaqaIBdJFgieYPnQ07 👉Data Engineering Project Series: https://www.youtube.com/playlist?list=PLBJe2dFI4sgukOW6O0B-OVyX9c6fQKJ2N 👉Become Full-Time Freelancer: https://www.youtube.com/playlist?list=PLBJe2dFI4sgtza0sAnNFwo8KPG0GcO9Il 👉Data With Darshil Podcast: https://www.youtube.com/playlist?list=PLBJe2dFI4sgv_XmEDaXF3z1MNib7R3KUY ✨ Tags ✨ data engineering project, complete data engineering project for beginners, learn data engineering, data engineering roadmap ✨ Hashtags ✨ #airflow #dataengineering #project

About the course