Tech stack I worked with Kafka for real-time event streaming Flume for ingestion into the pipeline Spark Structured Streaming for processing data in real time Hadoop HDFS as a data lake InfluxDB for ...