Data & MLOps Engineer building scalable ML systems. Passionate about cloud, data platforms, and responsible AI. I have deployed Kafka pipelines that ran cleanly in staging for two weeks. No lag. No ...

A monthly overview of things you need to know as an architect or aspiring architect. Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with ...

This repository is to illustrate the basic concept and implementation of the solution of config-driven data pipeline. The configuration is a JSON file that contains the information about the data ...

In this article, you learn how to create an Apache Spark configuration for your synapse studio. The created Apache Spark configuration can be managed in a standardized manner and when you create ...