A production-style YouTube ETL pipeline built with Python and SQL Server. The project extracts channel information, playlists, playlist items and videos from the YouTube Data API v3 and loads them ...

aws s3 cp CAvideos.csv s3://youtube-raw-us-west-1-<aws-account-id>-dev/youtube/raw_statistics/region=ca/ aws s3 cp DEvideos.csv s3://youtube-raw-us-west-1-<aws ...

MotherDuck is launching Flights, an agent-native data pipeline that enables users to choose the MCP server and AI agent of their choice to build and deploy data pipelines in minutes using a flexible, ...

A new framework called SkillWeaver tackles AI agent tool routing by skipping full-library loading, cutting token use 99% on ...

Overview:Compare responsibilities, technical skills, and career opportunities across Data Engineering and Software ...

Preventing agent-generated infrastructure bloat through spec-driven governance AI agents waste massive cloud space, so block this bloat early with strict policy checks, illustrated using Terraform and ...

AI agents are now taking over repetitive work, identifying issues humans may miss, and helping teams maintain testing speed ...

TECH & Engineering. full-time Lissabon, Portugal. As a Process Intelligence Analyst & Engineer, you bridge the gap between ...