A Project Structure which helps you reuse work, looks fancy and lets you handle multiple AMIs gracefully.
Steps and resources which will help you understand Luigi beyond the basic concepts.
Important concepts and best practices which will help you get started and become comfortable with the tool faster.
Is it expected behavior that my Luigi tasks are vanishing from the web interface?
How to make sure that your Luigi Task runs again, despite having finished successfully before?
No. You will be able to handle large amounts of data, using whichever tools you need.
An easy way to combine similar data from multiple Luigi tasks.
How to process custom and Golang-style log files.
Sometimes, 1024 GB are just not enough.
How to use the _requires() function to create non-input dependencies for a Luigi task.
A quick code example on how to get data out of PostgreSQL with a simple SQL query.
Is the data you work with distributed across several services and databases and tedious to use?