How to run serial (multi-step) background jobs?
After a user submits a URL via a form I need to run a handful of jobs in series
What would be a good way to do this?
- User submits URL
- Use the URL to build file paths
- Download file 1 from file path and upload to S3
- Download file 2 from file path and upload to S3
- Call an API endpoint with the S3 URLs
Steps 3 and 4 could be done in parallel but they both need to be complete before step 5 can run so I'm just going to keep it simple and run all in series.
The naïve option seems to be simply call each step at the end of the previous one. I found this article on multi-step jobs but it seems overly complicated.