need someone who has used and can set up on AWS 2 node cluster, apache beam with hop and create two pipelines with java code.
1. load data, input will be excel (with 3 columns having file names that are relative paths to 1 of 2 zips) for simplicity the zips and excel will be on a shared path that both nodes can access. This job has to be configured so that each row is part independant task and can happen seperately/ resume etc
2. read a csv, call a few apis to get attachments referred in the csv (pdf, images) save and zip those files. This is a cumulative job: all rows of csv have to be processed and then zip (or zips depending on number and size of files) has to be made as last step. Logic to save file, create zip will be provided
Need help with setting up AWS environemnt on ec2 nodes
Programming in Java to set up this cluster
Divide and monitor jobs between the 2 nodes
To sign NDA before work so can share some company private swagger info
Java 11 or 17 with maven or gradle (prefer gradle)