Use Aws glue for ETL

Hi, I am an experienced developer in big data tools. I have utilizes spark, Metastore and glue for a lot of times for etl pipelines. I can surely do poc first and then do rest of mappings. Let's discuss the architecture and go a head

Hi Dear, I am AWS Glue expert. I have migrated 19TB database from xml source to postgresql database using AWS glue. The source files were generated from cobol database. AWS Glue works on their Lambda based architecture. So performance is a major plus point in AWS glue. The another good thing is, since it is managed by AWS only, there is no need of looking after infrastructure. Your first 3 questions are dependent on your data source. I will explain once I have insight of your current need. Workflow standpoint of view, Glue supports many sources like xml, csv, relational database like ms SQL, my SQL, pg SQL etc. So based on your data structure and database infrastructure, we will decide one of the workflow to transform data from your source to destination. Glue has provided their own transformation logic that we need to consume in Python. So comparitively it gives more filtering and sorting to transform data in right structure. Please let me know your convenient time to discuss further so we can plan for POC. Thanks, Vimal Santoki

Hi, This is Vipin, an alumnus of IIT Kanpur. I have 6+ years of experience in customized software, web and mobile app development and has expertise in Android, iOS, Python, Magento, AWS, PHP, HTML, Java, Angular and Ionic framework. I would like to work on your project and looking for long term work. Kindly connect to discuss in detail and check my recent work in portfolio section as well. Hope to talk more soon Thanks

Hello, sir. How are you? I have just read your project description and I can help you with my experiences. Don't waste your golden time to looking for a developer! I'm new freelancer, but I will make the perfect product you are satisfied with. My clients have never been frustrated with my work and I hope to make you one of them. I'm always ready for you, So we can discuss more details. Thanks and Regards

Hi, this is the Migration project and I already worked on this type of task. If you can make contact with me we can close this task soon. Regards, Rahul Soni 9148250559

Hi, Thanks for sharing your requirement and I will be more than happy to help you. Let me share with you my expertise which I have gathered for over 6 years in Informatica ETL, Hadoop & AWS : - Hadoop Ecosystem : Yarn, SPARK, HDFS, Hive, SQOOP - ETL Tools : Informatica PowerCenter, Informatica BDM, Informatica EDC - AWS services : Glue, EMR, Lambda, Step Functions, S3, CloudWatch, EC2 I can start with the POC, and then move forward. Please feel free to ping me for the same. Regards, Gunjan Khandelwal

Hi, I am interested to work on the project. Though I am new to freelancing, I have worked on ETL projects on my job. Looking forward to work on the project. Thanks, Rajashri

Came across your need in ETL, would like to help on it. I had been been working with MNCs from past 4 years and had seen diversity of projects.

Hi. My name Vitaly. I'm ETL Informatica developer from Ukraine. I have good experience in Informatica, and I worked with AWS Glue also for migrating the data from databased. My detailed CV published on Linkedin. Please answer in chat, I'm online during US work hours. Best regards

Hello, Warm Greetings! I am a web developer and I have 4+ years of experience in this field. My skills are PHP, HTML5, CSS3, Bootstrap, Javascript, jQuery, CMS, node js, React js, AWS, MongoDB.  So I can do your project in a short time and I can start work immediately. I hope we can discuss your project in more detail via chat. Looking forward to hearing from you.   Thanks

Reasons WHY I am Fit for this project. Please find the reasons why I find this project a prefect one: 1. This project requires a dedicated person for a long time, so I am available for more than 6 months for this project. 2. I am available for 40 hrs a week. 3. I have production development experience in all the technologies required in this project, so it will be very convenient for me to handle this project. 4. I ensure quality work and given time lines, so you can be assured for on-time delivery of your project with quality. 5. I have recently developed Amazon cloud based project, where a data pipeline was created to extract and ingest data then further cleaning and finally generating reports for business analysis. The technologies used were : AWS services like Lambda, Glue, S3 buckets, Azkaban, Apache Spark, Python and a lot more. Additionally About My Technical Skills: I have 8 years of IT experience, during which I have designed and developed projects that are based on platforms like : BigData Hadoop, Amazon Web Services (S3, Glue, Lambda), Google Cloud Services, Apache Spark, Data Lakes, Data Marts and development of ETL pipelines, data ingestion frameworks using Java, Scala, Play and many more. So, you can consider me as a right person for your project, in terms of technology, experience and honesty.

Hi ! I have good experience into AWS Glue ETL, data crawlers and GlueETL transformations and have recently worked on similar assignment. Please share more details to do the POC and I will provide you high quality results.