Collection & Analysis of Texts from the Congressional Talks

Project Description: Hello all: I am a researcher at a university and I need someone who is experienced in crawling and data collection to help me with the following: 1. From the congress database (open to public) collect the congressional speech texts from the past 5 years. 2. Find in the texts, the most commonly used 2 / 3 word phrases by the democratcs and the republicans for each year starting 2006 -2011. In specific you will, 1. Go to [login to view URL] Steps: 1. Separately for the Democratic members and Republican members (Letters R and D) 1.1 Select "house" and "senate” 1.2 Specify the date (each year separate) 1.3 Download and combine all the records for each year. You will need to click on each and then download the text. For each file there is a text or pdf option for each of those records. 2. Collect the speeches in one document, and then take out the most commonly used words. 3. Separately for Republicans and Democrats, identify the top 500 most commonly used phrases for the Republicans and for the Democrats for each year. (No need to enter keywords to search, or select congress, or pick date received.) If we can manage the above, I have follow up projects that I can potentially work with you given we mutually agree on it. I am looking for someone I can work with for a long term if I am happy with the work.