Async web requests in bulk - need sample codes

Closed Posted Dec 26, 2014 Paid on delivery
Closed Paid on delivery

I have an async code example which is attached to this bid. I wanted to change this code so that:

1) TestAsyncWithDelay function accepts a new proxy everytime ProcessUrlAsyncWithReqCount is called.

OR

2) TestAsyncWithDelay2 function runs with _activeRequestsCount>5000 without causing the program to crash.

Right now, for 1) above, I am unable to pass a separate proxy to ProcessUrlAsyncWithReqCount.

httpClient can be passed a proxy only when it is created (via the HttpClientHandler). The proxy apparently cannot be changed after the httpClient is created. Thats the problem as I cant have multiple httpclient objects created everytime a proxy is required. Hence I would need capability to create httpclient only once but yet fire [url removed, login to view] over a different proxy each time.

For the 2) above, although I can give a separate proxy to the HttpWebRequest for each request, the code crashes with Accessviolationexception when the number of concurrent request is set to anything above 1000. If 1) above is not possible, then 2) needs to be fixed so that the code does not crash in my machine (16 GB, Windows 7 Enterprise, 64 bit, i7-2.9, 4 core)

Code in VS 2013, c#. Vb.Net will do also.

Have changed the scope a bit to include the feedback I have received from the bidders

Small change in scope for people who have already bid on this:

Needed a class(c# or Vb.Net) that will use:

1) httpclient or httpwebrequest method to access URL async (should use the async method available from .Net 4.0 onwards to retrieve the contents of the page)
2) Multiple URLs will be given, and this would mean multiple such httpclient or httpwebrequest objects to be created that can process these request asyn
3) A throttling mechanism should exists to limit the number of concurrent URL hits (eg. 500)
4) Each call may or may not be behind a proxy, so code should be capable of haviing that support
5) When you try to call multiple URLs, there are chances that some tasks will get cancelled due to timeout - this would mean that since the number of async calls are too much, older calls are not getting network bandwidth to complete and times out. This should be very minimum (out of 5000 URLs, only 50 such calls may be cancelled - your code should be capable of handling that)
5) I will send you the proxy list, the URLs for this job
6) The whole thing should be inside a class called HttpBrowser (public methods: Navigate(URL, proxy), NavigareBulk(URLList, proxyList) and a test form

Visual Studio 2012 and above will be accepted.

C# Programming VB.NET

Project ID: #6907996

About the project

4 proposals Remote project Active Feb 1, 2015

4 freelancers are bidding on average $37 for this job

murtaza1981

Hi, Please feel free to discuss the project with me ............................................................................ Thanks, Murtaza

$30 USD in 1 day
(9 Reviews)
4.0
MSDev2012

Dear sir , I have created several systems before , I am ready to complete your application and i have strong knowledge with web development since 2007 using .net , sql and MVC ready to start working now thanks and rega More

$25 USD in 1 day
(20 Reviews)
3.9
AE310782

Dear Employer, I have the required experience in Async methods and how to manage these methods to work with specific data and how to send different parameters and values. I have tried to download and extract the at More

$77 USD in 1 day
(6 Reviews)
3.8
ekajgho

Over 14 years of experience in Microsoft Technologies. MCTS (Microsoft Certified Technology Specialist) on Web Applications Development with Microsoft .NET Framework 4 (Exam Code: 070-515). Functional experience incl More

$30 USD in 1 day
(1 Review)
1.1
patelbrijesh1309

Dear sir/ma’am, Thank you for the opportunity to submit a proposal for PHP Application development. Working together, I believe we can create an application that is simple to navigate and has professional yet welcom More

$100 USD in 7 days
(0 Reviews)
0.0
jolip

Please a.d.d Me S.k.y.p.e I.d---jolip05 for more discus *********************************************************************************************************

$10 USD in 1 day
(0 Reviews)
0.0