What it needs to support:
1. I need a way to send standard HTTP requests with an URL and some parameters (POST/GET/PUT, etc). At best, I would like this call _NOT_ to be asynchronous.
2. A way to store cookies from request to another, like a cookie jar;
3. I need to get the raw body of the request - the way server responded to the client;
4. I need also a formatted HTML (tidy/*) of how the document actually looks like.
I would like the bidders to start their bidding post with "+++". This is how I can tell if you actually read the post. I will simply ignore/remove the others - there are too many to read and many of them are nothing but automated posts/spam.
The language you choose is not important as long as it respects the things above. I would prefer Java or Python, but I can accept others - just let me know about this in your post. Again, this will be running in an linux docker, so be prepared for this.
I am a programmer myself and I know this can be done. I did it in the past, meanwhile technologies changed and it may take a while until I am up to date with everything. This is the reason why I need your help. Also, this means I have a fair idea of the effort needed, so don't go wild with your bid here.
* full source code;
* a small demo in docker on how you will crawl [login to view URL] - 4-5 pages at your choice, no need to interpret the results;