Collect Data and Organize Data

Hello! I am interested in a program using Java with threading to collect data from 2 websites and place data in .csv file.

PROJECT:

1) Collecting data

2) Organizing data and save as .csv file

## Deliverables

Hello!

I am interested in a program using Java with threading to collect data from 2 websites and place data in .csv file.

PROJECT:

1) Collecting data

2) Organizing data and save as .csv file

FORMAT:

Java with Threading (For FAST time)

I HAVE:

List of stock symbols I can copy and paste.

I) COLLECT DATA:

Column) Column Title

Data Source

Data Example

A) Symbol

(From list I provide)

MSFT

B) Total Short Interest

<[url removed, login to view]:US>

102635100

C) % Insider Ownership

<[url removed, login to view]>

14.00

D) % Institutional Ownership

<[url removed, login to view]>

55.40

E) Short: Prior Mo

(No Data - Only Column Header)

F) Shares: Float

<[url removed, login to view]:US>

8467873000

G) Avg. Daily Vol.

<[url removed, login to view]>

59,096,100

H) Sector

<[url removed, login to view]>

Technology

I) Industry

<[url removed, login to view]>

Application Software

J) Record Date

(Put "2007-Jan" in all cells)

K) Start Date

(Put "1/10/2007" in all cells)

L) Name

(No Data - Only Column Header)

M) Group

(No Data - Only Column Header)

N) Exchange

(No Data - Only Column Header)

O) Fake

(Put the number 1 in all cells)

P) Float2

<[url removed, login to view]>

8690000000

(#s have B, M or k: make B=Billion M=Million k=Thousand)

NOTE:

I know blooomberg blocks access after lots of data queries. Please make program so if data source "blocks" access, program will continue to collect data.

II) CLEAN DATA (IN ORDER SHOWN):

1) Remove all commas (,) from data

2) Remove all bad data & characters (ex. NA, ?, ...NA)

3) If B <= 0 then Erase B

4) If C <= 0 then Erase C

5) If C >= 100 then Erase C

6) If D <= 0 then Erase D

7) If D >= 100 then Erase D

8) If C+D >=100 then Erase both C and D

9) If F <= 0 then Erase F

10) If P <= 0 then Erase P

11) If B >= F then Erase F

12) If B >= P then Erase P

13) Last step after 1-12: Put a percentage symbol (%) in all cells that have empty cells (no data).

III) SAVE AS .CSV FILE For Excel Use

Complete!

QUESTIONS FOR CODERS:

1) I know one data source website above (blooomberg) blocks access during "large" data querries like this. Can you make program so id access triggers a block, program will work?

I have noticed with another program, that if block occurrs then i save data collected manually, then restart program where block occurred. This will work, but it is too labor intensive.

Are you able to create to accomodate for this?

2) I am curious approximatley how long would your design would take to parse the data for 5000 stock symbols?

1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done.

2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables):

a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment.

b) For all others including desktop software or software the buyer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request.

3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement).

## Platform

Java, Windows OS

Amazon Web Services Database Administration Engineering Java MySQL PHP Software Architecture Software Testing SQL Web Hosting Website Management Website Testing

Project ID: #2804757

About the project