Write some Software

Objective The objective of this project is to build a business process flow engine for lolagrove processing engine by using windows workflow foundation. Description The business process flow engine is primary control service within our processing engine. This will take user data and configuration data as input and process the data in a sequence of steps. In each step, the process calls a service and get the result. Based on the result of a step, it will either exit the process or continue the process or skips number of steps within the process. It saves the data in sql server database after finishing each step. As part of data processing in a step, the process may wait for long periods to get response from the third party services. The process server should be able to handle sending messages to third parties and wait for the response. The process flow engine is required to be able to handle thousands of data records concurrently as fast as possible. It is planned to handle at least 200 records per second. As soon as the record (data) passed to the engine, it should assign data record to an appropriate instance of a process flow. After each step (activity) within the process, the process status on that record should be persisted until the full process is finished. If an activity is defined as long running or it is waiting for response from third party service, it should put the processing of the record in hibernation and save status in persistent data store. The response to this activity should make the process start processing on the record from the step where it went to hibernation. Once the process is finished or if an activity defines to store the data in permanent data store, the process should be able to store the data in permanent data store (SQL Server). Performance Requirements • The process should be able to handle at least half a million records per an hour. • The process should be able to handle at least 1000 records concurrently • The process should be highly available : 100% availability Process Flow Diagram As shown in the attached diagram (base process flow [login to view URL]), the process consists of pre-processing, processing and post processing steps as main steps in the process. Each of these steps is another set of steps. Each step calls a service and gets result. Dispatcher calls the proposed business process flow engine. In data segregator step, the process calls data segregator service to find out if the record can proceed to next step of the process flow. Then it moves to pre-processing which is another process flow engine, which contains multiple steps. These steps are different for each data record. The Pre-Processing step pre-process data and sends to dispatcher to dispatch to permanent data store. If the record found not good to proceed to next step, the process stops here for the particular data record. If it continues to next step, it goes to processing which is another process flow engine. All rectangular elements, in the above attached diagram (base process flow [login to view URL]) are different process flow engines works separately.