1. product-info-api will read products csv file via spark structured streaming (readStream) and write (writeStream) all records to kafka topic with specific batchId. This event will be triggered via POST method by uploading csv file.
1. product-info-api will read products csv file via spark structured streaming (readStream) and write (writeStream) all records to kafka topic with specific batchId. This event will be triggered via POST method by uploading csv file.
2. This API will consume the kafka topic via sspark structured streaming (readStream) and write (writeStream) transformed records to another csv. This event will be triggered via GET call by passing batchId
2. This API will consume the kafka topic via spark structured streaming (readStream) and write (writeStream) transformed records to another csv. This event will be triggered via GET call by passing batchId
3. Postman script and input batchId are available in src/main/resources/postman folder.
3. Additionally GET method will return processed data with CollectionAccumulator concept in spark.
4. Postman script and input csv files are available in src/main/resources/postman folder.