yes. the problem is, we are fetching products from an API. and since we thought processing power will be a limiting factor, we thought that sorting out duplicates would reduce load.

but since the different microservices which process the data are taking different times we are using the sql tables as a pool. this should help upcscaling by using multiple microservices.

cloud services are yet not a solution as we are still in development.

haha. relating to a switch to ssd? or in which direction?

I think steam in general is a proof that its a service issue