How to store real-time data to database

Here is my situation:

  • I have a WebSocket connection that feeds me data real-time
  • I have an async callback function that gets the data and inserts into a queue
  • I have another thread that reads from the queue and stores it into the Postgres database (using python’s psycopg2 library)

The problem is the real-time data comes in faster than the time it takes to insert into the database, and in a couple of hours, the server runs out of memory. (Is this because psycopg2 library is slow?)

A simple solution is to create more threads that insert into the database; however, this will cause the data to be out of order. Is there a database that sorts the data? Other suggestions would be very much appreciated.

By the way, is there a way I could build a WebSocket client using docker/Kubernetes that can do load balancing for me?