How to protect certificates and keys in peer to peer application

I’m making a peer-to-peer cross-platform application (in Java & Kotlin), and I want to encrypt conversations between tens of users, concurrently.

However, due to lack of knowledge in security good practices & protocols and due to this being my first time actually getting into informatics security, I’m sort of confused.

My goal is that every peer connection has a unique session that shares as little as possible to the other peer connections, in order to minimize any risk if one connection proves to be vulnerable. The connections will be implemented using TLSv1.3 or TLSv1.2 (I do not intend to support lower protocols due to the risks involved with using them).

However, I’ve done some rudimentary research on my own, and I cannot wrap my head around the question, is having a keystore and a truststore on the (classloader) directory of my application a security vulnerability? Could it ever be one?

I am aware that keystore stores the private and public key of the server, and truststore stores the public key of the server, which it verifies & uses when contacting the server. How can I protect my keystore’s and truststore’s certificate password, when they must be on the application’s directory? Does it need to be protected, even?

What encryption algorithm should my keystore use? I’m heading for really strong encryption, future proofing as much as possible along with keeping up as much backwards compatibility as I can without reducing the application’s security.

Is there an issue with the certificates being self-signed considering I’m solely using them between peers of the same application?

Considering I’m doing Java, do SSLSockets/SSLServerSockets create a "brand new session" for every new connection, as in, do they reuse the private or public key? Are private keys generated when making a handshake with a client?

Thank you for taking the time in advance, privacy is a really big focus of the application itself.