JWT Authentication and authorisation for B2B systems

We are planning to use openID connect to protect our B2B REST endpoints(backend servers).

For the sake of simplicity naming our back-end servers as A and B

Step 1, to protect server A, Server A team Generate following in oidc system(CA SSO): ClientId and secret in oidc system. Ask for specific aud (ex:’serverB’) for server B.

Step 2, Share ClientId and secret information from Step 1 to server B.

Step 3, Server B generates token(jwt) using the clientId and secret from step 1

Step 4, Server A parses the jwt token, Server A checks for expiry time is not greater than current time, Server A checks for audience to match with the one created in Step 1(aud==’serverB’)

If in Step 4 passes then Server A proceeds with executing business logic otherwise throws Authentication error.

My question is, Is Step 4 validation is enough to protect a REST endpoint? is it OK to do validation within server A? generally there should be an authentication server to which Server A should send the token received from Server B, correct?

Few more information, The REST endpoints are behind firewall with https protection.