ReplaceAll doesn’t replace all, factor out negative sign first

I have this matrix:$ $ \left( \begin{array}{ccccc} \{\{0\},\{0,0,0\}\} & \{\{1\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} \ \{\{-1\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} \ \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,z,-y\}\} & \{\{0\},\{-z,0,x\}\} \ \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,-z,y\}\} & \{\{0\},\{0,0,0\}\} & \{\{0\},\{y,-x,0\}\} \ \{\{0\},\{0,0,0\}\} & \{\{0\},\{0,0,0\}\} & \{\{0\},\{z,0,-x\}\} & \{\{0\},\{-y,x,0\}\} & \{\{0\},\{0,0,0\}\} \ \end{array} \right)$ $

When I run ReplaceAll (/.) on it using this$ $ \left\{\{\{1\},\{0,0,0\}\}\to X_1,\left\{\{t\},\left\{\frac{2 x}{3},\frac{2 y}{3},\frac{2 z}{3}\right\}\right\}\to X_2,\{\{0\},\{y,-x,0\}\}\to X_3,\{\{0\},\{z,0,-x\}\}\to X_4,\{\{0\},\{0,z,-y\}\}\to X_5,\{\{0\},\{0,0,0\}\}\to 0\right\}$ $

it doesn’t replace everything:

$ $ \left( \begin{array}{ccccc} 0 & X_1 & 0 & 0 & 0 \ \{\{-1\},\{0,0,0\}\} & 0 & 0 & 0 & 0 \ 0 & 0 & 0 & X_5 & \{\{0\},\{-z,0,x\}\} \ 0 & 0 & \{\{0\},\{0,-z,y\}\} & 0 & X_3 \ 0 & 0 & X_4 & \{\{0\},\{-y,x,0\}\} & 0 \ \end{array} \right)$ $

I expect:$ $ \left( \begin{array}{ccccc} 0 & X_1 & 0 & 0 & 0 \ -X_1 & 0 & 0 & 0 & 0 \ 0 & 0 & 0 & X_5 & -X_4 \ 0 & 0 & -X_5 & 0 & X_3 \ 0 & 0 & X_4 & -X_3 & 0 \ \end{array} \right)$ $

Is there an automated way of doing these replacements?