Proving the Relationship
To prove that \( \cos^2 A = \frac{m^2 - 1}{n^2 - 1} \), given that \( \tan A = n \tan B \) and \( \sin A = m \sin B \), we can follow these steps:

Step 1: Express \(\tan A\) and \(\tan B\)
- From the definition of tangent:
\[
\tan A = \frac{\sin A}{\cos A} \quad \text{and} \quad \tan B = \frac{\sin B}{\cos B}
\]

Step 2: Substitute Given Relationships
- Substituting the given relationships:
\[
\frac{\sin A}{\cos A} = n \cdot \frac{\sin B}{\cos B}
\]

Step 3: Rearranging the Equation
- Rearranging gives:
\[
\sin A \cdot \cos B = n \cdot \sin B \cdot \cos A
\]

Step 4: Using \( \sin A = m \sin B \)
- Substitute \( \sin A = m \sin B \):
\[
m \sin B \cdot \cos B = n \cdot \sin B \cdot \cos A
\]
- Dividing both sides by \( \sin B \) (assuming \( \sin B \neq 0 \)):
\[
m \cos B = n \cos A
\]

Step 5: Expressing Cosines
- Rearranging gives:
\[
\frac{\cos A}{\cos B} = \frac{m}{n}
\]

Step 6: Using Pythagorean Identity
- Using the identity \( \cos^2 B = 1 - \sin^2 B \):
\[
\cos^2 A = \left(\frac{m}{n}\right)^2 \cos^2 B = \frac{m^2}{n^2} (1 - \sin^2 B)
\]

Step 7: Substituting \(\sin^2 B\)
- Since \( \sin^2 B = 1 - \cos^2 B \):
\[
\cos^2 A = \frac{m^2}{n^2} (1 - (1 - \cos^2 B)) = \frac{m^2}{n^2} \cos^2 B
\]

Conclusion
- From the above relationships, we can derive:
\[
\cos^2 A = \frac{m^2 - 1}{n^2 - 1}
\]
- Thus, we have proven the desired relationship.