Answer
$a.\displaystyle \quad\coth^{-1}(2) - \coth^{-1}(\frac{5}{4})$
$b.\displaystyle \quad-\frac{\ln 3}{2}$
Work Step by Step
$(a)$
Use table: "Integrals leading to inverse hyperbolic functions"
3. $\displaystyle \int\frac{du}{a^{2}-u^{2}}=\left\{\begin{array}{ll}
\frac{1}{a}\tanh^{-1}(\frac{u}{a})+C, & u^{2} \lt a^{2}\\
\frac{1}{a}\coth^{-1}(\frac{u}{a})+C, & u^{2}\gt a^{2}
\end{array}\right.$
on the interval $[\displaystyle \frac{5}{4},2]$, the graph of $x^{2}$ is above the graph of $x$, so we apply the $\coth^{-1}$ case.
$a=1, u(x)=x, du=dx,$
$\displaystyle \int_{5/4}^{2}\frac{dx}{1-x^{2}}= \left[ \coth^{-1}(\frac{x}{1}) \right]_{5/4}^{2}$
$= \displaystyle \coth^{-1}(2) - \coth^{-1}(\frac{5}{4})$
$ (b)$
Using the formulas in the box above these exercises,
$\displaystyle \coth^{-1}x=\frac{1}{2}\ln\frac{x+1}{x-1},\quad |x| \gt 1$
$\displaystyle \coth^{-1}(2) - \coth^{-1}(\frac{5}{4})=\frac{1}{2}\ln\frac{2+1}{2-1}-\frac{1}{2}\ln\frac{\frac{5}{4}+1}{\frac{5}{4}-1}$
$=\displaystyle \frac{1}{2}[\ln 3-\ln\frac{\frac{9}{4}}{\frac{1}{4}}]$
$=\displaystyle \frac{1}{2}[\ln 3-\ln 9]\qquad $... ( $9=3^{2}$ )
$=\displaystyle \frac{1}{2}[\ln 3-2\ln 3]$
$=-\displaystyle \frac{\ln 3}{2}$
