Answer
See below
Work Step by Step
Given: $A=\begin{bmatrix}
1 & 2 \\ 3 &4
\end{bmatrix}\\
\rightarrow \det(A)=1.4-2.3=-2$
$B=\begin{bmatrix}
1 & 1\\ 1 &1
\end{bmatrix}\\
\rightarrow \det(B)=5.1-4.1=1 \ne 0$
By Theorem 3.3.16, we have $B^{-1}=\frac{1}{\det(B)}adj(B)=\frac{1}{1}adj(B)=adj(B)$
Obtain:
$C_{11}=(-1)^{1+1}M_{11}=1\\
C_{12}=(-1)^{1+2}M_{12}=-1\\
C_{21}=(-1)^{2+1}M_{21}=-4\\
C_{22}=(-1)^{2+2}M_{22}=5$
hence, $M_C=\begin{bmatrix}
1 & -1\\ -4 & 5
\end{bmatrix}\\
\rightarrow adj(B)=M^T_C=\begin{bmatrix}
1 & -4\\-1 & 5
\end{bmatrix}$
Find $B^{-1}=\begin{bmatrix}
1 & -4\\-1 & 5
\end{bmatrix}$
Since $\det(B^T)=\det(B)\ne 0$
then $B^T$ is invertible
and $A^{-1}B^T$ is also invertible.
Hence, $(A^{-1}B^T)^{-1}\\=(B^T)^{-1}(A^{-1})^{-1}\\=(B^{-1})^T A\\=\begin{bmatrix}
1 &-1 \\-4 & 5
\end{bmatrix}\begin{bmatrix}
1 & 2 \\3 &4
\end{bmatrix}=\begin{bmatrix}
-2 & -2\\11 & 12
\end{bmatrix}$
