Answer
(a) $|\langle A, B \rangle|=7\leq\| A \|\| B \|=\sqrt{6}\sqrt{10}=7.75$
(b) $\| A+B \| =\sqrt{30}=5.47\leq\| A \|+\| B \|=\sqrt{6}+\sqrt{10}=5.61$.
Work Step by Step
Let $A=\left[\begin{array}{rr}{0} & {1} \\ {2} & {-1}\end{array}\right], \quad B=\left[\begin{array}{rr}{1} & {1} \\ {2} & {-2}\end{array}\right]$,
$\langle A, B \rangle =a_{11} b_{11}+a_{12} b_{12}+a_{21} b_{21}+a_{22} b_{22}$, then we have
$\langle A, B \rangle =0+1+4+2=7$
$\| A\| =\sqrt{\langle A, A\rangle}=\sqrt{a_{11}^{2}+a_{12}^{2}+a_{21}^{2}+a_{22}^{2}}=\sqrt{0+1+4+1}=\sqrt{6}$
$\| B \| =\sqrt{\langle B, B\rangle}=\sqrt{b_{11}^{2}+b_{12}^{2}+b_{21}^{2}+b_{22}^{2}}=\sqrt{1+1+4+4}=\sqrt{10}$
$\| A+B \|=\| \left[\begin{array}{rr}{1} & {2} \\ {4} & {-3}\end{array}\right] \|=\sqrt{1+4+16+9}=\sqrt{30}.$
Now, we get
(a) $|\langle A, B \rangle|=7\leq\| A \|\| B \|=\sqrt{6}\sqrt{10}=7.75$
(b) $\| A+B \| =\sqrt{30}=5.47\leq\| A \|+\| B \|=\sqrt{6}+\sqrt{10}=5.61$.
