Answer
(a) $cos(A+B) -\frac{16}{65}$
(b) $sin(A+B) -\frac{63}{65}$
(c) quadrant III.
Work Step by Step
1. Given $cosA=\frac{3}{5}$ and $0\lt A\lt \frac{\pi}{2}$ (quadrant I), form a right triangle with sides $4,3,5$, we have $sinA=\frac{4}{5}$
2. Given $sinB=-\frac{5}{13}$ and $\pi\lt A\lt \frac{3\pi}{2}$ (quadrant III), form a right triangle with sides $5,12,13$, we have $cosB=-\frac{12}{13}$
(a). Use the Addition Formula, we have $cos(A+B)=cosA cosB - sinA sinB=(\frac{3}{5})(-\frac{12}{13})-(\frac{4}{5})(-\frac{5}{13})=-\frac{16}{65}$
(b). Use the Addition Formula, we have $sin(A+B)=sinA cosB +cosA sinB=(\frac{4}{5})(-\frac{12}{13})+(\frac{3}{5})(-\frac{5}{13})=-\frac{63}{65}$
(c). Since $sin(A+B)\lt0$ and $cos(A+B)\lt0$, we have angle $A+B$ in quadrant III.