Answer
a) $\dfrac{-4x+17}{2x-1}$; $D=\left(-\infty,\dfrac{1}{2}\right)\cup\left(-\dfrac{1}{2},3\right)\cup(3,\infty)$
b) $\dfrac{3x-3}{-2x-8}$; $D=\left(-\infty,-4\right)\cup\left(-4,-1\right)\cup(-1,\infty)$
c) $\dfrac{-4x-10}{2x-4}$; $D=\left(-\infty,-1)\cup(-1,2)\cup(2,\infty\right)$
d) $\dfrac{3x-4}{-2x+11}$; $D=\left(-\infty,3\right)\cup\left(3,\dfrac{11}{2}\right)\cup\left(\dfrac{11}{2},\infty\right)$
Work Step by Step
We are given the functions:
$f(x)=\dfrac{x-5}{x+1}$
$g(x)=\dfrac{x+2}{x-3}$
Let's note:
$D_f$=the domain of $f$
$D_g$=the domain of $g$
We have:
$D_f=(-\infty,-1)\cup(-1,\infty)$
$D_g=(-\infty,3)\cup(3,\infty)$
a) Determine $f\circ g$ and its domain $D_{f\circ g}$:
$(f\circ g)(x)=f(g(x))=f\left(\dfrac{x+2}{x-3}\right)=\dfrac{\dfrac{x+2}{x-3}-5}{\dfrac{x+2}{x-3}+1}$
$=\dfrac{\dfrac{x+2-5x+15}{x-3}}{\dfrac{x+2+x-3}{x-3}}=\dfrac{\dfrac{-4x+17}{x-3}}{\dfrac{2x-1}{x-3}}=\dfrac{-4x+17}{2x-1}$
As $D_g$ does not contain 3, we exclude 3 from the domain of $f\circ g$. Also because -1 is not in the domain of $f$, we must exclude the value of $x$ for which $g(x)=-1$:
$\dfrac{x+2}{x-3}=-1$
$x+2=-x+3$
$2x=1$
$x=\dfrac{1}{2}$
$D_{f\circ g}=\left(-\infty,\dfrac{1}{2}\right)\cup\left(-\dfrac{1}{2},3\right)\cup(3,\infty)$
b) Determine $g\circ f$ and its domain $D_{g\circ f}$:
$(g\circ f)(x)=g(f(x))=g\left(\dfrac{x-5}{x+1}\right)=\dfrac{\dfrac{x-5}{x+1}+2}{\dfrac{x-5}{x+1}-3}$
$=\dfrac{\dfrac{x-5+2x+2}{x+1}}{\dfrac{x-5-3x-3}{x+1}}=\dfrac{\dfrac{3x-3}{x+1}}{\dfrac{-2x-8}{x+1}}$
$=\dfrac{3x-3}{-2x-8}$
As $D_f$ does not contain -1, we exclude -1 from the domain of $g\circ f$. Also because 3 is not in the domain of $g$, we must exclude the value of $x$ for which $f(x)=3$:
$\dfrac{x-5}{x+1}=3$
$x-5=3x+3$
$2x=-8$
$x=-4$
$D_{g\circ f}=\left(-\infty,-4\right)\cup\left(-4,-1\right)\cup(-1,\infty)$
c) Determine $f\circ f$ and its domain $D_{f\circ f}$:
$(f\circ f)(x)=f(f(x))=f\left(\dfrac{x-5}{x+1}\right)=\dfrac{\dfrac{x-5}{x+1}-5}{\dfrac{x-5}{x+1}+1}$
$=\dfrac{\dfrac{x-5-5x-5}{x+1}}{\dfrac{x-5+x+1}{x+1}}=\dfrac{-4x-10}{2x-4}$
As $D_f$ does not contain -1, we exclude -1 from the domain of $f\circ f$. Also because -1 is not in the domain of $f$, we must exclude the value of $x$ for which $f(x)=-1$:
$\dfrac{x-5}{x+1}=-1$
$x-5=-x-1$
$2x=4$
$x=2$
$D_{f\circ f}=\left(-\infty,-1\right)\cup\left(-1,2\right)\cup(2,\infty)$
d) Determine $g\circ g$ and its domain $D_{g\circ g}$:
$(g\circ g)(x)=g(g(x))=g\left(\dfrac{x+2}{x-3}\right)=\dfrac{\dfrac{x+2}{x-3}+2}{\dfrac{x+2}{x-3}-3}$
$=\dfrac{\dfrac{x+2+2x-6}{x-3}}{\dfrac{x+2-3x+9}{x-3}}=\dfrac{3x-4}{-2x+11}$
As $D_g$ does not contain 3, we exclude 3 from the domain of $g\circ g$. Also because 3 is not in the domain of $g$, we must exclude the value of $x$ for which $g(x)=3$:
$\dfrac{x+2}{x-3}=3$
$x+2=3x-9$
$2x=11$
$x=\dfrac{11}{2}$
$D_{g\circ g}=\left(-\infty,3\right)\cup\left(3,\dfrac{11}{2}\right)\cup\left(\dfrac{11}{2},\infty\right)$