Answer
$\text{a) }
(f\circ g)(x)=10x^2-35x+28
\\\text{b) }
(g\circ f)(x)=25x^2-55x+24$
Work Step by Step
$\bf{\text{Solution Outline:}}$
With
\begin{array}{l}\require{cancel}
f(x)=
5x-2
\\g(x)=
2x^2-7x+6
,\end{array}
use the definition of function composition to find $
(f\circ g)(x)
$ and $(g\circ f)(x).$
$\bf{\text{Solution Details:}}$
Using $(f\circ g)(x)=f(g(x)),$ then replace $x$ with $g(x)$ in $f$. Hence,
\begin{array}{l}\require{cancel}
(f\circ g)(x)=f(g(x))
\\\\
(f\circ g)(x)=f(2x^2-7x+6)
\\\\
(f\circ g)(x)=5(2x^2-7x+6)-2
\\\\
(f\circ g)(x)=10x^2-35x+30-2
\\\\
(f\circ g)(x)=10x^2-35x+28
.\end{array}
Using $(g\circ f)(x) =g(f(x)),$ then replace $x$ with $f(x)$ in $g.$ Hence,
\begin{array}{l}\require{cancel}
(g\circ f)(x)=g(f(x))
\\\\
(g\circ f)(x)=g(5x-2)
\\\\
(g\circ f)(x)=(5x-2)^2-7(5x-2)+6
\\\\
(g\circ f)(x)=(25x^2-20x+4)+(-35x+14)+6
\\\\
(g\circ f)(x)=25x^2-55x+24
.\end{array}
Hence,
\begin{array}{l}\require{cancel}
\text{a) }
(f\circ g)(x)=10x^2-35x+28
\\\text{b) }
(g\circ f)(x)=25x^2-55x+24
.\end{array}
