Answer
$$\frac{d^2y}{dx^2}=-10\sin(5x)-25x\cos(5x)-2\cos(2x)$$
Work Step by Step
$$\frac{dy}{dx}=(x\cos(5x)-\sin^2x)'=x'\cos(5x)+x(\cos(5x))'-2\sin x(\sin x)'=
\cos(5x)+x(-\sin(5x))(5x)'-2\sin x\cos x=
\cos(5x)-5x\sin(5x)-2\sin x\cos x$$
$$\frac{d^2y}{dx^2}=\frac{d}{dx}\Big(\frac{dy}{dx}\Big)=(\cos(5x)-5x\sin(5x)-2\sin x\cos x)'=
-\sin(5x)(5x)'-5((x)'\sin (5x)+x(\sin(5x)')-2((\sin x)'\cos x+\sin x(\cos x)')=
-5\sin(5x)-5\sin(5x)-5x\cos(5x)(5x)'-2\cos^2x+2\sin^2x=
-10\sin(5x)-25x\cos(5x)-2\cos(2x)$$
