Expand [tex]\sin^5x[/tex] using the Pythagorean identity, [tex]\cos^2x+\sin^2x=1[/tex]:
[tex]\sin^5x=\sin x\sin^4x=\sin x(1-\cos^2x)^2=\sin x(1-2\cos^2x+\cos^4x)[/tex]
Then use the identity,
[tex]\cos^2x=\dfrac{1+\cos(2x)}2[/tex]
which makes
[tex]\cos^4x=\left(\dfrac{1+\cos(2x)}2\right)^2=\dfrac{1+2\cos(2x)+\cos^2(2x)}4[/tex]
and
[tex]\cos^2(2x)=\dfrac{1+\cos(4x)}2[/tex]
Putting everything together, we get
[tex]\sin^5x=\sin x\left(1-(1+\cos(2x))+\dfrac{1+2\cos(2x)+\frac{1+\cos(4x)}2}4\right)[/tex]
[tex]\sin^5x=\sin x\left(\dfrac38-\dfrac12\cos(2x)+\dfrac18\cos(4x)\right)[/tex]
[tex]\sin^5x=\dfrac38\sin x-\dfrac12\sin x\cos(2x)+\dfrac18\sin x\cos(4x)[/tex]
Now use the identities,
[tex]\sin(\alpha+\beta)=\sin\alpha\cos\beta+\sin\beta\cos\alpha[/tex]
[tex]\sin(\alpha-\beta)=\sin\alpha\cos\beta-\sin\beta\cos\alpha[/tex]
[tex]\implies\sin\alpha\cos\beta=\dfrac{\sin(\alpha+\beta)+\sin(\alpha-\beta)}2[/tex]
Then
[tex]\sin x\cos(2x)=\dfrac{\sin(3x)+\sin(-x)}2=\dfrac{\sin(3x)-\sin x}2[/tex]
[tex]\sin x\cos(4x)=\dfrac{\sin(5x)+\sin(-3x)}2=\dfrac{\sin(5x)-\sin(3x)}2[/tex]
So we end up with
[tex]\sin^5x=\dfrac38\sin x-\dfrac14(\sin(3x)-\sin x)+\dfrac1{16}(\sin(5x)-\sin(3x))[/tex]
[tex]\sin^5x=\dfrac58\sin x-\dfrac5{16}\sin(3x)+\dfrac1{16}\sin(5x)[/tex]
so that
[tex]a=\dfrac58,b=-\dfrac5{16},c=\dfrac1{16}[/tex]