Let [tex] x [/tex] be the unknown number. So, three times that number means [tex] 3x [/tex], and the square of the number is [tex] x^2 [/tex]
We have to sum 528 and three times the number, so we have [tex] 528+3x [/tex]
Then, we have to subtract this number from [tex] x^2 [/tex], so we have
[tex] x^2-(3x+528) [/tex]
The result is 120, so the equation is
[tex] x^2 - 3x - 528 = 120 \iff x^2 - 3x - 648 = 0 [/tex]
This is a quadratic equation, i.e. an equation like [tex] ax^2+bx+c=0 [/tex]. These equation can be solved - assuming they have a solution - with the following formula
[tex] x_{1,2} = \dfrac{-b\pm\sqrt{b^2-4ac}}{2a} [/tex]
If you plug the values from your equation, you have
[tex]x_{1,2} = \dfrac{3\pm\sqrt{9-4\cdot(-648)}}{2} = \dfrac{3\pm\sqrt{9+2592}}{2} = \dfrac{3\pm\sqrt{2601}}{2} = \dfrac{3\pm51}{2}[/tex]
So, the two solutions would be
[tex] x = \dfrac{3+51}{2} = \dfrac{54}{2} = 27 [/tex]
[tex] x = \dfrac{3-51}{2} = \dfrac{-48}{2} = -24 [/tex]
But we know that x is positive, so we only accept the solution [tex] x = 27 [/tex]