C. Jack will need 9 boxes; the cost is $448.20
The area of the floor is the product of its length and width:
... A = LW = (15 2/3 ft)·(10 1/3 ft) = (150 +5 + 6 2/3 + 2/9) ft² = 161 8/9 ft²
8 boxes will cover 8·(20 ft²) = 160 ft², so is not enough tile.
Jack will need 9 boxes.
The cost of 9 boxes of tile is ...
... (9 boxes)·($49.80/box) = $448.20
The cost is $448.20.
_____
Comment on this solution
The floor area calculation assumes tiles can be cut and fit without any issues. More likely, 11 rows of 16 tiles would be used, for a total of 176 tiles. Fortunately, 9 boxes contain 180 tiles, so 9 boxes is enough.