Area is proportional to (side length)^2.
or, put it in another way,
side length is proportional to sqrt(area).
so for the given case,
area = 144, sqrt(area)=12, side length = 10.
area = 64, sqrt(area)=8, side length = X
By proportion,
X/8 = 10/12
solve for X
X=(10/12)*8=20/3 = 6 2/3 = 6.67 (approx.)
So side length of second polygon is approx. 6.67 in., or 6 2/3 inches.