d. The book is theirs but it's very old.
The word their never uses an apostrophe. Therefore options A and C are wrong. "Its" shows ownership. In the sentence, "it" is not owning anything. "It's" (with an apostrophe) is a contraction meaning it is. D is correct because you need the contraction for it is very old. Just a sidenote - technically, none of these sentences are grammatically correct because there should be a comma after theirs. "The book is theirs" is a complete sentence. "it's very old" is a complete sentence. When combining two complete sentences with a conjunction, there needs to be a comma before the conjunction: The book is theirs, but it's very old.