I think this is still rooted from the experience of China under the Japanese Colonization. Violence against women and men were pretty much high at that time. Culturally, both countries have a negative perception of each nation, causing a drift between them.
Another would be because of the difference in terms of economic development of both countries. Japan focuses more on the capitalist side while China is more inclined to communism than capitalism.
