I just had this same issue. Infuriating when it does this since the game is intentionally made to encourage you to NOT refresh the game on a regular basis because you expect that it is doing autosaves to the Hentai Clicker main server.
This is enraging for me mainly because I was just about to rank 6 Vanessa (literally... I was only 20 affection away from being able to do that) and now I am back to needing 2.1K affection to get her to rank 6.