The Talmud in ראש השנה relates that when ישמעאל was faint when he was sent away with Hagar, the angels in Heaven made an argument with G-d. They pleaded with Him not to answer the prayers of the young lad and he should not find a well of water for sustenance.
After all, they argued, Yishmael’s descendants would harm the Jews in many ways, including starvation. The Gemara then quotes a passage in the name of רב יצחק: We are only permitted to judge a person for his actions at that moment.
This was learned from the Pasuk in the Torah: כי שמע אלוקים אל קול הנער באשר הוא שם, that Hashem heard the voice of the youth in his present state.
In general, this is the ruling. That is, we don’t punish sons for the sins of the fathers, and not the fathers for the sins of the sons. We judge as things are now right in front of us.