No matter what the chances are there's always going to be outliers that are super lucky or super unlucky. No one has a problem being super lucky so well ignore that end of the spectrum. The only way to give a real "safety net" for the unluckiest is to guarantee a drop. The two best ways I can think of to fix the affection system is A: after x number of plays against a girl, award shards/100 shards, obviously awarding 100 would make the x a much larger number. or B: after every fight add n% chance to drop shards with n increasing every time you don't receive shards, and resetting when you do.
I'm making up some numbers for examples. A would guarantee a shard drop every 30 fights, or give 100 shards after say 1000 fights. Maybe even add both. B would give a 1% base chance to drop shards, with .5% increase every time you don't receive shards, resetting when you do. The drop chance would look like 1%, 1.5%, 2%, 2.5%... until it resets back to 1% when you win a shard.
A would remove most of the randomness while B would keep things random, just with an increased chance. Personally I like option B more, though it may be harder to implement from a technical standpoint.