Hitting timing in Rtts in All-Star Difficulty.

[IvanPerez_YT_PSN]

This is the first time I really play on All-Star difficulty and I understand it's harder. But why as I swing, I'm either late or early? And when I finally have a good swing, it's either a pop up or a ground ball. Most of the hits I get is because I'm late. But if I swing a tad early, it's gonna be too early. How's is that possible?