For the Cardinals I go by the rule of "Are they known primarily as a Cardinal"? Luckily the Cards have enough players that I can pull that off. It might be tougher for other teams.
For example I would not put Larry Walker on my Cardinals squad like others do. He played 144 games for them in his final two seasons. He is primarily known as a Rockie to me.
But to each their own. There is no hard a fast rule other than the player should have played for the club at some point.