Yeah, I have an extended Friend's list on my computer.
Having over 100 friends and making new ones, I have to copy and paste some gamer tags to a word file.
Otherwise my friends list would be full and thus unable to make new friends.
When I have Xbox One with a max of 1000 friends, I shouldn't have this problem for a while.
I'd start keeping a record of your friends.
I'm going to start doing that as well.
You want to know all the friends you've had are still there.
Instead of relying on your memory, your friends list which is unreliable if they can leave without notice, or a list of members in a group that doesn't include those you met by chance.