Athletic shoes (especially white ones.. or white clothing in general), shorts, shirts with logos/name brands, and as you mentioned, baseball hats, all make one stick out as an American. There're lots of little mannerisms that can distinguish you as well.
I found that Europeans like to claim that they can guess your nationality no matter how hard you try to disguise yourself. However, this claim didn't hold up too well when they tried to guess where I was from (especially when alone).. "German? French? Spanish? Italian? English...?"
It is fun to attempt to blend in with the culture, but I don't think it's that big of a problem for it to be necessary to hide your nationality. I can see the security issues... but yeah... if you're really worried, I was told that if someone asks where you're from, reply: "I'm from Canada, eh...."