I spoke with a japanese friend and, while his english isn't great, he was at least able to give me the romaji for it and confirm my translation. I'm glad he was online, because these characters are out of my league.
In text, it's 統一武道会 which is spelled "Touitsu Budoukai". It means "Unity Martial Arts Association" or "Unified Martial Arts Association". I hope this helps a little.
How did you happen to come into possession of this piece of clothing? Certainly there must be an interesting story around here somewhere.