running on a caught fly ball

Question
I play in a men's slow-pitch league. Recently, while on third base, the batter flied out to center field. I waited on the base for the catch to be made and then ran home. After I scored, the pitcher and catcher of the other team told their cut-off man to throw to third base because I left early.
The plate umpire gave no indication that I had left early, but agreed with the defense and called me out.
Is this a call that the umpire should make, or should he only make a ruling when appealed to by the defense?

Answer
Hi Dino,

leaving a base early on a legally caught fly ball is an appeal play by the defense.  The umpire should never give even a hint to the defense that the runner left early.  if they don't appeal you're not out.

The defense did appeal, they had a "live ball appeal" of a base left too early by throwing to 3rd and saying you left early.

Mark