Did I make the right decision why do I feel like crap?

I was engaged for 4 years and moved 1000 miles from home and he wasnt ready financially to come with me. I have been waiting for him to move but lately its really taxing on our relationship. He never calls me or wants to talk to me and thats the only line of communication we have right now. I went out one night and had the time of my life and met someone really amazing. He had a fiance too we became friends but there is a major connection there too. He broke off with his fiance and told me he loved me and wanted to be with me. I accidently said that I loved him too because I was feeling it at the time. I decided three days later that this is a mistake and I love my fiance more than anything so I broke this friendship whatever it is off. Now he says that i broke his heart and I feel terrible! Shouldnt I feel better that I did the right thing?