Should I have this cyst removed?

I wouldn't risk anything like that. Just because it's getting smaller doesn't mean it couldn't enlarge again. Get it removed.