I'm short on time this morning but I wanted to comment quickly.
You could cut out a leprechaun or shamrock to take the place of the ghost. I'm sure that you can find a poem for a mischievous leprechaun.
Fill a small bag with chocolate gold coins, lucky four leaf clovers...etc.
When my son was in second grade, they got a visit from "Lucky the Leprechaun" (me!). I turned their chairs upside down and wrapped green crepe paper all around the legs of the chairs, creating a maze. Then I printed out puzzles for the kids to do and left gold coins for them. I wrote up a big letter that was similiar to something that you would do for a "booing".
Fun! If you do something, let us know what you come up with!
