I do that kind of thing, too. Here are some of my favorite treats:
Beauty treats:
press-on nails (@$3)
bubble bath
foot lotion & scrub (lotion & a loofah $1/ea - dollar tree, use many times)
Like you said, the milkshake, or a slushy, or even a small candy bar or something when you've been out running errands you didn't want to run! Strawberries or melon. A homemade milkshake with fresh or frozen fruit, or jelly.
Herbal tea - from home grown herbs so it's cheaper and fresh!
A nice smelling candle burning while I do a chore improves my attitude about the chore. ;-)
Allowing myself some time for a book or movie or to sit outside.
For the kids, I like to spontaneously break out the bubble stuff or I have a book where you make stuff like clay, slime, etc. from household items and I'll pick something from there that I think I can "tolerate" the mess.
I used to like to pick up a "just because" greeting card to give to my DH - a treat for both of us because it was so fun to pick out.
I'd love to get more ideas from what other people do for cheap treats, pampering, etc.