Well, I've ended up going through several digital cameras (and everyone in my house has gone through a few) so I have a good idea of them I would say.
Here are a few things to look for that you might not think about:
A high optical zoom (not digital; this reduces quality as you zoom)
Good manual settings (even if you're not into photograpy, these are very useful have when you can't get a shot right. The HP cameras and cheaper Kodak cameras I have used do not have many configurable settings so they can be harder to take good pictures with in some situations)
Now, from my experience, the best cameras that have good features while still being fairly easy to use are Canons and Fujifilms. If you are willing to pay a little more, look at this camera as welll as those in the Fujifilm S series. The nicer Sony cameras are good from what I have used them, but that is not very much.
Amazon.com: Canon Powershot SX110IS 9MP Digital Camera with 10x Optical Image Stabilized Zoom (Black): Camera & Photo
If you want a simple point and shoot camera that can get you good pictures, go for a Sony Cybershot or Nikon Coolpix. They have a good balance of features and price. The zoom is not as good, usually. Avoid Kodaks. They're not awful, but not as good.
Another thing is that it's cheaper to get SD cards usually than the other types, so if you plan to buy a lot of memory cards so you can keep pictures on them and such, you may want to get an SD camera.