Just a word on New Jersey's public transit -- there are two separate systems to use. The first is the PATH, which is a subway system that connects Manhattan to Hoboken, Jersey City, Harrison and Newark. Hoboken is really small (geographically speaking) and has one stop that puts you in walking distance to everything. Jersey City has three stops, so where you want to go depends on that. There's also a small trolley-type of system (we call it light rail) that connects those towns.
Then there's New Jersey Transit. NJT stops at Penn Station in Manhattan and then goes to most of the rest of North Jersey. You would take this to get to Princeton. You can also take it to the Jersey Shore -- it's a bit of a hike from Penn Station but that might be the most quintessential Jersey experience there is. The Jersey Coast line can get you to Asbury Park (spiritual home of Bruce Springsteen, the Stone Pony and the pinball museum Tom touts -- albeit, it's a pretty dicey place, but I haven't been there in a few years). It can also get you to the more typical shore towns like Belmar, Bradley Beach and Point Pleasant. Out of those three, Point Pleasant's the one I would suggest -- they have a Boardwalk with a bunch of rides and you can get fried candy bars and stuff like that.
Also, I think the best cuisine the state of New Jersey has to offer is the hot dog. You can't get hot dogs like you can anywhere else in Jersey. I don't know Jersey City or Hoboken (JC didn't have anything going for it until the past 10 years; Hoboken I avoid like the plague because of its frat-guyish nature but Maxwell's is AWESOME) all that well so maybe someone else can have some suggestions. Jersey City also has a Little India neighborhood which has some pretty great food, but being a Londoner, that might not be too big a deal.
Princeton is worth a trip. That's one of my favorite small towns in the US. It's a nice little college town (Princeton University) with some really nice shops and places to get a bite to eat and the Princeton Record Exchange is really great. They also have a really good college radio station down there.
It might be a bit hard to do Jersey City/Hoboken and Princeton in one day. You'd have to take the PATH from Jersey City to Newark Penn Station (confusing) and then onto New Jersey Transit to Princeton. Princeton's about an hour on the train (depending which one you hop on -- it's on what's called the Northeast Corridor and there's certain express trains that get you there faster). I think the Princeton train station's a bit outside of the town so you'd have to take a shuttle bus from the train station to the heart of the town, but someone might know better than me.
North Jersey's best part is that there's just a lot of nice small towns to bum around in. If you're pressed for time, you can take New Jersey Transit to Montclair (kind of a college town that has decent restaurants and a few good bars, including a spot called Tierney's which is one of my five favorite bars in the world), Morristown (similar to Montclair but has a lot of history stuff), South Orange/Maplewood (two suburban towns right next to each other, both have pretty nice little downtown areas and have a few good Irish bars. I really like Cryan's in South Orange a lot.) and Ridgewood (pretty haughty but has a lot of decent restaurants -- Baumgarten's is really well known.) You could also go to New Brunswick which is home to Rutgers (New Jersey's state university) and eat at the Greasetrucks, which are these insane sandwiches that can kill you.
And don't EVER worry about asking a New Jerseyan about places to go. We 100% get the short end of the stick. We're the butt of jokes from the rest of the country constantly. And people who live in New York act like we don't even exist.
I live in Philadelphia these days but North Jersey is where I grew up and will always be my home.
