Ahh the joy that awaits you.  It's refreshing to see someone that hasn't
played before.

Exiting the game, make sure you save, then just open the tray, remove the
disc, shut the tray (you probably could have guessed that step), and shut
off the ps2.  There is no option to really "exit" from within the game,
you basically just shut it off (after saving, a point that can't be
stressed enough).

From GTA III on, all the GTA titles (it coulda been this way before too,
but I haven't played any of them) keep parts of the game world locked off
until you complete certain sections.  Each contact will give you missions
to do and when you have completed the right chain of missions it will
automatically open up the next section of the world and give you new
things to do.

I highly recommend surfing over to gamefaqs.com and checking out some of
the guides that people have written.  These worlds are immense compared
to other games and there are literally hundreds of things to occupy your
time. Indeed, after playing all the story missions in most GTA titles you
are only 50-70% finished with the game.  My opinion says don't read ahead
on the missions too much, it kind of spoils some of the fun when you know
exactly what is going to happen, but the guides to the stunt jumps and
the hidden packages are handy.  The mission walkthroughs are good for
when you are stuck on a mission, it can give you a tactic to try that you
might not have thought of.

Good Luck.  Have fun.  Come back and let us know how it goes.