If you just arrive in Default, It should be on your left across the water on a little island. The portal is in a cave with a sign labeled next to it "Portal Island."
If you can't find it, you can look on your tiny map in the top right corner of your window and look at the blue dots, those represent portals. Once you find portal island, run towards it!