When is the right time to have sex?

had a date with a guy who obviously likes me. We went to a nice restaurant and after that I got an offer to have a drink in his place, which eventually I refused since it was a first date and I prefer getting to know the person a bit before getting into sex.

I would like to know what you find an optimal time to start having sex when dating, in case you want to have a serious (long term) relationship.

Danielle