There are a lot of individual circumstances, but generally:
Prison is where people who are convicted of felonies (crimes worthy of >1 year in custodial sentence) are sent. Jails are where people awaiting trial, or convicted of misdemeanors (crimes worthy of <1 year in custodial sentence) are sent.
Jails are run by counties or cities, and are for persons awaiting trial (i.e. not been convicted) or persons found guilty and serving short sentences.
Prisons are run by a state or by the Federal government, and are for holding persons who have been convicted of a crime and sentanced to a longer term.