Branches and tags are just "Stick-it-notes" you attach to the DAG. They can be removed around at whim, but they only serve to mark a point in the DAG for your convenience. The GC process will kill parts of the DAG with no such branch/tag, but it only happens after some days by default.
Were git able to rewrite history, it couldn't use SHA1 checksums as names for the content. Note that a rebase alters the SHA1 stack you have, so it is regarded as something different. But you can't do that easily with published stuff as it would render further work impossible.
Fast forward is the recognition that a stick-it-note can be moved linearly ahead.
You won't be rewriting history, you will be writing new history and then forget about the old. But it is still there should accidents happen.