From Mozilla: On Windows, it is not possible to remove or rename files and directories which are being used. In order to handle this, we need to add a little complexity.
The Firefox executable will launch a helper application copied to somewhere outside of the installation directory, most likely the system's temporary directory.
The helper application renames the existing installation directory to a new directory with ".tmp" appended to its path name. If the rename operation fails in the middle of the operation because of a file being open by another application, it reverts the change.
Besides some access modes you can rename files in use. I did even bothered and tried on firefox.exe I'm typing this - it works fine.