Stamford Raffles "took" Singapore from the Malay sultanate in the 1820s. In 1963, we finally got it back, but let them go barely two years later due to ingrained cultural differences that manifested during the meantime.
With the rise of so many capable cities on the mainland, perhaps pursuit of unification is just not worth the trouble anymore.
This is very different from HK and China.
Maybe to you? It's clear from China's actions that they think otherwise and they're willing to do whatever it takes to ensure control and unification.