(Note: I really appreciate getting this kind of feedback openly from the community, so thank you :-)
For example, the code build system could have variables for the name and maybe the logo and other trademark/brand-ish things, and the public codebase could be configured by default to call itself Timescale Community DB or Timescale Custom DB or some other name instead of TimescaleDB. Your private build would simply substitute the json file with those data values and maybe point to logos that aren't in the repo instead of generic ones that are, or something similar to that.
You'd also have the option to use any mixture of trademark law or copyright conditions to restrict the commercial version's name and branding assets.
All of the options I described above are used in reality by various projects out there. For example, the git repository for VS Code OSS has a product.json file with most of the customization points (not all) that MS changes in building their supported VS Code release, TeX and Red Hat apply naming restrictions, and Red Hat also has rules in their support contract.