In my experience, they're always just used as some abstraction over amounts of time, which doesn't seem particularly useful but also isn't objectionable. What's weird to me is how specific the patterns are sometimes; I've worked on more than one team that insists on only using Fibonacci numbers for story points, but also that anything as large as 8 should be broken up into separate tasks, which effectively means that they used the range 1-5 but forbid usage of 4. On one of those teams, during one planning meeting someone mused that they wished there were something they could use to represent "less than 1", and I suggested we try putting 0.5 story points into JIRA, which to everyone's surprised actually worked, so 0.5 became the only other allowed value.
