I'm not an expert in logic, but I can help a bit here.
For reference, here is the theorem in the blog post:
There is a Turing machine program e, such that
(1) PA proves that e accepts only finitely many inputs.
(2) For any particular finite subset A of N, there is a model M of PA such that inside M, the program e accepts all and only the elements of A.
(3) Indeed, for any subset A of N, including infinite sets, there is a model M of PA such that inside M, program e accepts n if and only if n is in A.
What you call the "loose" definition of finite is indeed what's being used here. The way (1) is stated, it implies the definition of "finite" must be within PA itself (otherwise PA couldn't prove it). Your "loose" definition is the only way to define "finite" within PA itself.
You are correct that in the presence of non-standard numbers, a set that PA proves is "finite" could actually be infinite in the model. Indeed, that explains how (3) is even possible!
As for whether A can contain non-standard numbers, the answer is no. Note that A is a variable defined in the meta-theory, since it is in the statement of the theorem (2) and (3). Therefore the "standard" versus "non-standard" label doesn't apply to A. The trick here is that there's an implicit "conversion" that needs to happen when going from A in the metatheory (where it is a subset of N) to A as a set of numbers in M. When that conversion happens, all the elements will be standard. This is a more precise way of phrasing (2) that might help, where I've used quotes to indicate exactly what's a statement in PA (as opposed to meta-theory):
(2') For any particular finite set A in N, there is a model M of PA such that the statement "e accepts n" is true in M iff n is in A.
For reference, here is the theorem in the blog post:
There is a Turing machine program e, such that (1) PA proves that e accepts only finitely many inputs. (2) For any particular finite subset A of N, there is a model M of PA such that inside M, the program e accepts all and only the elements of A. (3) Indeed, for any subset A of N, including infinite sets, there is a model M of PA such that inside M, program e accepts n if and only if n is in A.
What you call the "loose" definition of finite is indeed what's being used here. The way (1) is stated, it implies the definition of "finite" must be within PA itself (otherwise PA couldn't prove it). Your "loose" definition is the only way to define "finite" within PA itself.
You are correct that in the presence of non-standard numbers, a set that PA proves is "finite" could actually be infinite in the model. Indeed, that explains how (3) is even possible!
As for whether A can contain non-standard numbers, the answer is no. Note that A is a variable defined in the meta-theory, since it is in the statement of the theorem (2) and (3). Therefore the "standard" versus "non-standard" label doesn't apply to A. The trick here is that there's an implicit "conversion" that needs to happen when going from A in the metatheory (where it is a subset of N) to A as a set of numbers in M. When that conversion happens, all the elements will be standard. This is a more precise way of phrasing (2) that might help, where I've used quotes to indicate exactly what's a statement in PA (as opposed to meta-theory):
(2') For any particular finite set A in N, there is a model M of PA such that the statement "e accepts n" is true in M iff n is in A.