
Python Extension Proposal 498: Literal String Formatting - ptest1
https://www.python.org/dev/peps/pep-0498/
======
asgard1024
I am against it, because it allows arbitrary Python expressions inside format
strings. It's too complicated and lets user have two different ways of doing
things (not Pythonic) - one to calculate expression inside string and the
other to calculate it outside (which should IMHO be preferred). This should
maybe go to the standard library, but please not into the language.

I think a better approach would be to just add special formatter operators (if
they aren't already there) that would just call str() or repr() or ascii() to
whatever is presented to them (and maybe take some optional arguments such as
length or padding).

~~~
task_queue
> I am against it, because it allows arbitrary Python expressions inside
> format strings.

I was for this PEP until your post made this reality apparent. I'll take
security over convenience.

~~~
imakesnowflakes
I will go further and say that I will even take readability over convenience.

This is a step in the reverse direction. Please don't do this. I am not sure
why this is even considered. We already have ways to do this _clearly_. Let us
not add another way to do this in a less readable way that is a lot more
easier to write. That is a deadly combination.

Features in python are geared towards more readable code (I know about the
stuff you can do with things like comprehensions, but hey I think their power
justifies them enough). This will lead to people using this format due to
initial convenience, but ends up regretting doing so.

Please remember that code is read more often than it is written. So a
requiring a little verbosity if that can enhance readability even a little
bit, is good. I hope these kinds of good things about python does not get
removed.

I am coming from 9 years of experience with PHP. And I will say that this is
not worth it. And this is actually one of the features I have come to like in
Python now.

And that is not considering the implications of having expression evaluation
inside strings...

~~~
mkesper
I actually think the proposal would be more readable because you don't have to
think about what it means. Compare:

"{} of very long text here...".format(value)

f"{value} of very long text here..."

~~~
imakesnowflakes
I thinks this is where our difference in comes from.

When we are reading code, we are not usually reading the strings contained
within.

We are reading variables names, we are looking at where they are used, where
they are assigned, stuff like that.

I very rarely look inside the strings when we are actually reading code. When
reading code, strings are just black boxes where nothing can happen. So we can
completely ignore them.

With this pep, that will change. And readability takes a big hit. Strings are
no longer black boxes.

And actually you can get the " {value} if very long text" format right now
using the format function, but it right now requires an explicit
list/dictionary of variables.

This explicit list of variables really helps in the context of _reading_ code.
And forcing users to write down that explicit list of variables to use, is
good.

~~~
Camillo
How do you know where the string ends? If your editor has syntax coloring and
you look for the end of the string-color block, then the expressions inside
the string will also have a different color and will pop right out. If you
don't use syntax coloring you have to scan for " or ', and you'll learn to
scan for { too.

~~~
imakesnowflakes
>How do you know where the string ends?

I usually just go to the end of the line. or if that is not possible, I think
I usually look at the next closest thing that obviously is not a string and
scan backwards.

I mean, usually I didn't have to do anything consciously (or do a scan) to
spot the end of a string.

------
Walkman

        There should be one-- and preferably only one --obvious way to do it.
    

This would be the 4th way of formatting strings in Python.

~~~
chrismorgan
It would also render the previous ways obsolete. New-style formatting never
took over completely from old-style because it wasn’t sufficiently
compelling—`"%s %s" % (a, b)` versus `"{} {}".format(a, b)` doesn’t have a
clear winner. But `f"{a} {b}"`? Clearly superior. With the exception of
backwards compatibility matters (which _will_ be a nuisance for far too long),
there would really be no reason to keep using the old ways in most places.
i18n/l10n would really be the only mainstream reason for using anything other
than f-strings.

~~~
wylee
Your example isn't very compelling, but for longer strings with more complex
formatting, I'd say .format() is pretty compelling. I find that in general

    
    
        '{x} blah blah blah {y}'.format(x=x, y=y)
    

is more readable than

    
    
        '%s blah blah blah %s' % (x, y)
    

even if the former is a bit longer.

On top of that, there's a whole bunch of stuff you can do with .format() that
just isn't possible with %.

~~~
cnvogel
Even .format() is quite redundant for the "standard" usage, as we had string
interpolation using '%' with dicts for a long time.

    
    
        '%s blah blah blah %s' % (x, y)
        '{x} blah blah blah {y}'.format(x=x, y=y)
        '%(x)s blah blah blah %(y)s'%{'y':y, 'x':x}

------
declnz
I would be very happy to see _some version_ of this accepted.

When it comes to native String interpolation Groovy has it, Scala has it, ES6
has it apparently; to a more limited extent Bash, PHP, Perl have it too of
course.

I can't help feeling that other devs are now coming to Python expecting this
kind of feature, and are disappointed to find three (harder, often less
readable) ways to do it instead. Got to keep up with the Joneses, etc...

~~~
svisser
It doesn't mean this is a good feature to have - it allows objects within
scope to now be directly included in a string, which isn't a secure thing to
do.

~~~
digisign
Only literals, not passed strings, not less secure.

------
schmichael
Nonononono. Python does not need more string literal specifiers. For a
language that avoid symbols and sigils like the plague, it already has an
absurd amount of string literal syntax.

Why not make a strfmt library on pypi that provides a single _fmt(s, args,
kwargs)_ function and let people call that? Why the obsession with more
builtins?

------
mangeletti
The first thing I thought of when I looked at the PEP was, "this is like a
string version of register_globals=on".

A string literal whose value automatically changes with the code surrounding
it sounds like a really bad idea.

I also noticed that the PEP uses str.format method as a strawman, ignoring the
fact that % string interpolation is very popular and does not need replacing,
which is at the core of this problem in the first place; Someone keeps trying
to replace something that does not need replacing.

Furthermore, I can't help but think that this would eventually become a
complete literal string DSL (if not one already) inside of Python.

I hope this PEP does not get accepted.

~~~
TazeTSchnitzel
> The first thing I thought of when I looked at the PEP was, "this is like a
> string version of register_globals=on", which is an unsettling thought to
> have about Python, my favorite language.

> The idea of having a string that is automatically dynamic and whose value is
> hardly predictable upon first glance, wholly dependent on the stability of
> the code surrounding it, sounds like an absolutely horrendous idea.

What?! It's not a dynamic string. It's a string concatenation expression with
syntactic sugar.

This isn't PHP's register_globals. It's PHP's "{$n + 1}".

What this does, you can already do. "Foo " \+ bar + " baz" already exists.
This is merely nicer syntax.

~~~
imakesnowflakes
The difference is,

> "Foo " \+ bar + " baz"

is not a string literal, neither is one using format or % function. All of
those things return dead strings. This pep is about creating a kind of 'live'
_string literal_ , which python does not have right now (or need IMHO). So
this is not merely a nicer syntax.

~~~
svieira
It is not live (unless I am mis-understanding what you mean by live) - it is
evaluated _once_. That is to say:

    
    
        def magic(a, b):
            return f"{a} and {b}"
    
        s = magic(4, 8)
        print(s)
        a, b = 3, 7
        print(s)
    

prints `4 and 8` in both cases.

~~~
imakesnowflakes
It is live in the sense that you can take that _string literal_ and put it in
a different context and it will result in a different evaluation, possibly
yielding different value for the string and possibly different side effects
(after calls to evaluate expressions).

------
JoshTriplett
This is a massive improvement. I currently use .format(), sometimes with
__locals(), but f-strings would improve this massively.

Now if only they didn't require Python 3, so I could use them on the
production systems I'm working on...

~~~
ant6n
I switch back and forth between format and %, and never use __locals in the
format. It 's annoying, every time a string is written, to try to decide which
way is better for this instance. That said, is there a way to do ('%.3f' % x)
with this?

~~~
JoshTriplett
f'{x:.3f}'

------
dalke
I am having some problems trying to understand the implementation. What would
the AST from evaluating "f'{a+1}'" look like? Will there be a special AST node
for f-strings, or will it be pre-structured into the AST?

If it's a special node, is it the responsibility of the byte code generator to
parse the string? My belief is that it's part of the parser's job, so the AST
will never contain an f-string.

What does a syntax error report look like? Or traceback? Will it be able to
narrow down the part of the string which causes a problem?

Can f-strings include f-strings, like:

    
    
        f"{a + (f' and {b+1}')}"
    

I assume the answer is 'yes, and you shouldn't do that', which I can accept.

Support for arbitrary expressions inside of an f-string means that the
following is allowed,

    
    
        def f(a, b):
            return f"{yield a} = {b}"
    
        print(f(1, 2))
    

and will work, and will print _something_ , but it won't be "1 = 2". Nor will
any but heavy-weight analysis tools be able to figure out that this 'f' is a
generator.

I am less happy accepting that a magic string can turn a function into a
generator. Take for example this code from around line 438 of
[https://searchcode.com/codesearch/view/18830026/](https://searchcode.com/codesearch/view/18830026/)
:

    
    
                    if attr=='yields' :
                        yield_unit = self._grab_attr_(obj,'yield_unit')
                        if yield_unit:
                            ret = '%s %s'%(ret,yield_unit) # FIXME: i18n?
                    return ret
    

The penultimate line could be rewritten, validly, as:

    
    
                            ret = f'{ret} {yield_unit}' # FIXME: i18n?
    

The introduction of a typo, from 'yield_unit' to 'yield unit', would
drastically change the function, and be very hard to spot.

    
    
                            ret = f'{ret} {yield unit}' # FIXME: i18n?
    

Yes, Don't Do That, but we know that people use things like syntax
highlighters to help understand the code and identify mistakes like this.

EDIT: the PEP says that the expression are "parsed with the equivalent of
ast.parse(expression, '<fstring>', 'eval')". That means that 'yield' is not
allowed.

~~~
TazeTSchnitzel
Presumably it'd be handled similarly to how PHP handles {} and $ in strings.
As soon as possible, you swap "{foo} {bar}" for (foo.__format__() + " " \+
bar.__format__())

~~~
dalke
It's not so simple. The full implementation has to do something like insert an
AST into the right place, because {foo} can be an expression like
f"{__import__('math').cos(len(s))}".

The Python tokenizer will pass an f-string to the AST builder, which has to
pass the string to another tokenizer to generate the new AST. Because
f-strings can contain f-strings, this process is recursive. The end result is
a new AST that replaces the original f-string.

------
erikb
One of those things when you read it you wonder why it wasn't done that way in
the first place.

~~~
imakesnowflakes
Exactly when you need to apply this principle...

[https://en.wikipedia.org/wiki/Wikipedia:Chesterton's_fence](https://en.wikipedia.org/wiki/Wikipedia:Chesterton's_fence)

>In the matter of reforming things, as distinct from deforming them, there is
one plain and simple principle; a principle which will probably be called a
paradox. There exists in such a case a certain institution or law; let us say,
for the sake of simplicity, a fence or gate erected across a road. The more
modern type of reformer goes gaily up to it and says, “I don’t see the use of
this; let us clear it away.” To which the more intelligent type of reformer
will do well to answer: “If you don’t see the use of it, I certainly won’t let
you clear it away. Go away and think. Then, when you can come back and tell me
that you do see the use of it, I may allow you to destroy it

~~~
erikb
Well, now you talk about that the reform is wrong (the suggested one or the
previous ones?) but I said that I agree with it. So it feels like between the
post you are responding to and the Fence there are some arguments missing,
right?

------
jonathaneunice
For those that want something close today, there's
[https://pypi.python.org/pypi/say](https://pypi.python.org/pypi/say)

    
    
        fmt("Hello, {name}! You have {len(msgs)} waiting.")
    

Interpolates local variables and expressions. It uses the format method, and
has all of format's output formatting.

~~~
RubyPinch
say looks nice, but it still suffers from exactly what this pep is trying to
fix

    
    
        >>> def test(x=1):return lambda:say.say("{x}")
        >>> test()
        NameError: name 'x' is not defined
    

[https://github.com/syrusakbary/interpy](https://github.com/syrusakbary/interpy)
seems like a closer solution, could be modified to also support format specs

~~~
jonathaneunice
I think you're right that Interpy and PEP 498 have a different, earlier
binding strategy than say does. I've found the late / local binding of say()
and fmt() convenient. Are there use cases where that earlier binding is
valuable or critical?

Your example highlights the binding strategy, but more typical would be:

    
    
        >>> def test(x=1):
        ...     return fmt("{x}")
        ...
        >>> test()
        '1'
        >>> test(12)
        '12'
        >>> test('woobers')
        'woobers'

------
voyou
There's an interesting competing PEP which allows for the way in which the
expressions are interpolated into the string to be customized:
[https://www.python.org/dev/peps/pep-0501/](https://www.python.org/dev/peps/pep-0501/)

~~~
ant6n
I couldn't get myself to read beyond this example

    
    
      i"Substitute $names and ${expressions} at runtime"
    

It looks like bash.

~~~
adrusi
Is it bad to look like bash? Bash really is a beautiful language, and string
interpolation is one of the things that it's designed for.

~~~
ant6n
Well I don't like Bash, but I guess that's a matter of taste.

But I do think most people would agree that Python should look like Python.

------
ceronman
C# 6.0 is also adding literal string formatting:
[http://www.codeproject.com/Articles/846566/What-s-new-in-
Csh...](http://www.codeproject.com/Articles/846566/What-s-new-in-Csharp-
String-Interpolation)

------
stinos
I hope this gets implemented. A year ago or so we had several customers
wanting to custom format output filenames and directories in a desktop
application and we settled for something which is almost exactly this, so the
{identifier:format spec} idea, and ever since implementing it I wish any
language had it as we found it really convenient and having no apparant
disadvantages in comparision with printf-%-style in C or Python/streams in
C++/{}-style in C#/Python

~~~
chrismorgan
I don’t understand what you’re seeking. This is purely for string _literals_.
If you’re taking a user input string like `"foo-{date}.{extension}"`, you
could use `string.format(date=…, extension=…)`

~~~
stinos
Yes it's for literals only, just saying I like the inline style and
readability and succinctness of it and wouldn't mind it being implemented in
Python and other languages.

~~~
dalke
But this wouldn't solve your case because it could only be applied to strings
that are in the source code. Your users don't have the ability to change those
string. Instead, they want to provide a format string, which you fill in. That
is already available, eg, via str.format(kwargs), where you control the kwargs
and the user controls the str.

~~~
stinos
Yes, I know that as well. But this wasn't a Python application :] Maybe should
have mentioned that earlier..

------
RodericDay
_However, str.format() is not without its issues. Chief among them is its
verbosity. For example, the text 'value' is repeated here:_

    
    
        >>> value = 4 * 20
        >>> 'The value is {value}.'.format(value=value)
        'The value is 80.'
    

_Even in its simplest form, there is a bit of boilerplate, and the value that
's inserted into the placeholder is sometimes far removed from where the
placeholder is situated:_

    
    
        >>> 'The value is {}.'.format(value)
        'The value is 80.'
    

_With an f-string, this becomes:_

    
    
        >>> f'The value is {value}.'
        'The value is 80.'
    

Yeah I've had this thought before.

~~~
Retra

        >>> 'The value is '+value+'.'
        'The value is 80.'
    

Boilerplate? It's one extra character.

------
publicfig
In regards to the title, PEP stands for "Python Enhancement Proposal", not
"Python Extension Proposal"

------
deniska
You can even hack it into a string class if you don't mind using even more
scary hacks like monkey patching built in classes.

    
    
        def I(s):
            import inspect
            frame = inspect.currentframe()
            caller_locals = frame.f_back.f_locals
            return s.format(**caller_locals)
    
        def main():
            a = 12
            b = 10
            print I('A is {a} and B is {b}')
    
        if __name__ == '__main__':
            main()

------
kazinator
In TXR Lisp, from system prompt:

    
    
      # simple quasiliteral, denoted by backticks
      $ txr -p '(let ((x "Bob")) `Hello, @{x -10}`)'
      "Hello,        Bob"
    
      # word-quasiliteral (breaks into list on spaces)
      # denoted by hash-backtick:
      $ txr -p '#`@(+ 2 2) @(+ 1 2) a b c`'
      ("4" "3" "a" "b" "c")
    
      # op-argument references in quasiliteral
      # ret produces a function whose arguments depend
      # on the uses of @1, @2. ... and @rest in the
      # expression. The value of the expression is returned.
      # These references can emanate from a quasistring:
    
      $ txr -p "(mapcar (ret \`@1--@2--@rest\`) '(1 2 3) '(a b c) '(x y z)))"
      ("1--a--x" "2--b--y" "3--c--z")
    
      # Very basic indexing, slicing and field adjustment:
      $ txr -p '`foo @{(list 1 2 3) [0]} bar`'
      "foo 1 bar"
      $ txr -p '`foo @{(list 1 2 3) [2]} bar`'
      "foo 3 bar"
      $ txr -p '`foo @{(list 1 2 3) [1..:]} bar`'
      "foo 2 3 bar"
      $ txr -p '`foo @{(list 1 2 3) [1..:] 20} bar`'
      "foo 2 3                  bar"
      $ txr -p '`foo @{(list 1 2 3) [1..:] -20} bar`'
      "foo                  2 3 bar"
    

Interpolation into quasi-literals is very useful and expressive; I use it all
the time. Doing trivial things should look trivial in the code.

Oh, right: referencing splices and unquotes is possible from inside a
quasistring:

    
    
      $ txr -p '(let ((a 42) (b (range 1 5)))
                 ^(list `hey @,a @(list ,*b)`))'
      (list (sys:quasi "hey "
              @42 " " @(list 1 2 3 4 5)))
    
      $ txr -p '(eval (let ((a 42) (b (range 1 5)))
                        ^(list `hey @,a @(list ,*b)`)))'
      ("hey 42 1 2 3 4 5")
    

Safe to say, that one's not coming to a Python near you.

------
currywurst
This looks like template strings in ES6 [1], which is really well done ! I
look forward to using f-strings

[1]: [https://developer.mozilla.org/en-
US/docs/Web/JavaScript/Refe...](https://developer.mozilla.org/en-
US/docs/Web/JavaScript/Reference/template_strings)

------
TazeTSchnitzel
This is a very handy feature of PHP and would be useful in Python. I think
readability will be solved by syntax highlighting: expressions in an f-string
would be highlighted like normal expressions, rather than like string content.
This is what is already done for PHP.

------
brazzledazzle
I'm really happy to see this. I know it's petty, but this was the biggest
reason I decided to focus on learning the ins and outs of Ruby instead of
Python.

------
riffraff
I understand the need for expressions-in-literal.

I really don't understand why the unnecessary extra "!rsa" modifiers are a
good thing though.

~~~
TazeTSchnitzel
The fact they had to hack in a workaround so != works is a point against it.
And they acknowledge you can use repr()/str()/ascii() directly.

They want to keep it for str.format() compatibility, but I'm unconvinced. It
hurts readability, and is redundant (There should be one-- and preferably only
one --obvious way to do it.)

------
nxb
I just use Tornado templates for doing this. I made a wrapper to make it a
single function call. The syntax is very similar.

------
smegel
So long as they add a rule to PEP8 saying you can only use one of the string
formatting methods in a given source file...

~~~
deckiedan
Something like that would be very easy to add to a linter, even if not to the
official PEP8. (Although I totally agree, only use 1 method, whichever it
is...)

------
bechampion
i like it ..a lot!

