Hacker News new | past | comments | ask | show | jobs | submit login
Excel team considering Python as scripting language: asking for feedback
594 points by smortaz on Dec 14, 2017 | hide | past | favorite | 268 comments
Folks, Microsoft is officially considering providing Python support in Excel (finally). If you are interested in this, please visit their uservoice page and let them know what you think. Thank you!

https://excel.uservoice.com/forums/304921-excel-for-windows-desktop-application/suggestions/10549005-python-as-an-excel-scripting-language




Much as i would love for the power of Python in Excel it is important that whatever is done is consistent across the office experience. Some of us old enough to remember the multiple versions of VB-whatever across Excel, Word, Access and that in itself was a blow to productivity.

Yes they should choose Python, and in the process decide if it will be Python with a .Net library (standard and core as seperate libs please!) or IronPython. This in itself is an important first choice.

Then it has to be done in a mechanism that enables the exact same libs and user written python code to work in the same way across all the Office products.

Other languages have been suggested, all good choices with their own merits. Lua is a good chocie for compactness and speed. C# is lingua franca of commercial developers so would suit ISVs but i think too heavy for end user scripting. Nim or even freepascal maybe? Lastly whatabout just using VB.net consistently? VB is a great language for newbs and casual/adhoc programmers .... but it gets ignored because of problems with consistency of implementation by MS.

Last point i would like to make is IMHO the choice needs to be based on: - ability to transpile to javascript so that Excel365 can be scripted from webapps - install-free deployment; should be built into Excel in a way it can be used without any user install for dev or runtime - standard vanilla language, not a variant

Disclaimer: A huge fan and long time user of python here. I also spent largest part of working life in ISV world rather than end user land.


The reason Python is a popular choice is because it's a very polyvalent language (not specialized for any task but quite good at a lot of them) BUT the data analysis tookit in the Python ecosystem rocks (numpy, pandas, etc). Lua doesn't have a good data anaylisis story. C# is not made for scripting. VBA is not great outside of Office.

Now given that cPython 3.6 has f-string, and that you probably will use string formatting a lot if you script a MS product, I would implement this version specifically. It's the most recent stable version of the standard implementation anyway.

Actually no, I would not implement anything. I would embed the cPython 3.6 interpreter and stdlib, and just provide a binding to it. This limit the problem of bad / inconsistent implementations that crippled MS products in the past.

Then I would say "we guaranty to provide a 3.6 compatible cPython + stdlib for the next x years" so that people can be confident to write Python stuff. Otherwise, a bad implementation would be worst than no implementation at all.


The office 365 support could be implemented using the jupyter kernel protocols rather than transpiling to javascript. That would give them a consistent behavior across desktop and web and even open the door to support more languages in the future if they really wanted to.


> VB is a great language for newbs and casual/adhoc programmers

Is it, though? For example, VB puts four different ways of passing arguments right in your face: values by reference, values by value, object references by reference and objects references by value. Python only has the last one and doesn't make you think about it.

And without "Option Explicit" VB does something very unhelpful (also seen in PHP): misspelled variables are created on read and silently converted to the required type, giving you nonsense results. (With "Option Explicit" enabled it gets verbose and tedious instead.)


I'd say what Python does is passing object references by value.


Right. I listed them in the wrong order and wasn't quick enough with my edit. :)


C# Scripting is a thing now. I've used it on projects. C# makes an excellent scripting language, and in the MSFT space there's probably more of an established practice in that language. I can't imagine why they would use any other language - except perhaps F# or PowerShell.


Very well thought out answer and agree with the consistency part, don't target popularity, target consistency


Python?!?! "Significant whitespace" is an abomination from deepest layers of hell.


Dear Excel team,

Take a look at the top three results for "excel python" on bing.com for some great ideas on how to incorporate python into excel.

---

1.) https://www.python-excel.org

2.) https://www.pyxll.com/

3.) https://www.xlwings.org/


Thank you - all awesome tools. Kudos to the devs behind these bridges. We did one ourselves, but better ones have come along since then:

https://www.youtube.com/watch?v=Gu31s7kXPOg


There is also pyspread as a full spreadsheet implementation using and for Python: https://manns.github.io/pyspread/


> Pyspread expects Python expressions in its grid cells, which makes a spreadsheet specific language obsolete. Each cell returns a Python object that can be accessed from other cells. These objects can represent anything including lists or matrices.

Huh, on the one hand sounds like a such a simple way to deal with it that I can have trouble imagining a more elegant approach.

On the other hand, does it still respect the ways that Excel updates cells? Sicne that kind of requires immutability I think.


> https://www.python-excel.org/

The https link doesn't seem to work, try this:

http://www.python-excel.org/


Haha, I love the little jab at Microsoft's NIH syndrome by suggesting bing search results - seems though that by looking at Python they're slowly getting over that.


Also DataNitro


Clickable Link:

https://excel.uservoice.com/forums/304921-excel-for-windows-...

[disclaimer+bias: on Python team @ msft & would love to see this happen!]


Interestingly, the survey doesn't seem to consider education as a use-case of this - has your team considered that aspect? Spreadsheets are the most common way that school students are introduced to programming, and the only 'programming' that a lot of teachers (and others) do. There are heaps of reasons for this, but any even moderate programming (beyond very simple operations) in a spreadsheet sucks. Something like this (if it's easy out of the box) could revolutionise programming education (and data education for that matter).

Are there plans to get this working with the Excel web-app, or just the desktop version? I'm just having a daydream about Excel, Juypter notebooks, and OneNote Class notebooks...


That's a very good point! I hope you provided that input :).

Our team provides a free service for anyone to run Jupyter Notebooks on Azure. It's currently a big hit with the Edu crowd. We're seeing lots & lots of universities upload & teach courses on it. eg:

http://notebooks.azure.com/richie

There's also an Intro to Python notebook on the front page.

Hopefully the Excel team will consider Jupyter integration, esp for the web-app version! You're right that it's a killer combo. Check out xlwing's video on using Excel+Jupyter - it's brilliant.


I think we will see and are already seeing a move away from using spreadsheets in Education, there are simply much better solutions out there now. Jupyter notebooks being one which allow reading and writing to Excel files. The interface of these is far superior for educational purposes than a spreadsheet. I've already dropped Excel for this reason.


So, would this imply numpy and scipy support in Python for Excel? How about support for Excel worksheets within Jupyter notebooks?


That's for the Excel team to decide. My personal hope is that they'll offer as a base:

* Python 3

* Numpy, SciPy, Pandas, Matplotlib, Altair, ...

* Pythonic bindings for Excel APIs (sheets, etc etc)

* VSCode's Python Editor + Debugger built in

* Some sort of conda based env/pkg mgmt

* Right-click, "Open in Jupyter" (data/code)

* <your wish here>


Transparent access to Excel tables as Pandas Dataframes would be so, so very awesome.

I mean, MS implemented these really nice Tables in excel, but then left us no way to easily query them with something like SQL.


I agree, SQL in Excel would be awesome, Index Match everywhere is a right pain to maintain - to replicate foreign keys. Its amazing what you can do with Sumifs and Countifs, but SQl is better. VBA is OK but NO search across modules makes it such a pain just to find your code, I wish VBA had better support for UDFs so that they can appear the same to users, as built in functions with tooltips. MS have done a really bad job of incorporating Power Pivot, it’s confusing and doesn’t feel like a natural part of Excel. I don’t like Pivot Tables they are too Dynamic and a pain too format.


You can add-in IntelliSense for VBA functions using the (free) Excel-DNA IntelliSense extension. See this detailed write-up by Charles Williams: https://fastexcel.wordpress.com/2016/10/07/writing-efficient...


Someone posted a link to a really nice 3rd part addin on the reddit thread, on mobile so can't find the link unfortunately but was very well done. Shame on Microsoft for half-assing Excel for the last decade.


Possibly querystorm, in case anyone is looking for it. It did look interesting last time I looked at it, but I've never tried it.


That was it:

https://www.querystorm.com


Doesn’t pandas.read_excel provide this?


Well, there is PowerPivot...


Yeah, matplotlib, scipy and numpy are kinda no-brainers there.


Please not matplotlib ... are we really stuck with matplotlib forever ..?


If you've got better suggestions I'm all ears. I've found it pretty great for debugging and being able to understand data in a rapid manner.


ggplot2. Simply better in every way. Hell, even base R is better than matplotlib. Full disclosure: I hate matplotlib so very, very much.


No, watch them pump out some horribly supported python library .


So what's the strategy here re: security? I assume it'd be a fairly stripped-down version of Python ... or will you just design the UI to say "you probably shouldn't execute files from people you don't know" etc?


From a security point of view how is embedding python different from embedding Visual Basic? This is not my area of expertise so I’m genuinely curious.


Potentially different. .NET allows signed binaries / libs / a fully signed execution context, Python has nothing equivalent AFAIK.


Excel VBA is not .Net though. It's plain old VB code in text files inside the .xlsx, and you just have to trust it (or not).


Aaah. Then probably about the same, yea :) Maybe less of a nightmare around dependencies than the pip world...?


I’d rather full fat Python but heavily sandboxed with some form of firewall. “Do you want to allow 2018_Stock_Report_Q1.xlsp to connect to api.erp.corp.mycompany.com?”. Codesigning with my something under my corp CA, manage permissions via Group Policy and let users, on a per-endpoint or per-permission basis, authorise things like filesystem access, network connections, etc.


Challenges exist & will have to be looked at for sure. Incidentally, one of the guys on our team just a gave talk on his PEP for "Secure" Python:

https://www.python.org/dev/peps/pep-0551/

There's a YT talk as well.


> @ msft

Well, I guess that explains why you care about Office in 2017.


That and the fact that the vast majority of companies use Office...


Which is also why it is surprising that Microsoft has spent so little time on office in 20 years. Outside of changing the colors regularly, there has been very few new significant functionalities between Office 2003 and Office 2016. And opening the VBA IDE is a nasty reminder.

I'd argue large companies are still running Windows because of Office. The cost of retraining people, redesigning all of these user processes and converting all those documents would be massive. Whereas most new corporate applications in the last 5 years have been mostly web based.

So I am surprised Microsoft under-invest in what is they main strategic lock-in in the juicy enterprise market.


Between 2003 and 2016 they took some massive desktop applications and put versions of them on the web and mobile...


> very few new significant functionalities between Office 2003 and Office 2016

The first release to feature the Ribbon, arguably one of the all-time major changes in Office, was 2007. That took quite a bit for most users to get used to.

An Office suite is not where you add experimental features for the hell of it. People use it to get the job done in so many different scenarios, any change will significantly impact entire industries.

Office programs are the "lawyers" and "accountants" of the software world: their work has been more or less the same since they existed, and any major change to them is a basically societal upheaval, so their approach will always be naturally conservative.


Reshuffling buttons when they added ribbon is what I would classify as "changing the color", not really a new functionality.

There are many things lawyers, bankers, consultants and accountants would need that Office doesn't do. Linking a spreadsheet to a powerpoint document is a nightmare right now. Linking spreadsheets between them too. There should be a way to express a UDF as a spreadsheet so that people who can't code could create their own UDF. I love Apple's Number canvas approach, where a sheet is not a grid but a canvas on which you can add grids or charts, and they overflow with a scrollbar. Etc.

There are lots of new functionalities they could add that would make people's life better. Instead these products barely evolved in 20 years. Click the "fx" button in excel 2016 and you will get the same non resizable dialog box with a tiny listbox and a search box that doesn't search anything than in Office XP.


If you think about it, only 20% if employees can use office at a medium to high level anyways - a lot less people to retrain. The rest are "I click this button then do this thing"


Imagine all of those simple tasks that you could do in seconds with a python script. Export to csv in UTF-8 with double quoted fields. Loop through cells

  for row in rows:
     do some cool stuff
Pull data from a REST API using requests rather than some VBA hack. I'm getting over excited already. Edit data for your ERP and then post it back to the API and update the db... This is exactly what finance/data people want. Write custom functions (with numpy maths!)! No more nested If's. This will keep Excel as the premium spreadsheet app. I love it


I already find myself reaching for Google Sheets quite frequently just to do this. I'd love to be able to use Python instead of JavaScript.


Does Excel have a "plugin framework" for new languages the way Jupyter does? Ideally any language (Python/Julia/R/whatever) can be added by a third party without Microsoft's direct involvement.

The plugin authors would only need to map a language or library's DataFrame mechanics to Excel's table layout. (Microsoft could even use Apache Arrow to represent the data.)

I imagine there would be lots of possibilities by providing "hooks" for language designers to connect to.


You can pretty much already do what you want with VSTO or XLLs. However having a standard scripting language installed on everyone's machine is a all different thing. It means you can add some scripted logic in your spreadsheet, give it to someone else, and know it will run without having to install or maintain anything.


This is incredibly important. When I worked at BigFinCorp, we always used vanilla Excel (VBA and ODBC to RDBMSs). So our sheets would always "just work".

Python would be great in this role, but to avoid separate configuration, you'd need to ship the runtime with the spreadsheet, right?

Or would this new Excel capability be built on .NET? In that case, C# (for the enterprise folks) and F# (for the bleeding edgers) would be fun, too.

Bottom line, the Developer mode of Excel should support more than VBA.


In the Microsoft world scripting is often mediated through COM:

https://en.wikipedia.org/wiki/Component_Object_Model

that is how people do stuff with VBA and Powershell and you can do it in Python with

http://starship.python.net/crew/theller/comtypes/


> Does Excel have a "plugin framework" for new languages the way Jupyter does? Ideally any language (Python/Julia/R/whatever) can be added by a third party without Microsoft's direct involvement.

Could they use Microsoft's old Windows Script Host thing?


I'd vote for Lua: faster and less heavy than Python... Plus my biggest concern for Python would be how well the team can sandbox it. I know in this regard Lua also has advantages. Disclaimer: I am _way_ more proficient in Python than Lua, I just think Lua is a better choice.


I've recently been using Lua, and I love it. That said, I find python a far more appropriate language. Think about the use cases and the ecosystem around python.The quality, breadth, and depth of the Lua ecosystem for the finance/education/data users of excel will lead users to favor Python.

The main thing that might be nice is tables as a data structure for spreadsheets, but somehow I picture pain.


Does Lua have anything like Numpy or Scikit-learn?


Yes, it has Torch.


Pandas?


https://github.com/AlexMili/torch-dataframe


I was thinking the same. Would Excel be able to embed it like so many other projects do?


Might be worth reminding everyone of a dead Microsoft project called VSTA, which was basically a live .net IDE embedded in Office. Effectively a mini visual studio inside any office application, where you can write any .net language. This would have allowed people to script in VB.net, C# or F#. Microsoft killed the project before it shipped (but licensed VSTA as a scripting tool for third party apps for a while). It's a pitty. I think this would have been a better solution than python.


Good point that the tooling would've been awesome. However, I don't think .net would have spawned the rich ecosystem of Python pkgs that we enjoy today. And /that/ is a huge part of the whole Python value prop. From astronomy to finance to bioinformatics to ML & everything in between, there's a pkg for that. We even did sponsored a port of numpy+scipy to .Net (via Anaconda). At the end of the day, there were just too many corner cases & too many pkgs to port for it to make sense.


What is interesting is also the implications of them now considering python.

First if it is python, it is not javascript, which seemed to have been their previous favorite candidate until now (not sure if it really picked up).

Second, the writing on the wall for VB as a language starts to look like a time square billboard. They basically put VB.net in maintenance / compatibility mode like webforms and winforms. I think it is clear that a new scripting language would be meant to ultimately replace VBA (which has been on maintenance / compatibility mode for 20 years!).


excel already supports javascript for custom functions, add-ins, etc


I wasn't aware one could do UDF with javascript. Thought it was for addins only.


any interaction with excel using javascript clears the undo queue - even the action of reading data, or even not doing anything. Makes the system almost completely useless.


VBA's one saving grace is that it's been available on every company computer I've ever used. Alt+F11 and I have an IDE. It's probably responsible for getting a few people involved in programming.

But it had some serious limitations and some unexpected behavior compared to "real" programming languages. I hope that adding Python to Excel can be my new "guaranteed to be there" language on coworker's computers and get even more people involved in programming.


So when I google how to do something in Python I will have to comb through solutions specific to v2, v3 and now also Excel? ;)

Aside from that it sounds like a cool idea.


I've probably mentioned this before on another news article. Not like it really matters. Python is nice as a beginner scripting language since it's (to my own judgement) the most easily readable language to its function set.

When working with software like office, you are going to have a lot of users that don't exactly spend their days programming, or studying syntax. Excel is a little different in their own regard. The functions system has a lot of depth to it, and makes sense to a cell system, but for some it can be a but overwhelming in terms of how its written, since everything has to be written exactly as a function. I guess you also have generic office macros, haven't used them since high school.

In saying that, could this somehow be a reason why the office team have decided on a python interpreter, instead of say JavaScript or Microsoft's work on Typescript. Because python is easy to write, fast to learn, therefore users or even really employees in large data management positions don't have to worry about their time input?


I use PANDAS in Jupyter Notebooks a lot. Sometimes, exploring large data frames can get tedious and I find myself loading up excel to explore and browse around, usually when deciding how to normalize or clean things up. Using Excel as a PANDAS Gui this way would be really great, but also using Python to script excel would be useful. And I guess I would love to call bokeh / seaborn after making an excel cell selection.

I am looking forward to what a Python & Excel marriage will look like.


This post suddenly reminded me of Resolver Systems, who made a Python/.Net spreadsheet back in the '00s:

http://www.prweb.com/releases/spreadsheet/competition/prweb1...

I thought it was a good idea at the time, especially after having read "A Small Matter of Programming" which makes the case that spreadsheets are one of the very few successful end-user programming systems (the other being CAD):

https://mitpress.mit.edu/books/small-matter-programming


Port it as a plug in to old versions of excel as first class citizens on release.

There's been far too much, "here's a feature people will use so you're going to have to pay us yet again to open spreadsheets" in the history of excel. It's shit. It's awful. Don't do it. Using rich customers desires to extract money from those who neither need nor want the new feature is a crook's game. If microsoft aren't evil don't do it.


This would be great but it's a bit surprising given the move to add support for R in SQL server, Power BI, etc...


Yes I agree.


I made a prototype programming environment that is more like a spreadsheet interface into Python called Flowsheets that y'all might be interested in: https://www.youtube.com/watch?v=y1Ca5czOY7Q

Flowsheets has become my preferred data manipulation environment.


My biggest issue hasn't been VBA but the poor documentation of the Excel (Office) object model and the weird way of accessing cells and cell ranges.

Would this get better in Python? I hope so.


OK, I'll be honest. I usually avoid excel, except for excel to csv conversion. Then I write a python3 script starting with `import csv`.

This would make the product actually useful for me. VB is terrible. I don't even want to think about it. JS is a language I put up with when I have to work in a browser. Python is a joy to use especially in situations like this.

I can't tell you how many little apps I've had to write against CSV dumps from some stupid proprietary app without an API for Infosec or Capacity Planning or whatever that is essentially a report. I would do this instead in a heartbeat to avoid maintaining a service that gets used weekly/monthly/whatever.


I'm actually more concerned with good tools for working with code than the language, but Python would be an improvement over VBA, and conceptually allow them to use Python tooling they've developed for other uses in Office.


here's my feedback: embed sqlite in it and you're done. Nobody needs any more servers, you can literally replace a hundred thousand dollar "big data" center and team of PhD data scientists with a business school dropout. And you will get prettier, more actionable results.


Can everyone read this and vote this up in the survey? This is the real LPT.


This would be awesome! I've built some tools using VBA and found it to be the most painfully unpleasant programming language I've ever used. Python is by contrast the most pleasant!


No love for C#? Imagine the power of LINQ in Excel.


For C# and LINQ in Excel there is my ESharper add-in https://vlasovstudio.com/esharper/


This already exists. https://www.querystorm.com/


C# has become way to complicated for this kind of use case. In terms of feature bloat it's looking a lot like c++.


Unless the Excel team somehow integrates Python with the existing GUI design tool already found in Excel, it wouldn’t bring anything new to the table. There are already existing solutions such as DataNitro, which allow to code Python macros and functions and make use of Numpy, Pandas and Matplotlib. Microsoft is so far behind the curve it’s not funny. Their integration of either Python or IronPython in Visual Studio has been a mess for years.

See Http://www.datanitro.com


It would be great for a scripting language, but for field functions I would have preferred something that's more akin to FRP-oriented language (e.g. Scheme).


Excel itself is functional and reactive already, wouldn't adding an FRP language on top of that be a bit superfluous? Check out Felienne Hermans' work if you haven't already:

https://www.youtube.com/watch?v=0yKf8TrLUOw

http://www.felienne.com/publications


Excel is reactive already, but its field functions can't have variable bindings or procedural operation as far as I know (unless, of course, you write your own separate VBA function). I sometimes want to write in a complex one-liner and thought lisp/scheme would be a great choice.


Ah, fair enough. I mean, I'm not debating the power of Lisp, and it would be great first languages for someone not trained in programming. OTOH, there is of course the trade-off to make it as useful for as broad a selection of users as possible; Python has a lot more mind-share, and the SciPy ecosystem would be a great complement.

If Excel adds some practical convenience functions to the global namespace, would it be that hard to make Python fit the reactive paradigm in this context?

About those variables: maybe the Bumblebee tool she and her team made[0][1][2] would work for you

    Our language is an extension of regular Excel formulas, 
	 so you can write a transformation like
    
    A1+A2 <-> SUM(A1:A2)
    
    which allows BumbleBee to transform exactly the formula 
	 A1+A2 to SUM(A1:A2) or vice versa. 
	 
	 In addition to using exact cell references, you may also
	 use variables instead of cell references:
    
    {i,j} + {i,j+1} <-> SUM({i,j}:{i,j+1})
I hope it can be useful to you! :)

In the video I linked before she mentions it at 31m33s. Oh and here[3] are the slides for that video, in case you prefer those.

(Disclaimer: I don't use Excel myself; I'm just a programmer who wants to get more people into programming, so I was both happy and impressed when I saw her presentation on how people not trained as programmers use it to do really complex tasks)

[0] http://www.felienne.com/BumbleBee

[1] http://www.felienne.com/archives/2964

[2] https://figshare.com/articles/BumbleBee_A_Transformation_Env... Paper on BumbleBee

[3] http://gotocon.com/dl/goto-amsterdam-2016/slides/FelienneHer...


1) performance, including multi-sheet setups, should be better than the current Macros.

For maintainability reasons, in one personal experience, I moved in-cell macros to VBA, and saw significant degradation in performance (my excel was calculating financial instrument attributes, based on raw input data (store in separate workbook with the same sheet)).

So my ask would be that Python-based scripting would not just be the same as VBA in speed, but faster than in-cell macros.

2) There needs to be easy-to-use capability, to be able create XLS files with the new language macros, from within other programs. So that the new Python modules can be written out together with data from within other programming languages (and MS-supported libraries to replace or augment something like openpyxl, and similar libs for other langauages will be much welcomed)

3) I think Python as a choice for Excel is a good idea. It should be a language with 'duck typing', and that has support for Data Frames, Reactive programming. And in that light, I would like to see that Python's Pandas are first class citizens in being able to address Excel cells, participate in graphing, pivoting, etc.

4) Version control within the excel file... may be this is going beyond the original ask... but I think the spirit of introducing Python into excel, is around better SDLC/reusability. And so local and remote version control models should be supported.

5) I personally do not care if other MS Office tools (eg MS word) support Python. Would be nice.. but for my needs, this is not relevant.

6) a type-checking mode to be able to say, this Module will type check the data it uses... at run time.


An absolute no-brainer. Python is the new BASIC - very popular with people who have to program to do their work but aren't programmers. It's an excellent choice for a spreadsheet.


>aren't programmers

Also extremely popular with people who are programmers and need to do things.

Sorry, this is a frustration of mine lately. Python is a fantastic introductory language. It's also a fantastic general purpose language.

Newbies get turned off of it because it feels too easy, and they know that "real" programming is supposed to be hard.


Python is extremely permissive and flexible, even for a dynamic language - a trait that makes it easy to learn and easy to throw something together quickly. But when building software at scale, we need restrictions. For example: static types, immutability, pure functions, total functions, algebraic laws. It is restrictions such as these that give us the power to reason about code and the ability to compose it at scale. Beginner's do not understand how one can gain power from restriction. Skilled programmers do and most would not recommend Python for large-scale general purpose programming.


I know plenty of skilled programmers who would recommend Python for projects at any scale, and just as many who do it every day.

No true scottsman etc.


I disagree.

Python is great for scripting and for small projects, but I believe strong, static typing is an essential feature for large scale projects. I wouldn't want to use Python for anything that's predicted to end up with more than a couple thousand lines of code.


Last time I saw the Bank of America python codebase, it had ~6 million lines of code, was worked on by about 4,000 developers, and ran some core, performance critical functionality.

Python programming is an aesthetic that needs learning. Many of the worst written, and least maintainable python codebases I've seen are by programmers/teams coming from "proper" languages and don't think they have to learn how to write idiomatic python.


A bit dated now, but still a great read about a Python project done by Java developers: http://dirtsimple.org/2004/12/python-is-not-java.html

"So, the sad thing is that these poor folks worked much, much harder than they needed to, in order to produce much more code than they needed to write, that then performs much more slowly than the equivalent idiomatic Python would."


Yeah, I've seen that story played out so many times.

Best example I've seen was from a couple of Java devs transitioning to python, they re-wrote a system 3 times: 1st time: 35k loc 2nd time: 10k 3rd: 2k

The third rewrite was also significantly faster than the first two :)


> A bit dated now, but still a great read about a Python project done by Java developers: http://dirtsimple.org/2004/12/python-is-not-java.html

I included a link to this blog-post in the "letter of intent" (don't know the exact English term) that I sent to my potential employer just before my first interview for a professional (Python) programmer job, back in 2005. I got the job. Good times.


And it appears to be a recruiter goldmine, the staff turnover is significantly high (so I am told). A codebase of this size with no static type checking is not going to be fun.


Wait that "Quartz" monstrosity is still around?


Bzzzt - not really. Quartz in BoA, and Athena in JP Morgan (both built by the same folks) essentially takes Python, connects it to a bucketload of C++ and Java that makes up the bulk of the banks services, adds on a GUI layer, a pretty crappy object storage layer (shudders at Hydra...) and a half-baked object persistence layer that was always so slow. Sure, a lot of Python scripts get written for those platforms, but the heavy lifting -- pricing, trading, order books, risk systems, market data, connectivity all ends up being C++/Java, maybe in a Python overcoat.


> strong, static typing is an essential feature for large scale projects

This is empirically false.

There are (of course!) good reasons to consider static typing. But, in my experience, use of static typing has never been a first-order predictor of business or technology success. It's quite possible to build considerable value with, for example, a large Python 2.x code base.

A fun, and tangentially related, talk: https://www.destroyallsoftware.com/talks/ideology


That reminds me of Kapital [1] - a valuation and risk analysis system written in Smalltalk at JPMorgan, begun in the early '90s and as far as i know still going; 14,000 classes, 400,000 methods, hojillions of dollars of profit, twenty years in service, zero types:

http://www.esug.org/data/ESUG2004/ValueOfSmalltalk.pdf


> 14,000 classes..., zero types

In my view, classes are types. (Well, maybe I'm just spoiled by C++.)


In C++, classes do indeed define types. Templated classes define whole families of types. But in Smalltalk, classes do not define types.

I understand the word "type" to mean a property of a variable which restricts the range of values it can hold, and the set of methods which can be invoked on it. Smalltalk doesn't have any way to do either of those things, so it has no types.


But don't objects that belong to a class also effectively have similar restrictions imposed on them? (Otherwise, why have classes at all?)


It's true that a class with a set of methods defines a contract with its collaborators about what calls they can make (or what messages they can send, in Smalltalk terms), that is a lot like a type. Smalltalk calls this a "protocol". But they aren't enforced by the compiler; you can still send a message to an object that it won't be able to understand. What happens in that case is that a method called doesNotUnderstand gets called; a class can implement that to try to do something useful (you could implement a proxy this way, for example), but the default implementation throws an exception. I think that a genuine type would prevent the message being sent in the first place - the could would be rejected by the compiler, and would never get a chance to run.

Interestingly, it seems that this was planned for Smalltalk, but never implemented; a 1981 article about the design of Smalltalk [1] says:

"Also, message protocols have not been formalized. The organization provides for protocols, but it is currently only a matter of style for protocols to be consistent from one class to another. This can be remedied easily by providing proper protocol objects that can be consistently shared. This will then allow formal typing of variables by protocol without losing the advantages of polymorphism."

[1] https://www.cs.virginia.edu/~evans/cs655/readings/smalltalk....


All of it is possible, but specific features help in making better software cost less in time and money.

For example, nullability annotations reduce nil pointer exceptions.

Static typing removes the need to make certain typing unit tests, makes refactors easier to do in large code bases, makes it easier for compilers to generate faster code and so on. Think of them as compiler level linters.

Anyway, python has a static gradual typing mechanism. You should try it out :D


That's not "empirically false", it's just not empirically true. Static typing is a big boon to software development, and I use Python in my day job.


The OP said "I believe" so obviously we can neither empirically prove or disprove the statement. We can say with absolute certainty that static typing is not required for large codebases, because there are large codebases that are not statically typed.


Are there large codebases that are not statically typed? Yes? Then empirically it is false.

> big boon

But they argued ‘essential’. You’ve watered that down to nice-to-have.


Given that there are literally thousands of huge projects using Python at massive scale and hundreds of kLOC, it's clearly not essential.


"50 million cigarette smokers can't be wrong!"


An appeal to popularity is not really a logical fallacy if someone is arguing that something is (effectively) unpopular.


Depends if you define popular by most used or most liked.

Most used: everyone pays taxes. Paying taxes is popular!

A more practical example is javascript. I write some javascript, not because I like it (I hate it), but because that's the only way to make things happen in a browser. Javascript is popular. Does it mean it is liked / a good language?


If the statement is "there are no cigarette smokers", then "there are 50 million" is a reasonable counterpoint.


Code written in a statically typed language requires less testing. How is that not essential?


Code written in dynamically typed language requires less code, which often means less surface area for failure, and less tests.


> often means

No it does not, neither in theory nor in practice. Also, "less code" depends primarily on the language structure and not on whether the language is dynamically typed or not. (Haskell, for example, is more terse than Python.)


>No it does not, neither in theory nor in practice.

I've yet to come across something that was fewer loc in C++/java than in python.

That said, this is true even if I use pytype.


Most people writing VBA code for excel are only informal programmers. For example, I never took a CS course in my life but learned VBA and programming on the job. Since then I've learned python and other languages but when I'm doing a home project or something I always come back to python. It's just so easy. I was working on a macro the other day in Excel, it took me a couple hours to get it all working but I'm pretty confident that if I could've coded in python I could've banged it out in 20 minutes or less.

Edit: I suppose my point got a little lost. What I mean is that I highly doubt much Excel code is a "big project." It may feel that way when you crawl through some of the hideous VBA coding I've done but much of this is due to the inexperience of the coder and the realities of VBA. Give me python + 4 or 5 libraries and I could recode anything I've ever made in VBA in 1000 lines or less.


Unfortunately, there are several companies and hundreds of divisions of companies run entirely out of excel spreadsheets.


And integration with Python will be an enormous help for them.


Next up, get rid of Access


No one is proposing that we get rid of spreadsheets.

I understand, though: you hate VBA. And by extension, Access. But Excel, with or without Python, doesn't mean it can suddenly be used as a relational database. But you knew that.

So, what should replace Access?


Actually, a number of people are proposing this for certain applications. It is also reasonably likely to happen, at very least the spreadsheet is being challenged which will put pressure on spreadsheet vendors and most definitely cut into their marketshare. Adding Python support to Excel seems to be an attempt to resist this.


That's a fine personal opinion but I'll add my vote to Python being a wonderful choice for large programs.


>"Python is great for scripting and for small projects but ..."

You mean like Dropbox, Evernote, Ansible, OpenStack etc.


Or Google...


You would actually be hard pressed to find ANY major codebase that isn't using at least one dynamically typed language in at least some significant capacity.


Instagram would disagree with you. Seriously, Python is an absolute pleasure for managing large web projects and the lack of static typing has never been an issue for my company (spend the time writing tests instead!)


>(spend the time writing tests instead!)

Every sufficiently large test suite will contain an ad-hoc, informally-specified, bug-ridden, slow implementation of half of a proper type system.


Every sufficiently large statically-typed application will contain an ad-hoc, informally-specified, bug-ridden, verbose implementation of half of a decent dynamic language.

(I too enjoy a good pseudo-Greenspun)

More seriously something I've been pondering a lot recently watching the old pendulum swing back towards an enthusiasm for explicit typing is this:

* The advantages of static type systems are obvious and easy to articulate. * The disadvantages of static type systems are subtle and difficult to argue.

I started my career as a professional programmer when the pendulum was moving in the other direction. Essays by Paul Graham on Lisp and Python. The marvellous PJ Eby piece quoted above and Peter Norvig's "Design Patterns are artifacts of language flaws".

I just feel dynamic languages fit my brain better but maybe that's my own form of Stockholm Syndrome. Maybe I need to try a decent type system rather than the brain-damaged descendents of Java...


I think I've never really got the point of a good type system until I started using Elm and then wandered into the rest of the ML world, learning the so called "Type-Driven Development" method.

After some time doing that a Java project came up, so I grabbed Lombok, Vavr and started writing Java as if it was just another ML (immutability first, paying attention to side effects and so on) and the whole thing made sense. More sense than all those years of OOP teachings. The code was easy to debug, easy to reason about, easy to change. And it was Java. And that just stunned me for life.

Then of course, I started using TypeScript for React development and giggled like a little girl every time I had to refactor something, for I KNEW that it was very unlikely I'd have to stare at the debugger for long periods of time in a wild goose chase like I often had to with plain JS.

But the trick was to learn the way of doing things in the languages that really guide you towards that path.

I can definitely recommend that you try Elm if you're into frontend development, or something like F# if you want native. As far as docs go, the Elm guide and fsharpforfunandprofit.com are both great; the latter I can recommend regardless of your language choice for making typed functional programming make sense. I can also recommend the book Type-Driven Development with Idris, which has also been an invaluable resource to really understand that way of doing things.


What's the point in testing types if the tests that check it works pass?


No, the claim is that the tests that "check it works" are really testing the types (as an ad-hoc type checker) and wouldn't need to be written if a sufficiently strict static type system were used instead.


That's one thing I don't understand. The argument I hear against static typing is that it's too much work to write all this type metadata. But if you have to make up the lack of compiler checks with lots of tests for things that wouldn't require tests in a static language, we are not saving any work.


That's because you're not testing things that wouldn't require tests in a static language.

You're testing things that need testing either way, and incidentally also testing the types.


No, you're testing types and verifying the implementation details of the language. It's rare that a test for logic "incidentally" tests the type system. Usually both the logic and type checks are tested. It's just obfuscated because the bulk of the test is for checking the types and it's easy to look past that.


Yes, it doesn't test the "type system" (whatever that even means).

However, it tests values for correctness. Values have types. So if you are testing whether something has the correct value, you are also testing implicitly that it has the correct type, because for the values to match, the types must also match.


I love Python but I also don’t mind static types. Maybe I will realize some day what this argument is really about, but I expected that day to come by now.


Yes let's waste our companies time and money by writing tests for things the compiler could guarantee for free!


Not for free. You have to think and write types, maybe add some code to cast values between them or implement the same function twice for two different type signatures. Sometimes it still gains time, sometimes it doesn't.

Anyway, I doubt that a VBA replacement would need types. The use case is small scripts.


Python is strongly typed. You still have to think about the types. Except now you have to think about them every time you work with the code, not just the first time.


I work with Python and Ruby (some Elixir.) They are strongly typed in the same way. I hardly think about types. The code just works. The only scenario in which I have to stop and think is when I get some input, for example some JSON. Is that value I have to add to this counter a string or an integer? I can cast it to integer and that's it. To be fair, sometimes an integer gets where there should be a string and boom. Still, I prefer that to having to write types again as I used to when working with C and Java. I fix the code and I don't write tests to check the types of function arguments. Maybe I could accept some type inference, but no more string, int, generics, etc.


You have to think about the types for some small pieces of your code where those types matter. That's what hints are for. Python lets you, through type hints, only care about the typing in those small cases where it actually matters, and ignore them the rest of the time.


Most of it is caught as incidentals of tests you have to write either way.

There's room for both paradigms - it's kinda silly to argue strict superiority of either because there's just no empirical evidence that having static typing or not drastically changes bug count.

If you take a look at some of the studies out there that do exist (which there are, admittedly, few, and it's a fundamentally tough thing to measure), e.g. [0], it usually tends to be the case the both typed and untyped languages show up in the realm of "least likely to produce bugs"

[0] http://web.cs.ucdavis.edu/~filkov/papers/lang_github.pdf


I'd rather not write a bunch of tests that are really acting as a static type checker or, worse, testing the Python equivalent of compiler, linker and assembler output.

That's what the vast majority of testing is when it isn't simply testing mock code implementations.


Ah yes spend that valuable engineer time writing type validation tests a compiler could do instead of new features.

Put me with Op. I'll use Python for prototypes and small tools but get past that and I want a statically typed language. Not just for validation but also refactoring.


> but I believe strong, static typing is an essential feature for large scale projects

Python is strongly typed and has an available static typechecker, so, even if this is true, it doesn't rule out Python for this use case.


Sitting on 90K lines of Python here. It’s a breeze. Rarely see an error, and when I do, it’s from a third-party API failing to do its job properly (which is then successfully caught to avoid it causing problems). I’m interested in and use other stacks (mostly Elixir), but I don’t have any complaints about the language itself after 10 years.


This is to script spreadsheets, not build the next Netflix.

I would hate a typed language in there. Besides most static typing systems are ridiculously weak and introduce more headaches than they actually solve.


>This is to script spreadsheets, not build the next Netflix

With some of the spreadsheets I've seen, the next Netflix might just come out of it.

Just kidding, but only a bit. I've been at factories where if VBA stopped working, I literally don't know if we could have produced product that day.


> Besides most static typing systems are ridiculously weak

E.g. Go (and I still love Go, but it's pretty weak...)


Python is strongly typed, and Python 3 annotations with tools like MyPy brings static type checks.


Have you used MyPy? I'm currently looking to adopt it, but the feedback I've been getting is that it's a lot more painful to use than Typescript (which imho sets the gold standard of "optional typing").


I haven't used typescript, but I have used MyPy (Or, more accurately, pytype), and its absolutely a joy to work with. I've also used closure (the JS type system that isn't TypeScript) and I prefer pytype, fwiw.

My only complaint about Pytype is that there's no `Char` type at compile time (ie `for x in "a string"` -> Iterable[Char] instead of Iterable[str] during typechecking). But alas.


Unfortunately, MyPy doesn't work well with libraries or third party packages and treats external objects as `ANY` type. You can work around this with stubs but it's not fun writing type annotations for third party objects / functions. typeshed exists for the standard library and various popular third party libraries[1] to solve this very problem.

[1] https://github.com/python/typeshed


Python 3 supports optional type annotations that you can check with something like flake8.


You should have a little bit more experience with Python and you would see how huge systems can be written in it with no problem. It needs a bit more discipline, but above 100k lines software written in any other language would need the same discipline also. (I'm working with 100kloc codebase right now and seen almost 300kloc. That was messy because mostly juniors wrote it :D but still bearable and profitable.)


Try Cython, the benefits of both, IMO.


At last! hehe I was wondering why no-one mentioned Cython. Did I miss the part where everyone learned why it's not a good idea? For me it's the best of both C and Python worlds.

Write a Python program. Compile it as a Cython program. (Already faster, with no changes.) Add C types to the speed-critical parts. (Up to many 100s of times faster than Python)


People will tell you that instagram uses python or yelp or other big name projects use python. Certainly, type checking is not Essential to a large project the same way utensils are not essential when you eat. You can just use your hands to shove all the food into your mouth.


As an Indian, I'm not sure what I'm supposed to make of your analogy...


Perhaps you can infer that someone who thinks eating with their hands is uncivilized will have equally ignorant opinions about computing.


And what you are inferring is incorrect. I never said eating with your hands is uncivilized. You inferred it in your request for someone else to infer something.


Oh, stop backtracking. If I were you, I'd take the metaphor further and explain why eating with utensils is more hygienic. Type safety, food safety, ... You could write some flavorful prose (ha!).


Wait, was the underlying message not supposed to be "everyone should conform to my world view"?


That was not the underlying message. The underlying message is: I have my world view, I am offering it, you can agree, disagree or conform. The choice is yours. I would never force anyone to conform. Where in my post did I say that?


You latched onto the word "conform" when the word "should" was more important. You implied that type systems are better in the same way that utensils are better. If that wasn't your intention, your analogy was extremely confusing.


It's controversial. It could be good or bad, depending on your culture and opinions.


Well, there are rules to using utensils, and eating some types of food using them would seem uncivilized if not plain ridiculous. (Eating without hands, though, is surely uncivilized in the eyes of most people.)


I used to think that but I can't recall the last time the compiler saved me when augmenting or refactoring someone else's code - the IDE beats it to the punch every time - and static typing is not substitute for a good test suite.

On the flip side, i use interfaces and dependency injection constantly in java to work around static typing. In python, i write probably ~20% the amount of code because i never use interfaces, wiring logic or convert types.


Or at least a compiler to catch foot shooting (I used to agree 100% but elixir sort of changed my mind)


I bet thats why so many AI, Backends, ML, and IoT projects are written in Python

because it has no long term potential


I agree and I would add to that that many users of these scripting tools in Office are novice, and dynamic typing makes the language non self-discoverable. Static typing lets the IDE give a lot more feedback on invalid syntax, what can be done from there, etc. So I think this is a disservice we make them.


Saying dynamic languages are for novices hints more at you being one than anything else.

Not everyone likes IDEs (Emacs and VIM are still by far superior to many) and not everyone wants to deal with all the extra code and boilerplate and ad-hoc data classes that comes with static typing, to name a few.

Dynamic languages have faster iteration times and from experience that can yield higher quality software. They're easier to fit in the functional paradigm, better to model data transformations, and a bunch of other goodies.

You can't judge something without taking into account the context in which its used. And for scripting something like Excel dynamic is clearly superior.


I am not saying dynamic languages are for novice. I am saying people who will be using office's scripting are more often than not programming novices (like they are with VBA).

You may like plain text editor but for someone who doesn't know how to program, typing a variable then dot, and having a drop down of what is available from there, with an embedded documentation and direct IDE feedback on what is correct or incorrect syntax immediately after typing every character is super useful. RTFM isn't novice friendly.


Sorry I misread your first comment.

Its still possible for dynamic languages to have auto-completion. There's way more information available at runtime than at compile-time.

Besides, IDEs tend to have the entire world in most autocompletions, which is not useful either.

There would be no IDE here, you'll probably still write code from within Excel and advanced users will use separate source files to leverage their editor of choice.

The novices you mention will not want to leave Excel. A dynamic runtime with reflection is all you need to give a friendly experience. That doesn't prevent type hints, inference or autocompletion.


It's actually surprising the Excel team isn't considering TypeScript. TS is a modern language that they control, gives them an out for eventual browser compatibility, and they've already built great tooling (better than the visual studio tooling for python)


Wow!! Sheetjs looks cool! What kinds of math libraries do you support inlieu of Scipy /numpy. I am looking to mostly do some Exploratory Data Analysis


> eventual browser compatibility

You'd be hardpressed to reinvent excel in a browser. Sheets is a big, advanced web app but it doesn't even come close.


Yup, it teaches algorithms very effectively with low boilerplate; personally it's my favorite way of expressing dictionaries/hashtables and other data structures. Generators and iterators are super powerful concepts that I didn't learn in college and wish I did earlier.

It also allows a developer to manipulate individual bits, which is pretty amazing for security research or other cases when you want to get low level but don't want to get in the weeds with C.


It's also a fantastic general purpose language.

Why do you find it so fantastic? What stands out for you?

Python was my first real programming language, but I don't see anything very attractive about it now - save the libraries.


Honestly I can't imagine having to use anything else now. I look at every other language and it seems like a giant step back, syntactically.


Although, certain consumers (the financial industry comes to mind) might have so many massive, old, linked, and undocumented VBA-enabled spreadsheets that they would probably prefer VBA to at least be supported on a legacy basis for ever and ever.


Converting and updating those is going to generate a lot of contractor opportunities.


Javascript would be another example of language that many people has been exposed to.

I would even say it would fit better, not because the language is better, but because indentation wouldn't go in the way for quick & dirty one-offs.


Glad to hear that the Excel team is finally listening to your requests on this one. It's been a long time coming. Too bad they weren't interested during the TC days. If it does happen, it would be great if you could get the Cloud Numerics effort rebooted as well. There might still be a few people from our old team at NERD capable of resurrecting it.


I would love this, but for a unique use case: we generate _a lot_ of workbooks using Python. Some of them have hundreds of thousands of automatically generated formulas. We also have to write tests for workbooks. If we could embed Python, which all of our developers know, into Excel, it would make testing our formulas a bliss, rather than the burden it is today.


Excel already has VBA, C add-ins, COM interop, VSTO, javascript addins, and random functionality that's only available by using functions left over from Excel 5.0 macros (before VBA). These all have different weird limitations.

What Excel absolutely does not need is for Microsoft to add another scripting language in a half-assed fashion and then get bored and stop again.


Is anyone using javascript to write excel macros?

That functionality has been out but a while right?

Do people use it? What do they use it for? What makes them step back and reach for pandas or salesforce instead?

For me, working across multiple excel files sucks right now. Excels handling of datatypes kind of sucks. And pretty spreadsheets that people like looking at and working with have a bunch of annotation in them so you end up extracting the sub tables you need anyway - why not into csv/dataframes.

Tableu makes a lot of this better. Why not copy them instead of giving us another language to cheer about then ignore.

Why not integrate power bi more tightly with Excel and ship it in the basic version of office, then let us script in there?


There are now plenty of Python modules for working with Excel and Excel format files. Why are they trying to compete with those? As a university teacher I have dropped the use of Excel from my classes. I'm really not convinced this would convince me to go back. The philosophy of the product is just wrong. The should probably consider retiring Excel itself while maintaining support for Excel file formats. If they were to get behind some of the existing Python module initiatives or do something similar this would not be bad either, but using Excel as a front-end to Python is just a bad idea.


>Excel as a front-end to Python is just a bad idea

I think a lot of people would like this because you can look at more data on your screen at once with excel compared to jupyter notebooks. What's the downside?


The downside is

i) reproducibility. In the academic/research world there is a big move towards reproducible research, the nature of the Excel interface is not conducive to that. A standard scripting language is fine that plays with Excel but you don't want Excel necessarily as the main access point and the menu driven approach isn't conducive to keeping a permanent record of what buttons you click on. ii) In multi-cultural settings Excel defaults to the users system language, these days you simply jsut work with students laptops rather than using a standardized lab, so unlss you are polyglot working with Excel is just a really bad idea. Jupyter provides a standard interface and the default is always English, but supports multilingual content. iii) Python modules provide a huge range of solutions for many different domain applications, it's hard to see how Excel as an interface would provide support for every one of these modules. What I mean is how would Excel work as an I/O interface to a Python script that was doing something other than working with data? Notebook solutions already do this quite well.


Also for visualizing there are things like pandastable and DataExplore and other tools. You won't fix Excel by adding Python to it, but you do need the ability to process legacy xls and xlsx files so they need to at least continue that support.


Just do it! We 've been waiting for decades for a replacement of visual basic.


Aim for a front end notebook interface like Jupyter or Mathematica. Most excel i've had to play with was to store/manipulate data _and then_ C&P charts or data to word for the report. Seriously, notebooks.


You can basically use Jupyter as a frontend to Excel now anyway and never need to open Excel except to read and write to Excel files that you are sharing with Excel users.


that's cool - hadn't realised. i still dream about a word-excel mashup. imagine a table in word that when you click on it revolves around to reveal a spreadsheet and you hook the 'other side' to the calculation cells. literally separate data and text/analysis but packaged as a unit.


As well as agreeing to this, I would add the following:

* I would support both Python 2 and Python 3. Yes I know that the push to Python 3 is ongoing but there are a lot more libraries to be found for 2. Don't bother with IronPython.

* I would integrate pip or whatever package manager Python uses these days so that people can download their own modules and use them in the Excel sheets. Not only that, the packages should be restored automatically when the Excel sheet is loaded.

* It goes without saying that Python should be sandboxed.

* VS Code integration in Excel with full syntax highlighting and autocomplete - again a no-brainer.


> I would support both Python 2 and Python 3.

Nah. They can spare a lot of pain and confusion by not having two different Pythons with different sets of libraries in Excel, and they have this opportunity because they don't have to support people's old Python 2 spreadsheets. There aren't old Python 2 spreadsheets.

The libraries that are stuck on Python 2 really don't seem relevant here. There aren't many of them anymore [1], and I don't think people will be trying to run graphite-web or some busted old OpenID provider in their spreadsheet.

(I originally used Fabric and Twisted as examples in the above, but hey, those are on Python 3 now. Also not things you'd be likely to use in a spreadsheet.)

[1] http://py3readiness.org/


Most of the big data science libraries are sunsetting support by 2020, so I don't think supporting Python 2 is a great idea. It used to be true that there were lots of libs stuck on Python 2, but I'm struggling to think of any major ones that'd be relevant to Excel that haven't moved forwards nowadays. No way it'd be worth the confusion of having two different versions of Python and the documentation headache that'd be. There's a case to be made for keeping Python 2 around for stuff like internal software, but you're not gonna be running that in Excel.

http://www.python3statement.org/ <- look at the list of projects dropping support... NumPy, MPL, Pandas, SymPy etc.


> there are a lot more libraries

Not to get into a huge python debate. But this statement is just blatantly false.[0] Unless you are in a niche domain, the library you need either supports Python 3 or (increasingly, esp. among newer packages) is 3-only.

[0] http://py3readiness.org


How do you sandbox Python when pip can install any arbitrary package?


They should never do this because many people spent lots of time learning existing Excel scripting and they won't like it. Fortunately the user feedback will likely be very negative, so will stop this is it's tracks./s

Henry Ford: "If I asked customers what they wanted, they would have said faster horses"

Steve Jobs: "A lot of times, people don't know what they want until you show it to them"

Seriously. Just build it and and make it great. When you can show that you can do amazing things with it, release it. The downvoters will also then adopt it, if you made it great.


As long as the range object works in the same way, I don't care. Would be nice to have proper objects instead of having to use interfaces. It would also make a nice sandbox for beginners to learn python.


Please God no! I'm all for a scripting language but the poor quality and inconsistencies of Python makes my life so much harder in the 5 other applications where it is the ONLY way to script everything. The fact that it is whitespace sensitive I have found makes it harder for beginners who try to edit pre-existing files that mix tabs and spaces. That's one dumb thing but there are many other thongs that bother me about Python. Yes to a scripting language, please no to Python.


I know it's heretical amongst the Finance types... but one of the few reasons to use Sheets over Excel is for its scripting abilities. This will remove that advantage.


Alt+F11... Script away.


Anything you think you can solve with Excel plus Python is much better done in just Python.

This is like adding a tumor to Python.

Operable, luckily.


I hate to say it but why not ECMA Script (JS)? It’s everywhere and everyone knows enough to get by...


JavaScript is the obvious call.

App Script for Google Docs/Sheets is very powerful and one of my favorite features:

https://developers.google.com/apps-script/guides/sheets

It makes for an excellent API runner. You massage a data set and then run it against an API very easily.


I am all for more ways to hook into Excel since it is integral to many things I do for my job. There are some F# ways to play with Excel. I just wish MS would give more love to F#.

  [1]  http://fsharp.org/guides/data-science/#excel-interop


python isn't the bestest thing in the world but it's still a massive win for them, and a massive win for excel users. VBA had too many shortcomings and led to loads of scalability problems even for medium size apps.


To its defense VBA litterally hasn't been updated in 20 years. It's hard to hold a 90s language to the 2010s standards.


Fair enough. Although the jury is still out about it's value in the 90s too :p


Am already excited for this and would use it to the fullest.

But i think Microsoft in doing this should come up with a way we can plug any other language(julia,lua,...). Getting this right now would mean a lot.


Supporting R or Typescript/JavaScript would be great and probably closer to the Microsoft ethos (R is already very well supported in SQL server and PowerBI).


honestly don't know how they are going to integrate python for me to think about how it could affect my workflow.

it could be entirely possible that you could create both VB modules and Python modules in alt-f11 and functions and subs from either could be called from the spreadsheet. this would be great. if python had to be run in a separate app, not so great.

will python in excel rely on the environment(s) installed independently in the system or the excel python environment? if the former, how to standardize runtime across different machines?

support for the scipy libs and matplotlib would be nice. maybe matplotlib will generate plots in its own window - 6/10 if so. if matplotlib can generate plots as its own object within excel sheets - 7/10. if the plot is reactive to changes along its dependency chain - 9/10. if you can right-click format shape and adjust colors, lines, scales, axes - 10/10

support for python exception handling - well... ANY exception handling besides "ON" ERROR GO TO" / "RESUME NEXT" - 11/10

python with alt-f11's immediate and locals window + breakpoints + steps - 10/10.

what about windows api? office interop? internally built xlls? guessing these will remain with VB and will remain compatible. guessing excel/office object models will also remain the same structure in python for excel.

don't really care about output into jupyter. excel has "cells" too so you could technically output into a protected sheet if you wanted a read only report. we used to generate publications out of excel - chuck in some data into the sheets, matlab takes over the computation and spits it back into excel and a full report with cover sheet / table of contents / page numbers etc will be generated and we'd print and bind it professionally and send it back to the client.

just a philosophical musing... excel is very "functional". from each cell/cse-array in your raw data to each cell/cse-array in final output there really ought to be a single chain of functions. i see a few comments here and on reddit where people are looking forward to happily blasting for loops on each cell in book-wide subroutines. that breaks the "functional"-like quality of excel. that makes excel a million times harder to audit. excel at its best should look a lot like f#.


I am 90% certain that MS has abandoned its EEE strategy from the nineties.

But the scars from that era are so vivid in my mind, that I feel the need of a confirmation.


What do you want confirmation of?

That MS is now a gentle, caring beast? Large organizations are not like that, they are money making machines that adopt strategies to maximise profit. EEE is no longer such a strategy, though it worked very well for a long time.

Will MS ever go EEE again? In a heartbeat if they felt it would maximise profits.

Does that seem likely in the short to medium term? No, the industry is different now, and they have to play nice.


Look at "R" programming language, or look at "JavaScript" (-> TypeScript).

They still do the EEE strategy. Fool me once...


This is a long time coming. There's a big backstory to it. Let me explain.

"Exie" was the codename for Microsoft's attempt to create, in Excel, a language similar to MATLAB (and later, Julia). The name "Exie" meant that it would be integrated with Excel. It was based on Alan Edelman's Interactive Supercomputing (ISC), which Microsoft acquired in late 2009 [0]. ISC's main product was Star-P, an auto-parallelizing compiler in which users used a *p suffix in an array dimension to indicate parallelism across that dimension, similar to CoArray Fortran.

Unfortunately, the ISC acquisition was a failure. Microsoft already had agreements with MathWorks and Cleve Moler (makers of MATLAB) which prohibited competitive software like Star-P. So Microsoft had to scramble and find other ways to get analytic, REPL, MATLAB-like software out the door. Unbeknownst to Microsoft, Julia was already making faster progress than they ever did, but on similar goals. Julia is what Microsoft had hoped Exie to become, and it's not a coincidence that Alan Edelman has been involved in Julia.

Then, Steve Ballmer decided to abandon HPC and mathematical software, and to focus mostly on the cloud, which led to the very public dismissal of Bob Muglia.

The head of Microsoft Technical Computing, Kyril Faenov, unfortunately committed suicide a year later. His wife, the daughter of a famous Seattle architect [1], did not respect his Jewish mother's wishes on the handling of his remains. His mother fought and lost in court [2].

Maybe Microsoft embracing Python in Excel is the best that can be hoped for now. It's certainly better than VB. But Exie tried, almost 10 years ago, to create an Excel environment closer to modern-day Julia than anything else.

There were a lot of personal sacrifices in that effort. For example, Bill Blake, the former CEO of ISC, stayed at MS for a while during the transition, then moved to Cray as CTO for a while, and then died unexpectedly less than 4 months after he joined D-Wave. [3] [4]

There's certainly been a lot of casualties, in the attempt to bring Excel up to being more analytically-friendly. Is it really worth it?

[0] https://blogs.technet.microsoft.com/windowsserver/2009/09/21...

[1] http://archive.seattleweekly.com/news/956673-129/seattleland...

[2] http://caselaw.findlaw.com/wa-court-of-appeals/1735359.html

[3] https://www.cray.com/blog/cray-ceo-reflects-on-bill-blakes-l...

[4] https://www.dwavesys.com/blog/2015/04/bill-blake


An additional note for posterity here...one of the main reasons Microsoft was interested in acquiring Interactive Supercomputing was that ISC developed it's own M-language (Matlab) compiler M# built on Mono/.NET...basically a faster Matlab replacement than Octave, using features like multiple dispatch. One of the primary developers of the M# compiler at ISC was Jeff Bezanson, co-creator of Julia.

Exie used a bit of code from M#. When Exie was cancelled, the libraries were given a C#/F# facelift as Cloud Numerics, which was eventually cancelled, and then given a GUI facelift as part of the initial version of AzureML.

Disclaimer: I previously worked at Interactive Supercomputing, was part of the team acquired into Technical Computing/HPC at Microsoft, and also worked at Julia Computing.


Julia is open source...why don't they just incorporate it?


I hope they wont localise the function names like they do for cell formulas. Never have i met someone who enjoyed this.


I do quite a bit of VBA, but not much other programming. Could you clarify what you mean by localized?

Just that the cell formulas cannot be (elegantly) accessed from VBA?


The formula names are translated into different languages so if you are used to using Excel in your native language you can't easily use it in English or vice versa.


I would prefer JavaScript as scripting language. Or make support all clr compatible languages


F# should be best is Excel macros language. You may provide option to run Python from F#.


yes please !


This will be >>> better than using JS as a scripting language in Google Sheets.


pro: better compatibility with external libraries, syntax is nicer

con: GUI programming will suck unless .net adds "visual python"

I don't have statistics but I imagine people would prefer sticking to VB for ease of adding buttons, etc.


I thought there was a project for typed Excel. Any news on that one?


Do it!!!!!! To stay relevant do it!

I mean any language would be better than VBA.


Native virtualenv and python inside windows would be great :p


I'd like to see perl integration, that would be rad.


You can already do this with xlrd and xlwt.


Excel spreadsheets are cool because if you update data somewhere, it'll update all the formulas and products elsewhere.

How do you get that kind of "watch" behavior with `xlrd` and `xlwt`? You don't, you have to re-run a script. Sucks.


Try XLWings


Right. XLWings looks like a cool tool.

But the point I'm making is that: It's still a good thing for Excel to be integrating more tightly with python. It'll help all of those non-developer normies get the magic of coding.


Why not Typescript or some languages with stronger type support? Auto-complete is much more useful with a typed language.


Yes please.


Why not make it work with Windows Script Host for any compatible language?


Applications are open for YC Summer 2023

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: